Recompilation

[Darkholme]

Doing a decompile of FFT Would be very cool, (decompile it to c/c++ and then recompile to see if it works) but even if the decompile works, the resulting code would be nigh unreadable, and would need to be interpreted (though uncommented c is probably more easily documented than ASM).

I've been experimenting with fftpatcher on my wotl since yesterday; but I'm in my last year of a software design degree.

I don't have alot of ASM experience, and I've only ever done it for a SPARC, but I'd be able to read the ASM (though it might be slow going).

Documented ASM? That's probably harder than documenting some reverse engineered code.

Has anyone considered a different possibility?

We have the battle mechanics guide: What about a new app that can read data from the FFT Files, and *emulates* tactics, based on them? You could write a separate PSP app that reads the datafiles. That app would be able to be ported to other platforms, as you'd just have to compile it for another platform. You'd need to figure out where some of the data is, and what it does, but you wouldn't have to have everything figured out. And much of it could be gleamed from what's already been figured out by other people.
Plus: If you have a custom engine to start from, you can throw together an editor to work with it much easier than hacking or hex editing, and other projects could be made more easily.

Some people have said "Why bother" about WotL stuff; but one reason is portability. I don't have the time to sit down and play videogames on a console or my PC anymore, but on a PSP I can play it on a bus or in the bathroom or whatever.

[MysticKnightFF5]

You have my C++! And my Java!!

[Darkholme]

If I wasn't clear in my last post;

While I'm not going to try to take charge of this project, I was trying to say I'm willing to lend an assist.

[Atma]

@Darkholme
a custom engine would be an awesome achievement.  all the limitations and problems that all the people on these forums have been working with could be eliminated.
one problem is that it's a lot of work, and who here is willing to take on such a huge undertaking.  u'd essentially be making a game but coding it in a way that can pretty much whatever u want it to be.
on the flip side, all the work that has already been done on these forums and especially the potential work being suggested in this topic is also a ton of work.  so, in the long run a new engine would probably be the better choice and the fact that u could also improve the quality and customization beyond what is already in place would be fantastic!  (hd sprites ^.^)

[Darkholme]

Yeah; I guess.

I was suggesting it be written for the original psx architecture/PSP (Though a PC port would not be so so difficult), and for much of it (Maps; Unit stats; Events; Cutscenes, and possibly even graphics) have the program use the data from the psx iso or the psp iso. He have editors and extractors, why not just use the data that's there? Extract it, and just build an engine that reads the data? I think it would be alot less work than re-coding the game from scratch - most of it could be done with the knowledge already available in this community.

The data need not stay in the same format if we want expandable files. Bioware gave the specifications out for a few binary files that function alot like xml files; but much much smaller. There are a few editors available on nwvault.com for them, and I believe a few of those are open source.

I'd suggest we stick to paletted sprites, though it wouldn't be hard to move the max from 16 colors to 32.

Though I guess so long as we stuck with c++ we could do it for PC and Port it later. (But keeping in mind the PSP/PSX Screen size when developing the app, and cleanly separating our reliance on libraries, so we can just sub out the libraries for PSP libraries later). - Personally I never game on my PC anymore, and do my gaming almost entirely on the PSP nowadays.

If we want to do this really pro; we could get together a large number of us and start a small on-the-side company, and release our own TBSRPG based on the custom engine (using the official playstation development kits - we could all pitch in and buy one with the software, maybe a couple, and our more prominent coders could have access to the official PSP/PS3 Libraries while we develop). - That would be awesome (though that would be a goal for quite a while from now, if we ever went for it).

Tangent: I come from a java/c#/vb.net background, and most people's c++ is so poorly arranged (disorganized) it makes my head hurt - and some of the base libraries are a nightmare to work with. For me it would be ideal to either be able to make this in c# in visual studio or Java, in Eclipse (but then it's more difficult to port to a console if we later want to). I should note, that C#, AND Eclipse, both support compiling to binary, though it could take some work to do it for PSP Architecture.

These are all possibilities. I guess if we're going to do any of these things we need to figure out some stuff first. (I'll doublepost to break it up)

One thing I have been wondering - Is there a tool to pull out the cinematic cutscenes from WoTL as computer files/has it been done already?

[Darkholme]

IF any of these things are to be done; we would need to figure out:

1. Interest: Which of these approaches do we want to take? Is there enough interest in any of them? (Main Decision + Teambuilding)
- Figure out the rest of the ASM for the PSP Version.
- Reverse Engineer it and recompile it.
- Custom Engine that could run FFT, or could run something new.

Who would be interested and what could they contribute? Who would be making the decisions, and who would be in charge?

2. Start figuring out the details of HOW we want to approach whichever of the three we decided on. Plan it out, and Plan out who is going to do what.

3. Start Writing it?

I've got some Software Project Management training, though I'm not sure I have the time to commit to this thing to make me managing everything be a fast process.

[formerdeathcorps]

All right.  Good to see someone else on-board.

One thing I have been wondering - Is there a tool to pull out the cinematic cutscenes from WoTL as computer files/has it been done already?

Yes, using UMDGen on the PSP ISO, the movie files, in .pmf format, are in PSP_GAME/USRDIR/movie.

I was suggesting it be written for the original psx architecture/PSP (Though a PC port would not be so so difficult), and for much of it (Maps; Unit stats; Events; Cutscenes, and possibly even graphics) have the program use the data from the psx iso or the psp iso. He have editors and extractors, why not just use the data that's there? Extract it, and just build an engine that reads the data? I think it would be alot less work than re-coding the game from scratch - most of it could be done with the knowledge already available in this community.

The data need not stay in the same format if we want expandable files. Bioware gave the specifications out for a few binary files that function alot like xml files; but much much smaller. There are a few editors available on nwvault.com for them, and I believe a few of those are open source.

Let me see if I understand your idea:
1) We run this on pSXfin or ePSXe.
2) We extract map/sound/event files and rewrite each file in some new file format and then compile to binary (preserving all non-mechanics data).
3) We rewrite WORLD.BIN, SCUS, and BATTLE.BIN (since those are the primary mechanics files...mostly the latter two) in some higher level programming language and recompile to binary, replacing the existing files.

Essentially, you're saying to do all the real coding work in a higher level language instead of ASM (without even bothering to look at the original) so later if we want to expand the game (by adding 10 new status effects or more unit variables) or port it to PC or another platform, we can more easily adapt that.

1. Interest: Which of these approaches do we want to take? Is there enough interest in any of them? (Main Decision + Teambuilding)
- Figure out the rest of the ASM for the PSP Version.
- Reverse Engineer it and recompile it.
- Custom Engine that could run FFT, or could run something new.

In any event, I don't see how we can avoid the first option.
1) From experience, we figured out roughly the AI priority on actions, as well as implied formulas on skillsets (which I confirmed when looking through the weapon formulas), some of the changes and hard-coding in the PSP version (like Rafa/Malak have a maximum hit of 10 on their skills), and the BMG by Aerostar certainly has a lot of other useful data (though stuff like "quick is a status" is wrong as we found out when we looked at the ASM).
2) Some things we don't know too well (the binary format of FFT's sound files), but we can work around on a custom engine (by using the equivalent MIDI/MP3 files).
3) However, there probably still exist features/version differences (between pSX/PSP or JP/USA) we never imagined existed.  FFT contains features (like Sound Novels and Debug Maps) that's not referenced or rarely referenced (knockback).  How do I know similar features aren't hiding elsewhere?
4) Furthermore, there exists features that we know must exist (AI, for example) that we don't know the explicit address to or layout of (the recent find by Pokeytax on terrain is one such thing).  Of course, when making an engine, this can be worked around, but the original layout may contain surprises that we don't know about that could easily affect mechanics.  Thus, our knowledge of such "hidden" features in either version needs to be made precise.  This will still require ASM deciphering of the main mechanics files.  Of course, once that occurs, I don't see why we can't do the other two options (or use that rather than copy/paste to insert PSP options into the PSX or vice-versa).

I know...this approach may take longer, but a lot of the PSP ROM's battle mechanics routines will likely be the same or very similar to the FFT ROM's, so the process is more of comparison than total decompilation.

Who would be interested and what could they contribute? Who would be making the decisions, and who would be in charge?

I can definitely work on ASM either in deciphering or reverse engineering (after I finish hacking formulas), but I'm not too great with higher level languages.  I'd be willing to help make decisions or assign roles, but I don't think one person should be monopolizing that role.

[Darkholme]

Essentially, you're saying to do all the real coding work in a higher level language instead of ASM (without even bothering to look at the original) so later if we want to expand the game (by adding 10 new status effects or more unit variables) or port it to PC or another platform, we can more easily adapt that.

It seems to be the more efficient way to do it; particularly with the Battle Mechanics Guide and the work that's been done here. I don't think we would need to know the exact process they follow to run each thing, so long as the end result is the same.

Let me see if I understand your idea:
1) We run this on pSXfin or ePSXe.
2) We extract map/sound/event files and rewrite each file in some new file format and then compile to binary (preserving all non-mechanics data).
3) We rewrite WORLD.BIN, SCUS, and BATTLE.BIN (since those are the primary mechanics files...mostly the latter two) in some higher level programming language and recompile to binary, replacing the existing files.

1) I was going to say we run it on the actual hardware, but yeah, an emulator could work. (Or we design it for PC, but intentionally design it to be fairly easy to Port to the PSX/PSP).
2) We extract the Map/Sound/Event files and rewrite each into an expandable format instead of one with fixed sizes; and maybe also battle.bin (that's where the jobs, characters, and abilities and whatnot are, isn't it?)
3)I don't know really what World.bin and SCUS do though - I'm guess the user interface and the maps and whatnot - and that's easy enough to redo; so yeah.

In any event, I don't see how we can avoid the first option.

Why? I think that between what's in the BMG and what's in the Gameshark Guide and what's in the FFTPatcher, We know most of what we'd need (and can make up the rest to fill in the gaps - just section off that area with a comment that that area is a best guess, and if we ever learn more about it we can update it;)

1) From experience, we figured out roughly the AI priority on actions, as well as implied formulas on skillsets (which I confirmed when looking through the weapon formulas), some of the changes and hard-coding in the PSP version (like Rafa/Malak have a maximum hit of 10 on their skills), and the BMG by Aerostar certainly has a lot of other useful data (though stuff like "quick is a status" is wrong as we found out when we looked at the ASM).

Right. I think we'd have to right a custom AI, but having a general idea how the original works would help us figure it out. The AI would probably be one of the hardest tasks. On the plus side, we could make the NPC fight a little smarter.

2) Some things we don't know too well (the binary format of FFT's sound files), but we can work around on a custom engine (by using the equivalent MIDI/MP3 files).

A Custom sound engine should be easy enough; we can find a sound library online to handle whatever we need.

3) However, there probably still exist features/version differences (between pSX/PSP or JP/USA) we never imagined existed.  FFT contains features (like Sound Novels and Debug Maps) that's not referenced or rarely referenced (knockback).  How do I know similar features aren't hiding elsewhere?

You don't necessarily need to. They're hiding. Again, I say we best guess it, and if we turn out to be slightly off, we adjust. I dont think the new engine needs to have all the half-finished, unimplemented aspects of FFT.

4) Furthermore, there exists features that we know must exist (AI, for example) that we don't know the explicit address to or layout of (the recent find by Pokeytax on terrain is one such thing).  Of course, when making an engine, this can be worked around, but the original layout may contain surprises that we don't know about that could easily affect mechanics.  Thus, our knowledge of such "hidden" features in either version needs to be made precise.  This will still require ASM deciphering of the main mechanics files.  Of course, once that occurs, I don't see why we can't do the other two options (or use that rather than copy/paste to insert PSP options into the PSX or vice-versa).

That's what I was suggesting. This would be the part that mainly gets *written* instead of interpreted from a file. If the original layout contains surprises, they wouldn't likely be in our ripped data or they just wouldn't get used (Examples being the various flags you see in FFTPatcher labelled unknown or left blank - the engine would save them, but ignore them, until we know what they're for).

I know...this approach may take longer, but a lot of the PSP ROM's battle mechanics routines will likely be the same or very similar to the FFT ROM's, so the process is more of comparison than total decompilation.

Oh; I guess. I was suggesting we use what we have, and update it with things we learn.

I can definitely work on ASM either in deciphering or reverse engineering (after I finish hacking formulas), but I'm not too great with higher level languages.  I'd be willing to help make decisions or assign roles, but I don't think one person should be monopolizing that role.

Well that works out for me. Frankly, I'm I'm not that great with Assembly, but I can plan out and execute a software project at a higher level.

Though those formulas would be really handy. If you can figure out the formulas for the attacks and whatnot, then we don't need to guess and worry about "how far off" the formulas are from what they're supposed to be.

A) Maybe the two of us can figure out what we have to make the priority for the data ripping. My first priority would be Jobs, Monsters, and Abilities (The same data you see in FFTPatcher).

B) Then I'll build a data ripping app to take what we need, and reencode it into XML/GFF. This way, if we figure out more, we can adjust the extractor, and re-rip the data with the new stuff we need. I think that would be the first step.

C) Then we'd need to start building an engine that simulates the FFT engine, using the ripped data. That's the harder part. Hopefully there would be other programmers to help with part C. - it's a much bigger task.

D) Then we write the AI, Some kindof world map module (with random encounter chances), and something to interpret the ENTD data. Again; other programmers required for any real speed.

[Pickle Girl Fanboy]

Remember that the battle map portions of FFT are 3D, while the character themselves are 2D sprites.  And getting a working demo up and running is more important at this stage than planning how you will implement job classes and ability data, though it's still good to think about that stuff.  If you can get a simple hardcoded sprite moving on a 2D chess grid, then you can start on the 3D engine, camera angles, and stuff like that.  You can softcode and break everything into modules once the Tactical RPG engine starts to attract more developers; FFT wasn't made by one person, and it's unreasonable to believe that you can do this alone.

I recommend you use a language that has automatic garbage collection and is well supported.  I'm learning Common Lisp - very slowly - but I can find only one open-source GUI toolset.  I hesitate to recommend it to anyone here, but I believe it is an excellent alternative to Java, provided you have the patience to learn about it.  If you don't, then any compiled language with automatic garbage collection should be preferred to Java - this engine can't afford to be slow or big.

http://www.google.com/#sclient=psy&hl=en&site=&source=hp&q=common+lisp+as+an+alternative+to+java&aq=f&aqi=&aql=&oq=&pbx=1&bav=on.2,or.r_gc.r_pw.&fp=ff3e2739446bc197

[Darkholme]

Very true;

I don't think I want to use LISP; You're right about java though, it's a tad slow if it's in bytecode. Though if you compile it down to machine code; I'm told it runs just as fast as any other application. And the compilers I've seen can start with bytecode instead of a full source. So if we packaged the free compiler, it could be set up to compile it down during install. I was actually considering using C# for the programming. It has a garbage collector, and it's got a fantastic SDK, and has a more structured and neat flow than c/c++ code. A library called mono for linux allows for C# programs in linux, which would make it portable.

Those are some fair criticisms. It would definitely need an engine to work off of, before it would attract more coders. Implementing the classes now would be starting in the wrong place. However, having a broad, general plan of implementation isn't a bad thing. Planning the project isn't "Making the damn game!" as it were, but some amount of planning saves alot of headaches later. I don't think I've gotten to the point where I'm overplanning. Just laying out some broad ideas, and figuring out what the goals will be.

This idea isn't even to the point where I'd start a thread for it yet though... I've contacted Kivutar (He's making Tethical, an online PvP FFT Demo Game). We don't quite want to do the *Same* project, but we're talking about resources we could share, or the possibility of combining the two projects, since they are similar.

Can you explain how 3d sprites work? I was under the impression they were regular sprites, with some code that says which sprite to show by camera angle.

~DH

[Pickle Girl Fanboy]

Saint Ajora, I'm sorry.  Typo, I meant to say that they're 2D sprites.  C# is probably your best bet.

There's nothing wrong with planning.

One last thing: when you get a google code page set up once you're far along in your project that you need one, please, please include *.zip or *.rar or whatever of all your source codes for every version you release.

http://code.google.com/p/bsnes/downloads/list
^Something like this.

[Pickle Girl Fanboy]

http://www.backerstreet.com/rec/rec.htm

Holy crap.

[Darkholme]

That's nifty. I've seen the old version.

Decompilers are cool.

[MysticKnightFF5]

I enjoy how I was ignored...lol...
well, anyways, I'll help in any way I can.

[Pickle Girl Fanboy]

I can compile and do a bit of testing on ubuntu, but my bullshit is free

[Atma]

don't feel bad, Mystic.  i got ur LotR reference.   
it even made giggle.

[heiir]

I really hope you get enough people to do this, that'd be so epic. WotL would be way better if you could use most of the programs on here.

[Gotwald]

For me, the only way to make WotL better would be to remove the slowdown... whoever figures out how to make it so the FPS doesn't half when spell effects are played have many WotL players in debt to them.

[Darkholme]

I enjoy how I was ignored...lol...
well, anyways, I'll help in any way I can.

Sorry about that.

Anyways; my idea is to work on Tethical with Kivutar, and try to design it in such a way as to be port-friendly as much as possible, so that if we decide to port it to another system we dont have to start over. His project is fairly small in scope, compared to what I suggested; but if we combine our effort; we can use his demo as the starting point for the rest of the *Bigger* project, with an already mostly functional game engine that we just need to expand on; and online multiplayer.

I'm dealing with some crap for end of the year finals; and then I'm going to start trying to help him out with more than ideas.

For me, the only way to make WotL better would be to remove the slowdown... whoever figures out how to make it so the FPS doesn't half when spell effects are played have many WotL players in debt to them.

That's a hack worth investigating, and it's certainly achievable. FUSA does it by accident when you connect your PSP to your TV; though it tends to run things a bit slower overall when connected that way; the spell slowdown doesn't happen. So SOMETHING FUSA does is stopping the spell lag.

[MysticKnightFF5]

Well...what does FUSA edit? Only one thing: the display. ....Which means it's probably a PSP problem, not a WotL problem....