SegaXtreme

Home	Forums	What's new	Resources

Translating Grandia 1.1.1

TrekkiesUnite118 - Sep 9, 2018

...

Next>

TrekkiesUnite118

Sep 9, 2018

So I thought I'd snoop around in Grandia's files and do some comparing and contrasting with the US and JP PS1 releases to see how different they are. The thought process was to see if it would be possible to extract the English script out of the PS1 version, and possibly use that as a base to translate the Saturn version. So far this is what I found:

The File Structure between the two games is actually rather similar, unlike that of Lunar Silver Star Story.
That said there are still some differences. Some files don't exist on the Saturn version and some don't exist on the PS1 version. However this seems to be more aimed at bin files and system specific formatted stuff like audio files, video files, fonts, etc.
I believe I've found the script for the English version. In both versions there's a Field directory. In this directory there are numerous files with hex values for their file names. In the Saturn data these are .MDT files, in the PS1 version these are .MDP files. I'm pretty sure these contain the games script.
The English PS1 script is completely uncompressed in the .MDP files. You can bring them up in a Hex Editor and read the ASCII encoded script.
The Japanese Saturn and Japanese PS1 versions of these files aren't as easy to read. So there may be some compression going on here, or it could just be that I don't have a Hex Editor that can properly decode it. I tried one that claimed to have Shift-JIS Support and I also tried various different UTF variants but none really seemed to reveal anything too useful.
The font for the Saturn version is in the FDATA.BIN directory as mentioned before by CyberWarriorX, and his compression and decompression tools that he wrote appear to work just fine. If you run them on FDATA.BIN you can get the 4BPP font and can load it up in a tile editor of your choice.
The FONT does have an all Caps English font in it, but from the looks of things I'd bet this one isn't a variable width font.
I've not been able to really find where the menu text lives. In the PS1 version I've found a couple pieces here and there. For example in the Battle directory there's some pieces in it's DAT files, but these files appear to be one of the few areas where the Saturn and PS1 versions are structured differently. Basically some files the PS1 has don't exist on the Saturn and vice versa.

So with all of that, I'd say this is what would need to be figured out to get the ball rolling on a translation:

Figure out how the Japanese versions of the game encode their script files.
Figure out where the Menu text lives in both the English and Japanese versions, and how it's encoded.

If that can be figured out, I'd imagine then it would be a matter of the usual translation hacking stuff like adding in support for a Variable Width Font, creating a new font, editing in the English Script, etc.

Obviously this isn't anything trivial and I'm not trying to make it sound that way. However I figured posting this could give people who know more about this than me a place to focus their attention if they're interested.

		Mr^Burns		Sep 9, 2018
			Good work, it's annoying how I can't find the original Korean patch files for Grandia as it might have given some help, must have been released on the net prepatched.

		Mr Conan		Sep 9, 2018
			You may find a lot out by diffing the korean translated version Torico hinted about in another post. That would at least show where the files that need to get changed are for the script and how they were altered for Korean. Thought i read somewhere that a few battle menu things were left untranslated, but it was close to pretty much done

		TrekkiesUnite118		Sep 9, 2018
			So I did take a small peak at the Korean translated version and this is what I've found: File sizes appear to be identical to the Japanese version. The font has definitely been changed as I'd expect it to be. The MDT files have been modified, which would seem to confirm that those contain the script. ITGET.BIN has been modified. Looks like they changed character names from Japanese to somewhat English versions. Which looking at the changes and gameplay footage, I'm guessing this is where the menu text lives. This is probably the equivalent of the ITEM.BIN file in the PS1 version. CMDMN.BIN has been changed. Again this looks like either Menu text, guessing command related based on what's in it and the name. CNFIG.BIN has also been changed. This again looks to be like menu text, guessing Config menu based on the name. SHOP.BIN has been changed. This is most likely Shop Menu text by the looks of things. SVLD.BIN has been changed. Again looks to be Menu text, I'm guessing this is for the Save/Load screen based on whats in it and the name. All the AMAP#.ADD files have been modified. My guess is this is text in the map screens or dungeons? WDATA.DAT has been altered, my guess is this is Weapons info maybe? SWN6M.DAT has been altered, no idea what this one is. That's what appears to be different so far in the FIELD Directory. I did some quick spot checking in the BATTLE Directory and nothing really stuck out as changed, but I didn't really dig that deep into it. I also noticed while looking at these and back to the PS1 version that some of these files seem to have been either reduced or eliminated with a new WINDT.BIN file being mentioned at the start of many of them. This file looks to contain item names and descriptions, but I can't really find an equivalent to it in the Saturn version just yet. My guess is that this is maybe a consolidation of some of the DAT files in the Saturn version or possibly a new file to reduce duplicate text across files?

vbt

Sep 9, 2018

		Torico said:

I tried all the Cyjzero blogs they are all dead (translation forum included), no way to get something with the wayback machine too.

TrekkiesUnite118

Sep 9, 2018

So after some more digging today I was able to get the PS1 English font. After looking at it I'm starting to wonder if the English version actually supports VWF. Reason being the English font is 8x16 in size, but every letter is scaled to that size and is perfectly centered. And when watching it in action you can actually see some spots where the spacing looks a bit odd, specifically around thin letters like Lower case L's and I's:

https://youtube.com/zoactLpR29A

And here are the fonts for each version of interest.

Original Japanese Saturn:

Hacked Korean Saturn Font:

English PS1 Font:

There were some 8x16 Japanese characters in the English PS1 font as well, but I didn't see a reason to include them.

TrekkiesUnite118

Sep 9, 2018

So more fun info, in comparing the script files between the Korean version and the Japanese version I think I've figured out some of the encoding. I still need to figure out the control codes for the text boxes, but the text itself is actually pretty simple. Basically the hex values for the text correspond to the sequence number of the font character in the font table. However the values are the decimal value of the font sequence number + 31, and then turned into Hex.

So for example looking at the font charts above, the character A is 11th in the font table. If you want to have A displayed, you need to put the value 2A in the MDT file. This is because 11 + 31 = 42, and 42 in Hex is 2A.

I was able to test this theory by replacing the text in the intro scene so in the first two text boxes Baal should now say:

	
	
		
		
			??

WOOT

And low and behold:

As I said, I still need to figure out the control codes, but if someone wants to take a look at it, the file for this area is BA38.MDT in the FIELD directory. The offset for this cutscene and where I started making changes is x5AA.

Now the odd thing here is there's over 1000 characters in the font if I'm counting correctly. But this encoding can at most go up to 255 if I'm understanding it correctly. So I'm not sure how it's dealing with characters beyond that limit. However for an English hack we might not even need to care about that as we wouldn't need over 1000 characters to begin with.

So honestly, once the control codes are figured out I think this might actually be a pretty straight forward hack. The biggest issue to do first would be to change the game to operate on an 8x16 font instead of a 16x16 font so the English PS1 font can be used. After that it would just be injecting the new font and then editing the script files.

This is probably what they did for the Korean translation, but without the font size change, as Korean is still a Fixed Width Font so they could stick with the 16x16 font size easily.

		SaturnTeam		Sep 10, 2018
			Keep up the great work! Many years ago, I would translate the game as I went along. Unfortunately, It got to be too time consuming.

		Mr Conan		Sep 10, 2018
			Another major issue in translating this is the fact that it looks like the korean version got away without expanding the script lines (at least from the glances i took). Lunar has codes that start and terminate text that are sometimes followed by codes that jump around by some offset in the file. Grandia is probably similar, so unless you like cavespeak, someone will need to figure out how to properly resize the script files.

TrekkiesUnite118

Sep 10, 2018

So doing some more digging I've found a couple more nuggets of info:

The Japanese PS1 and Japanese Saturn version use the same encoding for their text.
The English PS1, Japanese PS1, and Saturn versions seem to use the same control codes with some catches
The PS1 versions seem to add additional values in the codes between text lines/new boxes.
Some codes are also used differently.

So with the control codes, I've figured this out looking at the file BA38.MDP/BA38.MDT in the different versions. This is the file that contains the intro scene I edited earlier among other things. When the scene starts, we can see this code before the text across all 4 versions:

Code:

    0901 090B 8020 0F03 0A0C 0030

It would seem that this indicates the following:

Start a new text box for a new character.
Text Box is on Top
Character Portrait is on the Left
The character is Baal.
Face palming.

When he speaks again however, we now see this:

Code:

    0901 0F03 0A0C 0030

This seems to indicate the following:

Start a new text box,
Same Position
Same Portrait Position
Character is Baal.
Face Palming.

The next big code we see is this:

Code:

    0901 0F03 0A0C 002E

This seems to indicate the following:

Start a new text box
Same Position
Same Portrait Position
Character is Baal.
Relaxed face.

Next in this scene the character Mullen is the one speaking so we now see this:

Code:

    0901 090B 8021 0F21 0A0C 0022

This seems to indicate the following:

Start a new text box for a new character.
Text Box is on the Bottom
Character Portrait is on the Right
The character is Mullen.
Serious? Face.

Next, Baal speaks again, only now with this code and it's slightly different in the English PS1 version. In the Japanese and Korean versions it's:

Code:

    0108 0F03 0A0C 002F

In the English version it's this:

Code:

    DC08 0F03 0A0C 002F

This would seem to reflect the following:

Use the previous text box settings (Position, Portrait Position, etc.).
Character is Baal again.
Mild Chuckle Face?

Next Mullen speaks again and we see a code that differs between the English version and the other versions.

In the Japanese and Korean versions it's:

Code:

    0108 0F21 0A0C 0021

In the English version it's this:

Code:

    AA08 0F21 0A0C 0021

This seems to reflect the following:

Use the previous text box settings (Position, Portrait Position, etc.).
Character is Mullen again.
Relaxed face.

So this would seem that the value 0108 does mean use the previous text box settings.
Next we see this code come up quickly:

Code:

    01 0F21 0A0C 0026

This seems to indicate that in the same text box mid sentence we need to change the face portrait to a different one. This is used in the cutscene to change Mullen's expression. It would seem that the value 01 is an interrupt to tell the game to change portraits.

Now I can keep going but I think the patterns should start to become obvious. I've also noticed the following values coming up to signify the end of a line in the text box or the end of a textbox:

Code:

    0E00

090F

The problem however is I can't seem to find consistency for which is used where. The PS1 sometimes uses 0E00 where the Saturn version uses 090F, but then there's instances where both are using the same values.

Now after the end code however, there seems to be varying bytes of data. The size of bytes seems to be the same in the Japanese and Korean Saturn versions, and in many cases the same values. However when we compare to the Japanese PS1 version, the size of bytes between these gaps is different. There's additional data added in. However there is still some data that matches, however bytes are flipped which is consistent with endian differences. This would seem to indicate these are perhaps pointers. I have no idea what the additional data that's added into the PS1 version is though.

Obviously more research is needed here as well as some experimentation, but this seems to be consistent when stepping through the script file in sync with the cutscenes so far.

		Mr^Burns		Sep 11, 2018
			Main site: https://web.archive.org/web/20110723233040/http://... Grandia Korean translation forum: https://web.archive.org/web/20110723233040/http://... And after a lot of korean hangul digging I managed to find the original patch files, but the Grandia.zip is password protected, as is all zips on the site and i'm too tired to figure out what the password is now but you can grab it in the meantime http://rollingdice.tistory.com/945?category=149356...

		TrekkiesUnite118		Sep 12, 2018
			So I decided to play around with the font tonight. I did a simple test where I took the English PS1 font and stretched it to now be 16x16 in size. I then copy pasted it into the Japanese Font file in ASCII order. This way in theory the script files could be edited into English with ASCII encoding to make things easy. And other than a few letters being off, it proved to be successful: I'm attributing the few incorrect characters here to two things: 1) I was probably off in my count for the ... character. 2) I probably missed a control code that I haven't figured out yet that controls Kanji characters. Obviously I didn't edit much beyond this point but the game still prints the font out ok. It's just now everything is truly gibberish as expected:

		Mr Conan		Sep 13, 2018
			Which emulator are you using to test? Mednafen? Last time i took a look Yabause unfortunately crashed at the game load screen.

TrekkiesUnite118

Sep 13, 2018

		klarth said:

I'm using SSF to test it.

		mrkotfw		Sep 14, 2018
			Excellent work. Keep up the great work!

		Mr^Burns		Sep 14, 2018

TrekkiesUnite118

Sep 15, 2018

So I've started putting together a font table for the game as I'm figuring out the encoding more. I have it done for the English portions of the Font, the Hiragana, and the Katakana as well as the special characters. I think I have an idea of how the Kanji encoding works, but it's going to be some trial and error to get it right. I figure if I can get this made it should make it at the very least easier to identify the Japanese text in a hex editor, as well as possibly make it easier to dump the script. It could also make it easier to spot the control codes.

I've also been thinking about the issue of getting the font size changed from 16x16 to 8x16. CyberWarriorX has some good notes in this old thread about how the font and it's compression system works:

https://segaxtreme.net/threads/saturn-hacking-note...

Basically I think this is going to require some ASM hacking of the text decompression routine and possibly the text drawing routine. The Decompression routine works by decoding the first 8x16 portion of the character, followed by the next 8x16 portion. I'd imagine this would need to be changed so only the first half is decoded and treated as one character. I'd also imagine that this would impact the pointer table of the FDATA.BIN file. Finally depending on if the game is spacing the text based on font width or if it's spacing it in 16x16 increments regardless of it's size, would determine if the text drawing routine would need to be modified.

This is something that is way above my skill level so I'm probably going to need some help on this one. I was planning on reaching out to CyberWarriorX and seeing if he had any thoughts on it or if he wanted to jump in and help. If anyone here wants to give it a shot feel free to jump in. I was also thinking about reaching out for help on ROMHacking.net.

I have however had some success figuring out a bit more about the control codes.

The code 090F that I before thought was end of text I believe is actually 1 code plus the start of another. The first code is 09. This seems to indicate a pause in text drawing. It seems to be used to pause the text to keep it in sync with pauses in the voices. The next code is actually 0FXX. The XX I've seen be any value from 01 up to 0A so far. This seems to be used for various different things. What I've found so far though is the following:

0F03 seems to be related to controlling the Left Portrait.
0F21 seems to be related to controlling the Right Portrait.

The next code I was able to figure out is this one:

Code:

    0A 0C 00 XX

This code tells the game to switch to a different portrait, where XX is the value for the portrait you want displayed.

So for example if you see the following:

Code:

    0F 03 0A 0C 00 XX

It would appear to mean "change the left portrait to portrait XX".

I was able to confirm this in testing by changing the portraits in the intro cutscene to arbitrary values:

There's still quite a few different values for 0F XX however, and I haven't quite been able to nail down what they all do yet.

		TrekkiesUnite118		Sep 15, 2018
			So after making a topic on RomHacking.net it was revealed that the 8x16 English font is actually in the Saturn version, and it's in ASCII format. To use it you just need to start your text line with the code 03 and then you can edit in ASCII encoded text. I tested this and got these results: So basically this makes things a lot easier I'd imagine. Now it's just figure out the control codes, any pointers in the files for text size and location. With that information it probably wouldn't be too difficult to write a tool to dump the JP script, and then another to inject an English script with pointer values adjusted for expanded text. Though it would still probably be beneficial to make some kind of hack to make the 8x16 font the default font so you don't have to waste a byte with the 03 control code. I'm guessing this is what the English PS1 version did as I'm not seeing any evidence of a control code like that being used in it's files.

		TrekkiesUnite118		Sep 16, 2018
			So I think I might be figuring out how the script pointers work based on comparing the PS1 versions, and then spot checking with the Saturn version. So here is what I believe to be one complete text routine from the file BA38.MDT at x05B4: This highlighted portion I believe is a header of some kind. It seems to change from text sample to text sample, so there's probably optional values that are in some text boxes but not in others. Now this next highlighted portion is the important part. This value I believe represents how many bytes until the end of this text box. In this example it's 5A: If we then move forward 5A that will bring us to the end of this block of data. The end of a block of text data seems to be tagged with this possible footer: After this the next text box starts. I've been able to keep following this pattern and consistently keep finding what appears to be the correct number of bytes and the start of the next text box. I'm going to keep testing this but so far it's looking promising.

TrekkiesUnite118

Sep 17, 2018

Ok, so I think I've cracked the code of the MDT/MDP file headers. This information seems to hold true for both the Saturn and PS1 files, the only difference being that in the PS1 files pointers are Little Endian where as in the Saturn files they're Big Endian.

The header is 512 Bytes in size.
The header seems to consist of just a pointer table
Each entry is 8 bytes total in size
The first 4 bytes seem to be the place in the file where the data lives
The second 4 bytes seem to be the size of the data.
The entry that tells us where the text lives and how big it is seems to be at x60 every time.
There's an entry at x58 that seems to point to a 24 byte piece of data right before the text. Perhaps this is a header or something?
Consistently I'm seeing in the English PS1 version that any addresses that come after the address defined at x60 have their values increased by the increase in size of the text data, and any values that come before it are untouched.
I'm also seeing again padding being removed in the PS1 files of the exact amount added to the text.
Another oddity I'm seeing is that the last entry in the table at x1F8 seems to also have it's size increased by the increase in text size. But I'm not sure what that value represents or what data was added.

Some other oddities I'm seeing is that some entries don't seem to make sense as the addresses are beyond the size of the file, but they're still updated in the English PS1 version.

With this info I imagine I should be able to make a program that can do the following:

Parse the MDT files
Extract the script portions
Put the MDT file back together with a new script portion.
Recalculate the pointer table.

The hard part to automate in code I'd imagine would be to find a portion of padding in the file that can be removed to keep the file size the same.

...

Next>