SegaXtreme

Home	Forums	What's new	Resources

VDP1 sprites priority

paul_met - Jul 20, 2019

		paul_met	Jul 20, 2019
		Hello. I wondered how to make the sprites that VDP1 draws translucent on top of VDP2 layers. I got it on the example of the game Device Reign. But almost all sprites become translucent at once. Although there are exceptions (character panel with a portrait and statistics). It seems that individual sprites are somehow given their own priority. But I don’t know whether to change the order of the sprites in such a way as to make only specific ones translucent.

		antime	Jul 20, 2019
		Transparency depends on the pixel data in the VDP1 frame buffer, and how VDP2 is configured to interpret it. By setting the CMDCOLR word of the sprite drawing command, you can select different priority and color calculation settings for each sprite. Chapter 9 of the VDP2 manual explains it, but it's not the easiest read. This thread... also discussed sprite modes and color banks.

paul_met

Jul 20, 2019

	antime said:

I can not figure out how to do it. Need to put a breakpoint at 25C00006? But it does not give any effect.

		mrkotfw	Aug 11, 2019
		Identify the command table that corresponds to the in-game characters. In the word that @antime... described, it should tell you whether it's actual color bank code, or a CLUT which could contain color bank codes. From that, find what sprite type the VDP2 is set to. From there you can figure out which bits correspond to what in the color bank code. You won't see the effect outright. The effect is done, I believe when the VDP2 merges the result frame buffer from the VDP1.

		vexatious	Sep 19, 2019
		Not sure how you're imposing transparent sprites but usually it involves a color table with lookup table. Don't know too much about Saturn compared to Mars but... Double buffering is also used for a more advanced method. Some programmers might call this "copy to buffer". This way usually solves most problems if you impose it only when necessary and only on required sprites-tiles-layers so as not to lose memory or waste it. This has the benefit of many other special effects possible like: shading, lighting, blending, motion blur, and etc. Don't confuse dual VDPs with double buffering; I think you want double buffering only on VDP1. By the way what game is that first picture? Edit: Oh wait nevermind. Didn't know "Device Reign" was a catalog.

		paul_met	Jun 10, 2020
		Returning to the topic of translucency of VDP1 objects, I again want to raise the issue of sprite priority (since I could not figure it out). There is such a table in the VDP2 manual (screenshot No. 1). What exactly is the Sprite Register#? And how to assign a particular sprite to any of the Sprite Register numbers? I experimented with the infamous Castlevania SotN to enable translucency for the background of the dialog box. I think that many have noticed that the game already uses the translucency of VDP1 objects (for example, the background of the candle). I found the sprite priority table in the game. Empirically, I found out that the candle's background uses Sprite Register 5 (1800F6h, 10-8 bits) and a value of 06h. While the background of the dialog box is Sprite Register 4 (1800F6h, bits 2-0) and the value is 04h. Since I do not know how to make the dialog box background use Sprite Register 5, I replaced the values at the address (1800F6h, bits 2-0) with 06h. As a result, the background of the dialog box became translucent (screenshot #2). But together with the background of the dialogue window, the portrait of the character becomes translucent. So the portrait of the character is also Sprite Register 4. How do I get the window background sprite to read the value at the address (1800F6h, 10-8 bits)?

		mrkotfw	Jun 10, 2020
		It's confusing, really. The priority bits in the color bank code (VDP1) is an index into the sprite register priority (VDP2). So let's say you set sprite register #0 to be 7, sprite register #5 to be 2. In your VDP1 command table, if your priority bit(s) is 0, then your sprite will have a priority of 7. If you set the priority bit(s) to be 5, then your sprite will have a priority of 2. I think you need to allocate a sprite register for what you don't want to be translucent, and another for sprites that you do want translucent. @fafling... can probably correct me if I'm wrong here.

		TrekkiesUnite118	Jun 10, 2020
		Looks like it also made the Health bars translucent.

		fafling	Jun 10, 2020
		@mrkotfw... The priority bits indirection may be confusing, but it allows to use any of the 8 priority numbers even when there are less than 3 priority bits in the sprite type. @paul_met... Like antime said, for sprites with palette textures, the priority bits are set by the color bank parameter of the VDP1 drawing command (contained by the CMDCOLR word of the command). The priority bits position in the color bank word are given by the sprite type structure described in chapter 9 "Sprite data" of VDP2 manual. The sprite type is how VDP2 parses framebuffer pixels that it receives from VDP1. During the dialog with Death : Sprite type 5 is used. So the priority bits are set by bits 12 to 14 of the color bank word. The color calculation condition, that framebuffer pixels must meet to be blended by VDP2, is that their priority number must be 6. Since only the sprite priority register 5 is set to that priority number, the priority bits of the color bank word must be set to 5 to draw a sprite blended by VDP2. The color bank word of the drawing command of the dialog box is 4580h so it sets prioriy bits to 4. The color bank is not always the same for all dialog boxes : the 1st ones before the fight with Dracula use 4550h, but that's again priority 4. The texture address in vram is also different but the texture looks identical. You need to find the code that sets the color bank word of the drawing command of the dialog box. It occurs most likely in work ram, before the transfer of the drawing command in VDP1 vram. The drawing command position in vram is most likely variable.

		paul_met	Jun 11, 2020
		Now it became clear why the palette banks are duplicated. Thank you. PS: I wonder why the game developers did not make the background of the dialog box translucent initially?

		rorirub	Jun 11, 2020
		It has been a long time since I checked, but my impression on reading the manual was that in VDP1, you can select sprite type, which just controls how many bits are used for palette entry, priority, and color command (transparency relative to vdp2). But the priority you set while drawing the VDP1 sprite is not the actual priority, it is just a pointer to the VDP2 priority registers. So you can have the VDP1 use priority register #1 and #2, but on the VDP2 those registers are set to values of 3 and 7. While this is confusing at first, it has the advantage of being able to use 8 different priority values even if the sprite only has 2 bits (4 total values) of priority information to them, and also you can change priorities by just setting a register in VDP2, without drawing anything in VDP1. But I assume the most important reason is so you can have the sprite set to a higher priority number than the amount of priority bits they allow. So for ex. you only have 1 priority bit in the sprite, but in the VDP2 you set that one value to be considered priority values 4 and 8 (for bit values 0 and 1). I mean, you can't really put the sprite in the correct z-order otherwise, with the amount of backgrounds you can set with the VDP2. As for Castlevania Saturn, that game is a gigantic mess that does the weirdest and most inefficient things. You'd probably need to re-code it from the ground up for it to look decent on the Saturn.

rorirub

Jun 11, 2020

	paul_met said:

Outside of the fact that it's a terrible port done by an incompetent team that misuses the VDP2 in the stupidest ways possible, maybe they did not like the fact that doing this would make every sprite underneath the transparent one disappear. Like in your screenshot, Alucards legs are cut off.

Some scenes are pretty sprite heavy, so this wouldn't be useful at all. For example when the dialog box is on the top, it would cut in half your health/magic meter, or hide some enemies from appearing, etc.

		TrekkiesUnite118	Jun 11, 2020
		Yeah, that could be it. Though at the same time they could have drawn the text box with VDP2 and the text with VDP1 instead and everything would have been fine.

paul_met

Jun 11, 2020

	rorirub said:

But Treasure was not embarrassed by the fact that no other sprites were displayed over Nicole's translucent cloak (Guardian Heroes, I mean).

		paul_met	Jun 11, 2020
		So, after activating the Half-transparent bits of the sprite, "alucard legs" appeared.

		fafling	Jun 11, 2020
		You'll have to check how it displays on the console, cause you're using half-transparency over a palette texture which is undocumented. I'm not sure that your screenshot shows the correct mix of palette codes with RGB pixels of the framebuffer, compounded with the color bank that's supposed to make VDP2 blend the result with the background.

paul_met

Jun 11, 2020

	fafling said:

Yes, this combination of translucency does not work on mednafen and SSF. I think that it will not work on the original hardware either.

		TrekkiesUnite118	Jun 11, 2020
		If you have a build for it I can try it later tonight for you. I found with Grandia that just about all the Saturn Emulators are fudging VDP2 in some way or another, so who knows which emulator is accurate there. I'm still guessing it wont work but you never know. If it doesn't though, I think the only way you'll really fix those is by having them render to VDP2 like what Grandia does so it's text boxes can be transparent. Though the first go with it erasing other sprites might be ok to use for things like the item pickup boxes and enemy name boxes that show up in the lower part of the screen since I'd imagine you'd rarely have an issue of it erasing things for a long enough time for it to be noticeable.

paul_met

Jun 11, 2020

	TrekkiesUnite118 said:

I attached the file. Insert it into the image of the original game using the CDMage utility. Translucency will only work in the scene of a meeting with death (as shown in the screenshot above).

		TrekkiesUnite118	Jun 11, 2020
		On real hardware it looks like you get the same issue as before with Alucard's feet disappearing.