Kuribo64
Views: 16,334,192 Home | Forums | Uploader | Wiki | Object databases | IRC
Rules/FAQ | Memberlist | Calendar | Stats | Online users | Last posts | Search
12-03-21 11:40 AM
Guest:

0 users reading Model formats - A better understanding | 1 bot

Main - Archived forums - SMG documentations and tutorials - Model formats - A better understanding Hide post layouts | New reply

Pages: 1 2 3
blank
Posted on 02-14-13 02:53 PM (rev. 2 of 02-14-13 02:59 PM) Link | #15358
Color channels are basically what produces the rasterizer color. There are two color channels, a primary and a secondary. Each channel takes as input a material color and an ambient color, and calculates an output color based on the channels lighting configuration. That color can then be used in the TEV stages.

But color and alpha can be configured separately, and the material in WoodCircleCutPlanet.bdl with the dark texture problem has lighting enabled for the alpha channel but not for the color channel.

Try looking here: http://libogc.devkitpro.org/gx_8h.html. The relevant functions are GX_SetChanCtrl, GX_SetChanMatColor and GX_SetChanAmbColor. The relevant information in bmdview2 in the Material struct is chanControls, color1 (arguments to GX_SetChanMatColor) and color2 (arguments to GX_SetChanAmbColor).

Posted by dash
EDIT: Possibly your statements are connected to your definition of the ColorChanInfo struct you provided earlier. That has 2 fields, enable and litMask. In the WoodCircleCutPlanet.bdl the array of those structures has 5 entries:
colorChanInfo: 000100020100ffff 010104020100ffff 010000010000ffff 000000000200ffff 000000020100ffff (5 many)

The material 6 structure has this for chanControls:
chanControls (?): 0000000100020003

The bmdview2 source looks at chanControls[0] (the first entry, 0000). The whitehole source just skips over the chanControls array.


Yes, that is the problem: bmdview only looks at chanControls[0] (the channel controls for the primary color channel) but not at chanControls[1] (the channel controls for the primary alpha channel).

dash
Posted on 02-15-13 12:20 AM (rev. 2 of 02-15-13 01:33 AM) Link | #15473
Posted by blank
Yes, that is the problem: bmdview only looks at chanControls[0] (the channel controls for the primary color channel) but not at chanControls[1] (the channel controls for the primary alpha channel).


Good lord this is hellishly complicated...

According to the debug dump from bmdview2 it is saying numChans=00 for all the materials. I sort of grasp the 3 different options for GX_SetNumChans(), but it was looking like the numchans was always getting set to 0. But it appears there is a table lookup, so that numChans=00 is indexing into a table:
numChans (?) (unk[2]): 01 54 68 69 (4 many)

Which would mean we actually are doing GX_SetNumChans(1). Similiarly it looks like each material has a tevcount number, but those only make sense if they're also indexing into a table:
tevStageCounts (unk[4]): 02 01 54 68 (4 many)

Now, I grasped the TevStageInfo how it has abcd selection, operation, bias, scale, doclamp and output select for BOTH the rgb and alpha. I hadn't noticed the alpha side of things until I saw the function calls in that libogc reference, and saw everything was duplicated for alpha as well...

But are you sure the chanControls[] table works as you suggest? It would make more sense (to me at least) if that table indexes into the colorChanInfo table, but NOT where chanControls[0] is associated with the rgb, and chanControls[1] is associated with the alpha.

See, you've got 0, 1 or 2 channels feeding into the TEV pipeline. You've got a big table of control settings for a channel. A material would need a small table indexing into the channel control settings table for anywhere from 0 to 2 channels being used. So whoever worked out this storage format leaves space for 4, to accomodate a bit of future expansion (one more channel).

There are the TevStageinfo lines:
TevStageInfo (ff) (ain, bin, cin, din) (op, bias, scale, doClamp, tevRegId):
ff 0e 02 0a 0f 00 00 00 01 00 06 07 05 07 00 00 01 01 00 ff
ff 0f 00 08 01 00 00 00 01 00 07 05 04 07 00 00 00 01 00 ff
ff 0f 0a 08 0f 00 00 00 01 00 07 05 04 04 00 00 00 01 00 ff (3 many)
We see each row has the TEV configuration for both the rgb and alpha machinery.

Now it seems to me the problem with bmdview2 is the variable number of TEV stages is implemented, but only (effectively) a single channel is implemented. All the materials in the WoodCircleCutPlanet.bdl use 2 channels.

I haven't come to understand exactly how the multiple channels are combined before being fed into the TEV pipe. This link provides some useful theory about the gamecube hardware, I saw it referred to somewhere (this website?) but don't remember where.

http://www.gamasutra.com/view/feature/131354/shader_integration_merging_.php?page=2

That page mentions channels. "The conditional add is facilitated using the two different color channels GX_COLOR0 and GX_COLOR1."
EDIT: I found how the 2 channels are combined: They aren't. This is from that gx_8h.html page: "Since there is only one rasterizer for color in the graphics hardware, you must choose which color to rasterize for each stage in the Texture Environment (TEV) unit. This is accomplished using GX_SetTevOrder()."

blank
Posted on 02-15-13 01:44 AM Link | #15498
Posted by dash
According to the debug dump from bmdview2 it is saying numChans=00 for all the materials. I sort of grasp the 3 different options for GX_SetNumChans(), but it was looking like the numchans was always getting set to 0. But it appears there is a table lookup, so that numChans=00 is indexing into a table:
numChans (?) (unk[2]): 01 54 68 69 (4 many)

Which would mean we actually are doing GX_SetNumChans(1). Similiarly it looks like each material has a tevcount number, but those only make sense if they're also indexing into a table:
tevStageCounts (unk[4]): 02 01 54 68 (4 many)


Yes, just about everything in the materials are indices into some table.

Posted by dash
But are you sure the chanControls[] table works as you suggest? It would make more sense (to me at least) if that table indexes into the colorChanInfo table, but NOT where chanControls[0] is associated with the rgb, and chanControls[1] is associated with the alpha.

See, you've got 0, 1 or 2 channels feeding into the TEV pipeline. You've got a big table of control settings for a channel. A material would need a small table indexing into the channel control settings table for anywhere from 0 to 2 channels being used. So whoever worked out this storage format leaves space for 4, to accomodate a bit of future expansion (one more channel).


chanControls[0] references the controls for the primary color channel (GX_COLOR0), chanControls[1] references the controls for the primary alpha channel (GX_ALPHA0), chanControls[2] references the controls for the secondary color channel (GX_COLOR1) and chanControls[3] references the controls for the secondary alpha channel (GX_ALPHA1). I'm certain of that.

dash
Posted on 02-15-13 07:55 AM Link | #15517
Posted by blank
chanControls[0] references the controls for the primary color channel (GX_COLOR0), chanControls[1] references the controls for the primary alpha channel (GX_ALPHA0), chanControls[2] references the controls for the secondary color channel (GX_COLOR1) and chanControls[3] references the controls for the secondary alpha channel (GX_ALPHA1). I'm certain of that.


Ok that does seem to make more sense.

I'm not even sure now what would need to change in order to fix this. As you say lighting is off on the rgb channel, but on on the alpha channel. I'm not even sure of what is supposed to happen. If the alpha is not 1.0, doesn't that imply transparency when the texture is rendered to the display? And wouldn't that mean you've got a transparent grassy texture combined with the wood wall? And in that case, wouldn't the order the textures are rendered in matter?

blank
Posted on 02-15-13 10:52 AM Link | #15540
It's a common misunderstanding that alpha is same as transparency, but it's not. Alpha is an extra auxiliary value that can be used for a lot of different things. In the gamecube/wii, what the alpha value is used for depends on stuff like the TEV stages, alpha compare settings and blend mode.

As for what to do to fix the problem, I don't really know either. Lighting isn't really implemented in bmdview2, and properly implementing it is more work than it's worth, in my opinion.

dash
Posted on 02-15-13 12:17 PM Link | #15547
Posted by blank
As for what to do to fix the problem, I don't really know either. Lighting isn't really implemented in bmdview2, and properly implementing it is more work than it's worth, in my opinion.


Still, I think I benefitted a bit from digging around (always nice to have an excuse to learn something). And the discussion might be of benefit to others who want to further refine the understanding of the bmd files.

One thing that is obvious from this exercise is the bmd files are just a wrapper for the settings values for the gamecube / wii SDK functions (which presumably just set hardware registers). That is, by understanding the API calls and what they're supposed to do it becomes pretty obvious what the fields in the BMD file are for. I imagine in the old days the first pass at figuring out what the bmd format was involved using a debugger to single step the CPU and find out how data is read from the bmd file and fed into API calls. It can't be very convoluted, for performance reasons. So I'd guess there wasn't a lot of assembly code to wade through.

Something I've wondered about. The devkitpro / libogc that is used to program the wii (Wiibrew)... it seems so polished and elaborate I'd always thought it was Nintendo's own SDK released to the public (or leaked).


dash
Posted on 02-22-13 12:49 PM Link | #16320
For the sake of closure I just wanted to note I fixed the problem in the bmdview2 codebase that was causing it to output a corrupt 3ds file. The updated code is on the googlecode page here:

http://code.google.com/p/bmdview2/

The original author probably would be kicking himself, the mistake was so trivial:
- for(size_t k = 2; j < curr.points.size(); ++j)
+ for(size_t k = 2; k < curr.points.size(); ++k)

JwopDk
Posted on 04-05-13 02:57 AM (rev. 6 of 04-16-13 07:54 AM) Link | #19184
I think it would be important to have more info on the MDL3 section of the BDL format as it is the only thing that differs from the BMD format (apart from a few bytes at the start of the file), and that the current converter isn't well documented.

Here is my revised version of bdl2bmd in C which can be found here: http://www.sendspace.com/file/mjvsej
(I forgot to reset the file size at 0x8 in the old version
and I made it so that it uses file buffers a bit more safely)
Let me know if it's a bit easier to understand than the original (I included source)

Arisotura
Posted on 04-05-13 07:52 AM Link | #19187
From what I could gather, the MDL3 section contains series of hardware commands that set up the Wii's GPU or something in the like. They're not required to get stuff displayed on a PC, but the Wii hardware needs them.

____________________
NSMBHD - Kafuka - Jul
melonDS the most fruity DS emulator there is

zafkflzdasd

blank
Posted on 04-05-13 11:52 AM Link | #19202
The MDL3 section contains the materials of the model, but in a format different from the MAT3 section. In the MAT3 section the materials are stored as arguments to the GX functions, while in the MDL3 section they are stored as raw GP packets.

Marionumber1
Posted on 04-05-13 03:08 PM Link | #19207
What is the reason for having both?

JwopDk
Posted on 04-05-13 11:43 PM (rev. 15 of 04-14-13 09:51 AM) Link | #19231
Ok, so I got some info from mdl3.py from obj2bdl and proceeded to write up a table of the basic header:
(note that this post will be updated as I find more info)
+-------------------------------------------+
| MDL3 Header |
+------+------+-----------------------------+
|Offset| Size | Description |
+------+------+-----------------------------+
| 0x00 | 0x04 | Magic "MDL3" |
| 0x04 | 0x04 | Size of section |
| 0x08 | 0x02 | Number of materials |
| 0x0A | 0x02 | Padding? (0xFFFF) |
| 0x0C | 0x04 | Packet Header Offset (0x40) |
| 0x10 | 0x04 | SS1 Header Offset |
| 0x14 | 0x04 | Matrix Index Header Offset |
| 0x18 | 0x04 | Unknown Header Offset |
| 0x1C | 0x04 | Index Header Offset |
| 0x20 | 0x04 | Material Name Header Offset |
| 0x24 | 0x1C | Padding |
+------+------+-----------------------------+

+-----------------------------+
| Packet Header |
+------+------+---------------+
|Offset| Size | Description |
+------+------+---------------+
| 0x00 | 0x04 | Packet Offset |
| 0x04 | 0x04 | Packet Size |
+------+------+---------------+

+--------------------------------------+
| SS1 Header |
+------+------+------------------------+
|Offset| Size | Description |
+------+------+------------------------+
| 0x00 | 0x02 | Channel Colour Offset |
| 0x02 | 0x02 | Channel Control Offset |
| 0x04 | 0x02 | Texture Gen Offset |
| 0x06 | 0x02 | Texture Offset |
| 0x08 | 0x02 | TEV Offset |
| 0x0A | 0x02 | Pixel Offset |
| 0x0C | 0x04 | Padding (0xFFFFFFFF) |
+------+------+------------------------+
| Loops for every material |
+--------------------------------------+

+----------------------------------------------------------+
| Matrix Index Header |
+------+------+--------------------------------------------+
|Offset| Size | Description |
+------+------+--------------------------------------------+
| 0x00 | 0x04 | Big endian floating point? (0x3CF3CF00) |
| 0x04 | 0x04 | Little endian floating point? (0x00F3CF3C) |
+------+------+--------------------------------------------+
| Loops for every material |
+----------------------------------------------------------+

+-------------------------------------------------------------------------+
| Unknown Header |
+-------------------+---------------------+-------------------------------+
| Offset | Size | Description |
+-------------------+---------------------+-------------------------------+
| 0x00 | Number of materials | Repeats the value 0x01 |
| Number of entries | Alignment | Aligns to the next 0x04 bytes |
+-------------------+---------------------+-------------------------------+

+--------------------------------------------+
| Index Header |
+------+------+------------------------------+
|Offset| Size | Description |
+------+------+------------------------------+
| 0x00 | 0x02 | Index (i) |
+------+------+------------------------------+
| for i = 0, i < Number of materials, i += 1 |
+-----+-------+------------------------------+
| i*2 | Align | Padding |
+-----+-------+------------------------------+

+------------------------------------------------------------------------------------------------+
| Material Name Header |
+-------------------------------------+----------------------------+-----------------------------+
| Offset | Size | Description |
+-------------------------------------+----------------------------+-----------------------------+
| 0x00 | 0x02 | Number of materials |
| 0x02 | 0x02 | Padding? (0xFFFF) |
| 0x04 | 0x04 * Number of materials | Unknown |
| (0x04 * Number of materials) + 0x04 | 0x04 * Number of materials | Names of materials in ASCII |
+-------------------------------------+----------------------------+-----------------------------+

Once I get enough info I might see if I can create a BMD to BDL converter by inserting
an empty (yet valid) MDL3 Section into the output BDL file and then maybe go from there.

EDIT:
I think it might be better to copy and modify the MAT3 section from
the input BMD and use THAT as the MDL3 section in the output BDL.

NWPlayer123
(post deleted) #19233

blank
Posted on 04-06-13 02:36 AM (rev. 2 of 04-06-13 10:35 AM) Link | #19243
Posted by Marionumber1
What is the reason for having both?


I don't really think there is a good reason. They probably added the MDL3 section because the GP packets are faster to load, and kept the MAT3 section because the BDL format is supposed to be an extension of the BMD format.

Posted by JwopDk
Once I get enough info I might see if I can create a BMD to BDL converter by inserting an empty (yet valid) MDL3 Section into the output BDL file and then maybe go from there.


That was also my first idea, but that won't work. You might want to read this thread (you'll have to sign up to the forum to see it) to see what have already been done.

EDIT: You might also find this thread interesting.

Arisotura
Posted on 04-06-13 05:42 AM Link | #19244
Now I wonder. Does a model still render properly if its MAT3 section is emptied? I have a feeling MAT3 and MDL3 kinda go together, and aren't really a replacement for eachother...

____________________
NSMBHD - Kafuka - Jul
melonDS the most fruity DS emulator there is

zafkflzdasd

blank
Posted on 04-06-13 07:22 AM Link | #19246
There is some texture matrix info and one unknown in the MAT3 section that is not in the MDL3 section, so MDL3 is not a complete replacement for MAT3. But other than those two things, the information in the two sections is the exact same, just stored in different formats.

JwopDk
Posted on 04-06-13 09:23 AM (rev. 6 of 04-06-13 11:22 AM) Link | #19248
@blank Yeah I was thinking a dummy MDL3 section might not work either and the
best way was to copy and modify the MAT3 section and use THAT as the MDL3 section.

EDIT:
Always check my previous post for new MDL3 info.
Also, I plan on doing a similar analysis on MAT3 so I can compare it with MDL3.

JwopDk
Posted on 04-12-13 11:36 AM (rev. 4 of 04-13-13 12:04 AM) Link | #19907
Ok, so after a week of being away and trying to understand the MDL3 section I finally understood what you guys were on about with GP packets.

To those who aren't up to speed, the core of the MDL3 section is basically full of hardware commands that it sends mostly to the blitting processor (the thing that renders or "paints" 3D objects, lighting effects and other things on to the 640x480 screen), which is a way that is slightly different and perhaps more efficient than the original MAT3 section, which from what I understand contains arguments to the functions (or in other words variables to the formulas) within the game. I think MDL3 might be more efficient than MAT3 purely because my instincts tell me hardware is faster than software, but I don't really know. Hell, I don't even know if Super Mario Galaxy uses both or just MDL3 or whatnot.

Anyway, realising this, I proceeded to create a rather lousy disassembler of sorts for the wii graphics code (what's used in the MDL3 section) which can be found here (source included 'cause I'm such a nice guy): http://www.sendspace.com/file/djjr4z

Finally, as promised, I made an analysis of the main header of the MAT3 section, not bothering to detail any of the many components it entails:
+-------------------------------------------------+
| MAT3 Header |
+------+------+-----------------------------------+
|Offset| Size | Description |
+------+------+-----------------------------------+
| 0x00 | 0x04 | Magic "MAT3" |
| 0x04 | 0x04 | Size of section |
| 0x08 | 0x02 | Number of materials |
| 0x0A | 0x02 | Padding? (0xFFFF) |
| 0x0C | 0x04 | Entry Header Offset (0x84) |
| 0x10 | 0x04 | Index Header Offset |
| 0x14 | 0x04 | Material Name Header Offset |
| 0x18 | 0x04 | Indtex Header Offset |
| 0x1C | 0x04 | Cull Mode Header Offset |
| 0x20 | 0x04 | Material Colour Offset |
| 0x24 | 0x04 | Number of Channels Offset |
| 0x28 | 0x04 | Channel Control Offset |
| 0x2C | 0x04 | AMB Colour Offset |
| 0x30 | 0x04 | Unknown #1 (Same offset as below) |
| 0x34 | 0x04 | Number of Texture Gens Offset |
| 0x38 | 0x04 | Texture Coordinates Gen Offset |
| 0x3C | 0x04 | Unknown #2 (0x0) |
| 0x40 | 0x04 | Texture Matrix Offset |
| 0x44 | 0x04 | Unknown #3 (0x0) |
| 0x48 | 0x04 | Texture Offset |
| 0x4C | 0x04 | TEV Order Offset |
| 0x50 | 0x04 | TEV Colour Offset |
| 0x54 | 0x04 | TEVK Colour Offset |
| 0x58 | 0x04 | Number of TEV Stages Offset |
| 0x5C | 0x04 | TEV Combiner Offset |
| 0x60 | 0x04 | Swap Mode Offset |
| 0x64 | 0x04 | Swap Mode Table Offset |
| 0x68 | 0x04 | Fog Offset? |
| 0x6C | 0x04 | Alpha Compare Offset |
| 0x70 | 0x04 | Blend Mode Offset |
| 0x74 | 0x04 | Z Mode Offset |
| 0x78 | 0x04 | Z Compare Location Offset |
| 0x7C | 0x04 | Dither Offset |
| 0x80 | 0x04 | Unknown #4 Offset |
+------+------+-----------------------------------+

And yeah. I hope this page gets some more interest.

element3260
Posted on 04-13-13 04:37 AM Link | #19984
I'm also working on continuing Thakis' work on a .bmd/.bdl loader. I'm fixed a few of his bugs/missing functionality, but I still have a ways to go. Right now I'm working on texture generation (where Thakis' support is super thin) and I've run into a jam. I can't quite figure out how the texture matrices (which are referenced by GX_TG_TEX0 or GX_TG_TEXCOORD0) are stored in the .bmd. Thakis seemed to think that they weren't stored as matrices, but as a pair of scales and offsets. I think this is incorrect. Does anyone have any experience in this area?

My code, which is DirectX/HLSL, is linked on my site here: http://realtimemike.wordpress.com/research/

P.S. blank: thanks for the GX_SetChanCtrl tips. I had originally implemented them as separate color and alpha, because of the /* chan id */ section of gx.h and because there are 4 chanCtrl indices in the mat section of a .bmd. Then I read somewhere on the net that the alpha and color were merged, so I changed it back. It broke some of my models so I thought that I had exposed more bugs, but it looks like I was right the first time. Thanks!

Arisotura
Posted on 04-13-13 06:36 AM Link | #19985
Some information on the 'texture matrices' would be really nice. I've been unable to implement it in Whitehole so far :(

I could try looking into the game's disassembly I guess. If I can find how the values are used and passed to the GX...

____________________
NSMBHD - Kafuka - Jul
melonDS the most fruity DS emulator there is

zafkflzdasd
Pages: 1 2 3

Main - Archived forums - SMG documentations and tutorials - Model formats - A better understanding Hide post layouts | New reply

Page rendered in 0.031 seconds. (2048KB of memory used)
MySQL - queries: 29, rows: 237/237, time: 0.023 seconds.
[powered by Acmlm] Acmlmboard 2.064 (2018-07-20)
© 2005-2008 Acmlm, Xkeeper, blackhole89 et al.