Kuribo64
Views: 15,696,483 Home | Forums | Uploader | Wiki | Object databases | IRC
Rules/FAQ | Memberlist | Calendar | Stats | Online users | Last posts | Search
09-24-21 02:22 AM
Guest:

0 users reading Model formats - A better understanding | 1 bot

Main - Archived forums - SMG documentations and tutorials - Model formats - A better understanding New reply

Pages: 1 2 3
SuperMario64DS
Posted on 02-02-13 11:52 PM (rev. 3 of 02-03-13 12:00 AM) Link | #12957
I am doing an extensive analysis on the two related formats used primarily by the GameCube and Wii, BMD and BDL.

As far as I know, no easily readable, fully complete analysis exists. The goal here is to completely break this format down into simple text, on this page here: http://wiki.tockdom.com/wiki/BMD_and_BDL_(File_Format)

What's currently on that documentation has been done me (With some assistance from other Custom Mario Kart members), very incomplete and bound to have some sort of error somewhere. I'm looking for anyone to help further the understanding of this format, many seem to yet a complete and readable analysis is still absent.

Pretty much what we're looking for here would be anything that could be found useful in an editor, such as allowing Mipmap display, opacity or skinning, things that aren't quite fully understood or figured out yet. We can't hold out, and deal with blocky, vertex colour-less unanimated models forever.

Plans to add BDL support to the popular Brawl editing software, BrawlBox, has been rumoured for quite some time, however, the creators won't even consider other formats without a full analysis.

Whether they do or not, it's always useful to have a complete analysis of the common formats used by the game. It took them approximately 8 years to figure out vertex colouring for Super Mario 64 (Considering the start of it's hacking existence, not that were ever trying to figure out during the majority of that period), so why let this newer game be limited to dull, really bright models that have to use dozens of funky workarounds to look proper in game?

My experience with programming is very limited, the best I've made would have been a small editor for MSBT Files, and some super ancient 2008 online game "Trainer", which is too embarrassing to talk about (Not to mention Assembly Sonic hacks, but we won't go into that right now). Perhaps if the analysis were more extensive, I could attempt to further extend my programming knowledge and attempt to make such an editor.

Before anybody may think it's un important, try this comparison:

This hideously bright place:

To this colourful shader/vertex colour implemented place:


Later on I'd like to perform a similar analysis on the game's main system as well, such as finding specific memory functions and RAM addresses - Anything we discover and put together will benefit the community. So please, if you know anything, please share it here or add it to the page mentioned above. After nearly 500 different BMDs and BDLs having had been analyzed by me, I'd like further assistance from others to help fully crack this format.

SquidEmpress
Posted on 02-02-13 11:59 PM Link | #12961
I am a bit confused. Are you talking about the Object database?

____________________
Super Mario LOLand

Thanks to Tierage for helping to port my Acmlmboard 2.5 layout to Acmlmboard 2.064!


shibboleet
Posted on 02-02-13 11:59 PM (rev. 2 of 02-03-13 12:00 AM) Link | #12962
Fix the 2nd link.

Other then that, nice :)

Blarg, read the thread again >.<
It's about the format.

____________________
a

blank
Posted on 02-03-13 09:53 AM Link | #13028
Posted by SuperMario64DS
Plans to add BDL support to the popular Brawl editing software, BrawlBox, has been rumoured for quite some time, however, the creators won't even consider other formats without a full analysis.


Have you actually talked to BlackJax about this, or have you just heard some rumours? Because I don't really feel it's worth the time to write a full description if I'm the only one working on BDL editing tools anyway.

SuperMario64DS
Posted on 02-03-13 01:52 PM Link | #13199
Posted by blank
Have you actually talked to BlackJax about this, or have you just heard some rumours? Because I don't really feel it's worth the time to write a full description if I'm the only one working on BDL editing tools anyway.


They have hinted support for other games before, Twilight Princess & such. Even so, this information should be readily available to anyone who needs it, there are other people within the Mario Kart community as well who have an interest in this format too.

NWPlayer123
Posted on 02-03-13 02:06 PM Link | #13220
This is kinda off-topic, but Twilight Princess can already be somewhat hacked, since Wind Maker supports Twilight Princess (to an extent), however, it uses BMD or BDL as well, so IDK why.
However, I do agree with blank being hesitant to fully figure all this stuff out.

____________________
"I hate playing musical chats" ~ Quote of the month

SuperMario64DS
Posted on 02-03-13 02:39 PM Link | #13244
Please, that's not helping. The longer you withhold any information, the longer you'll have to to wait for good, rigged, models. Again, there are many people interested in this format, many thinking of Zelda hacks, many in Mario Kart hacking, people here, and myself as well.

If you choose to not release this information, go ahead. Just keep in mind that you're ceasing any opportunities for this possibility of a more functional importer & editor being created.

blank
Posted on 02-03-13 02:42 PM (rev. 2 of 02-03-13 02:49 PM) Link | #13245
Posted by SuperMario64DS
Posted by blank
Have you actually talked to BlackJax about this, or have you just heard some rumours? Because I don't really feel it's worth the time to write a full description if I'm the only one working on BDL editing tools anyway.


They have hinted support for other games before, Twilight Princess & such. Even so, this information should be readily available to anyone who needs it, there are other people within the Mario Kart community as well who have an interest in this format too.


I do agree that the BMD/BDL file format should be documented in such a way, but if there's nobody who actually needs it, I don't really feel like writing it at the moment.

Posted by NWPlayer123
This is kinda off-topic, but Twilight Princess can already be somewhat hacked, since Wind Maker supports Twilight Princess (to an extent), however, it uses BMD or BDL as well, so IDK why.
However, I do agree with blank being hesitant to fully figure all this stuff out.


It's not about figuring anything out. Everything but a few minor things has been figured out. It's just about writing documentation.

EDIT:
Posted by SuperMario64DS
Please, that's not helping. The longer you withhold any information, the longer you'll have to to wait for good, rigged, models. Again, there are many people interested in this format, many thinking of Zelda hacks, many in Mario Kart hacking, people here, and myself as well.

If you choose to not release this information, go ahead. Just keep in mind that you're ceasing any opportunities for this possibility of a more functional importer & editor being created.


If anybody is interested in working on BMD/BDL editing, they can contact me. And I'm not withholding any information, I just choose not to write it up in an easy accessible way. The most important parts of the documentation can already be found in the source of bmdview2 anyway.

dash
Posted on 02-08-13 04:52 PM Link | #14394
It's my intention to mess around with the bmdview2 codebase, I created a googlecode copy of it here:

http://code.google.com/p/bmdview2/

I don't know how long I'll mess with it until I run out of steam... at the moment I want to make it easier to navigate around the model, maybe use the mouse to "drag" the screen. Similiar to google earth and white hole.

Arisotura
Posted on 02-08-13 06:48 PM Link | #14427
Interesting!

Something that would definitely be lovely would be to combine editing/importing abilities into bmdview2.

Oh and if you need help with reverse-engineering stuff or whatever, you can try asking me. No guarantee I'll be able to answer, though. But I'm available.

____________________
NSMBHD - Kafuka - Jul
melonDS the most fruity DS emulator there is

zafkflzdasd

dash
Posted on 02-11-13 01:24 PM (rev. 2 of 02-11-13 01:24 PM) Link | #14886
Posted by Mega-Mario
Interesting!

Something that would definitely be lovely would be to combine editing/importing abilities into bmdview2.

Oh and if you need help with reverse-engineering stuff or whatever, you can try asking me. No guarantee I'll be able to answer, though. But I'm available.


I've managed to get the mouse control working pretty well in bmdview2. The codebase I'm starting from seems to have a few issues, as in the text overlays are not visible, and the gl shaders don't seem to be rendering everything correctly.

http://code.google.com/p/bmdview2/

As regards reverse engineering the bdl/bmd format, I wonder if the format makes use of bump mapping?

NWPlayer123
Posted on 02-11-13 04:30 PM Link | #14892
IDK anything about the Wii's GPU or rendering, but I'm wondering if the code is missing TEV stages. I know that Whitehole's rendering works almost perfectly for stuff except transparency like in Sky Station with that one planet that you go inside with the giant bullet bills.

____________________
"I hate playing musical chats" ~ Quote of the month

Arisotura
Posted on 02-11-13 06:13 PM Link | #14912
It's not. TEV stages are in a fixed number of 16. You should learn stuff about the Wii's GPU before saying that kind of stuff...

____________________
NSMBHD - Kafuka - Jul
melonDS the most fruity DS emulator there is

zafkflzdasd

dash
Posted on 02-11-13 07:17 PM Link | #14919
I realized the text overlay is off by default, you had to hit tab to get it to show up. I changed it so it's visible all the time.

The issue with the shader not rendering correctly... I realized it is rendering the textures but they're just very dark. So I think it's a lighting issue. Hitting the 'g' key causes the program to fallback to some other rendering approach (standard GL without shader language?) and that looks like it matches the real program more closely.

Here's an image with the shaders on:
[image]
Here's one with the shaders off (using the fallback rendering, whatever it is...)
[image]

blank
Posted on 02-12-13 01:45 AM Link | #14984
The dark textures might be caused by ColorChanInfo, which thakis didn't get quite right. It should be

struct ColorChanInfo
{
u8 enable;
u8 matColorSource;
u8 litMask;
u8 diffuseAttenuationFunc;
u8 attenuationFracFunc;
u8 ambColorSource;
u8 pad[2];
};

As for bump mapping that's done through indirect texturing. The information on indirect texturing is stored in MatIndirectTexturingEntry

struct IndTexOrder
{
u8 texcoord;
u8 texmap;
u16 padding0;
};

struct IndTexMatrix
{
f32 offset_matrix[2][3];
s8 scale_exponent;
u8 padding0[3];
};

struct IndTexCoordScale
{
u8 scale_s;
u8 scale_t;
u16 padding0;
};

struct TevIndirect
{
u8 indtex;
u8 format;
u8 bias;
u8 mtx;
u8 wrap_s;
u8 wrap_t;
u8 addprev;
u8 utclod;
u8 a;
u8 padding0[3];
};

struct MatIndirectTexturingEntry
{
u8 unknown0; // always 0 or 1
u8 unknown1; // unknown1 = unknown0, one of these are numindtexs
u16 padding0;

IndTexOrder indtexorder[4]; // arguments to GX_SetIndTexOrder
IndTexMatrix indtexmatrix[3]; // arguments to GX_SetIndTexMatrix
IndTexCoordScale indtexcoordscale[4]; // arguments to GX_SetIndTexCoordScale
TevIndirect tevindirect[16]; // arguments to GX_SetTevIndirect
};

dash
Posted on 02-13-13 11:21 AM (rev. 2 of 02-13-13 11:39 AM) Link | #15198
Posted by blank
The dark textures might be caused by ColorChanInfo, which thakis didn't get quite right. It should be

struct ColorChanInfo
{
u8 enable;
u8 matColorSource;
u8 litMask;
u8 diffuseAttenuationFunc;
u8 attenuationFracFunc;
u8 ambColorSource;
u8 pad[2];
};

I made that change in ordering but it didn't improve anything. For the scene I'm viewing (WoodCircleCutPlanet.bdl, which is in the BigTree2Galaxy in whitehole) both the enable and litMask fields are 0 in all cases.

I dug a bit just to understand the discussion of indirect texturing, found out bump maps are often done by making use of a height map (an array of scalers) by finding the partial derivatives in x and y to get the surface normal... and also to understand a little better the gamecube pipeline. But I'm not sure if this is associated with the dark texture issue.



blank
Posted on 02-13-13 02:31 PM (rev. 2 of 02-13-13 02:38 PM) Link | #15210
I took a quick look at the shader generation code, and rasterizer color selection isn't implemented. That could be the cause of the dark textures.

dash
Posted on 02-13-13 10:40 PM (rev. 3 of 02-13-13 11:10 PM) Link | #15289
Posted by blank
I took a quick look at the shader generation code, and rasterizer color selection isn't implemented. That could be the cause of the dark textures.


I'm not sure of that, I checked the fragment shader code at it is outputting a color c0:
vec4 c0 = vec4(-1.447059, -0.400000, -0.098039, 0.172549);
if I force "ONE" to be used instead of c0, the grass looks correct. The code is from the MAT 6 (numbered from 0). WoodCircleCutPlanet.bdl_frag_6.txt

the colorS10 pick array contains
colorS10: 0005000600020003
and the colors are these:
colorS10: 00ff00ff00ff0000 0000000000000170 00ff00ff00ff00ff 0000000000000000 fe8fff9affe7002c 006500650065ffcb 012f012f012f011b (7 many)

It's like the material is asking for color 5 there but is getting 4... that fe8f ff9a ffe7 002c is the c0 rgba values in signed 16 bit...
EDIT: Whoops I was looking at mat7. mat6 colorS10 pick is this:
colorS10: 0004000100020003
EDIT2:
If I make it so the starting color register c0 comes from the color3 pick array, the scene looks much nicer, and the green is perfect on the grass.
const Color16& c = mat.colorS10[currMat.color3[i==0?3:i-1]]; //XXXX ????? (sunflower.bmd)

as opposed to
const Color16& c = mat.colorS10[currMat.colorS10[i]]; //XXXX ?????

But that change makes the Mario.bdl model look too white and overexposed.

Not sure if I'm looking in the right place.

blank
Posted on 02-14-13 03:47 AM Link | #15325
After a considerably closer look I can say that it is definitely lighting that's the issue. That particular material has the alpha channel lit but not the RGB channel. But bmdview2 only checks whether the RGB channel is lit. That and lighting isn't really implemented.

dash
Posted on 02-14-13 01:30 PM (rev. 2 of 02-14-13 02:19 PM) Link | #15352
Posted by blank
After a considerably closer look I can say that it is definitely lighting that's the issue. That particular material has the alpha channel lit but not the RGB channel. But bmdview2 only checks whether the RGB channel is lit. That and lighting isn't really implemented.


I'm completely lost, I'm trying to figure out what you're saying here. I don't know which controls in the material you're speaking of, I don't grasp the precise usage of "lit" or "channel". I'm very familiar with opengl and shader language, but the gamecube/wii graphics pipeline is relatively new to me and the bmd graphics format is new also. The only document I've located that provides info on the gamecube pipeline is this:

http://www.amnoid.de/gc/tev.html

EDIT: Possibly your statements are connected to your definition of the ColorChanInfo struct you provided earlier. That has 2 fields, enable and litMask. In the WoodCircleCutPlanet.bdl the array of those structures has 5 entries:
colorChanInfo: 000100020100ffff 010104020100ffff 010000010000ffff 000000000200ffff 000000020100ffff (5 many)

The material 6 structure has this for chanControls:
chanControls (?): 0000000100020003

The bmdview2 source looks at chanControls[0] (the first entry, 0000). The whitehole source just skips over the chanControls array.
Pages: 1 2 3

Main - Archived forums - SMG documentations and tutorials - Model formats - A better understanding New reply

Page rendered in 0.031 seconds. (2048KB of memory used)
MySQL - queries: 30, rows: 236/236, time: 0.023 seconds.
[powered by Acmlm] Acmlmboard 2.064 (2018-07-20)
© 2005-2008 Acmlm, Xkeeper, blackhole89 et al.