nestelami / zapd Goto Github PK

Example: many in deku tree, eg CI8 texture in room 0 at 0xE1C0 uses palette in scene file at 0xB810

Idea on how to solve: I am not familiar with the code base, but I think TlutOffset should be gotten rid of and replaced with just a Tlut attribute that would point to the right palette using the Name attribute of the Texture node used in the xml file to describe it.
This could work in my mind because in the oot decomp repo the scene and map files are all in a single xml

Example of above idea with above example:

<Root>
    <File Name="ydan_scene" Segment="2">
        <Scene Name="ydan_scene"/>
        <Texture Name="dekuTreeWallTexTLUT" OutName="dekuTreeWallTexTLUT" Format="rgba16" Width="256" Height="1" Offset="0xB810"/>
    </File>
    <File Name="ydan_room_0" Segment="3">
        <Room Name="ydan_room_0" Offset="0x0"/>
        <Texture Name="dekuTreeWallTex" OutName="dekuTreeWallTex" Format="ci8" Width="32" Height="64" Offset="0xE1C0" Tlut="dekuTreeWallTexTLUT"/>
    </File>
    <!-- ... other rooms ... -->
</Root>

This would be where for example an object file references a display list from game_play keep. ZAPD should be able to manage this via a reference in the XML file for the object.

• Scenes/Rooms
• Animation
• Text
• Audio

ZGame should be a member of ZFile. Each ZResource should then require a ZFile parent.

Rozelette@130029c
Rozelette already started a WIP support for MM Scene/Room formats.

Zel states that MM animation systems have the same data as OOT.

in the oot repo both <Limb Type="..."> and <Limb LimbType="..."> are used

$ grep -r assets/xml/ -e 'Limb.*\bLimbType' | wc --lines
2169
$ grep -r assets/xml/ -e 'Limb.*\bType' | wc --lines
427

From https://github.com/zeldaret/ZAPD/blob/master/docs/zapd_extraction_xml_reference.md#limb I assume the correct one is LimbType

ZAPD should at least warn about a missing "LimbType" because I assume all those <Limb>s setting Type just have the Type value ignored

I assume LimbType defaults to Standard, everything setting Type uses Standard except object_link_child.xml which has some Type="LOD"

EDIT: I don't even think ZAPD knows what it's doing
https://github.com/zeldaret/ZAPD/blob/82bd27604a4c8f8c1f01f9c2f54c87fab103c39f/ZAPD/ZLimb.cpp#L381-L384
Does this mean Type and LimbType are "aliases", is that intended

Need a pipeline for ZAP to be able to take external data and recompile via XML files into .c files for inclusion into decomp.

• Scenes/Rooms
• Objects
• Text
• Audio

In many cases when ZAPD loads a file and the file can't be found, it will just crash. It should output a message like "file not found."

• ZAPD should also support XML file validation per asset type.
• ZAPD should error if an invalid ZResource name is included in the XML.
• ZAPD should error when a ZResource is not in a ZFile.
• ZAPD should error when a ZFile is encountered inside of a ZFile.
• ZAPD should error when a ZResource node has a child.
• ZAPD should error when invalid ZResource attributes and attribute values are provided.
• ZAPD should error if parameters are out of range. For example, a size parameter larger than the type used to store it in ZAPD.

As noticed by mdta on the zeldaret discord ( https://discord.com/channels/688807550715560050/688851317593997489/942010735922393129 ), the pngs extracted by ZAPD are a bit off. For example IA8 images where they should be fully opaque are only ~97% opaque

This may only be an issue with IA8 or on all formats I'm not sure.
Maybe fixing it will need tweaking the png to .inc conversion too to preserve matching, I don't know either.

For example here's how angrylion (a highly accurate video plugin) converts IA8 to rgba32:

https://github.com/ata4/angrylion-rdp-plus/blob/6660e93acc94506933238c9a593494684d0d3bde/src/core/n64video/rdp/tmem.c#L332-L337

            i = p & 0xf0;
            i |= (i >> 4);
            color->r = i;
            color->g = i;
            color->b = i;
            color->a = ((p & 0xf) << 4) | (p & 0xf);

which indeed doesn't match how ZAPD converts IA8:

https://github.com/zeldaret/ZAPD/blob/14d2a734b3fdddf2091629edc59db94ab5548f3e/ZAPD/ZTexture.cpp#L262-L263

ZAPD will always align every extracted variable to at least 4 bytes. This shouldn't be the case when None is used as stated in #109.

This change shouldn't be hard to do, but it would be a bit tedious to do and properly test. since None is used a lot, and many places may be a Align4 needed instead.

Originally posted by @Rozelette in #109 (comment)

Kenix

• string
• Vector (vec3s, vec3f, vec3i)
• Primitive (u8/s8, u16/s16, u32/s32, u64/s64, f32, f64)

Jack

• arrays
• Standard skeletons
• Flex skeletons
• Standard limbs
• LOD limbs
• Skin limbs
• Link animation headers
• Legacy animation system (object_human)

Currently CI4 textures do no extract correctly, however this presents an opportunity to improve the handling of CI textures as a whole.
CI textures should be changed to use .tga since its the only format that can handle palettes while also having an alpha channel. This way the images could actually look as intended as well, rather than just being grayscale.

How to reproduce:

https://github.com/zeldaret/oot/blob/8e4d1ccd3fc0e1f04fcbc0a02c1a4b8ae7f6e8d2/assets/xml/scenes/overworld/spot02.xml

add this in the scene file description:

        <Cutscene Name="spot02_scene_Cs_003C80" Offset="0x3C80"/>
        <Cutscene Name="spot02_scene_Cs_005020" Offset="0x5020"/>

spot02.xml now looks like:

<Root>
    <File Name="spot02_scene" Segment="2">
        <Scene Name="spot02_scene" Offset="0x0"/>
        <Cutscene Name="spot02_scene_Cs_003C80" Offset="0x3C80"/>
        <Cutscene Name="spot02_scene_Cs_005020" Offset="0x5020"/>
    </File>
    <File Name="spot02_room_0" Segment="3">
        <DList Name="gSpot02DL_0026D0" Offset="0x26D0"/>
        <Room Name="spot02_room_0" Offset="0x0"/>
    </File>
    <File Name="spot02_room_1" Segment="3">
        <Room Name="spot02_room_1" Offset="0x0"/>
    </File>
</Root>

./extract_assets.py runs fine, but then on make:

$ make
python3 tools/asm_processor/build.py tools/ido_recomp/linux/7.1/cc -- mips-linux-gnu-as -march=vr4300 -32 -Iinclude -- -c -G 0 -non_shared -Xfullwarn -Xcpluscomm -Iinclude -Isrc -Iassets -Ibuild -Wab,-r4300_mul -woff 649,838,712 -mips2 -O2 -o build/assets/scenes/overworld/spot02/spot02_scene.o assets/scenes/overworld/spot02/spot02_scene.c
cfe: Error: assets/scenes/overworld/spot02/spot02_scene.c, line 2519: 'spot02_sceneCutsceneData0x003C80' undefined; reoccurrences will not be reported.
     { SCENE_CMD_ID_CUTSCENE_DATA, 0, (u32)(spot02_sceneCutsceneData0x003C80)  } ,
 -------------------------------------------^
Failed to compile file assets/scenes/overworld/spot02/spot02_scene.c. Command line:

tools/ido_recomp/linux/7.1/cc -c -G 0 -non_shared -Xfullwarn -Xcpluscomm -Iinclude -Isrc -Iassets -Ibuild -Wab,-r4300_mul -woff 649,838,712 -mips2 -O2 -I /home/dragorn421/oot64decomp/oot/assets/scenes/overworld/spot02 -o build/assets/scenes/overworld/spot02/spot02_scene.o /tmp/preprocessedqklwqbwc.c

Makefile:217: recipe for target 'build/assets/scenes/overworld/spot02/spot02_scene.o' failed
make: *** [build/assets/scenes/overworld/spot02/spot02_scene.o] Error 55

indeed:

static SCmdBase spot02_sceneSet6C10[14] = {
    SCENE_CMD_SOUND_SETTINGS(0, 2, 127),
    SCENE_CMD_ROOM_LIST(2, spot02_sceneRoomList0x006CB0),
    SCENE_CMD_TRANSITION_ACTOR_LIST(2, spot02_sceneTransitionActorList_006C90),
    SCENE_CMD_MISC_SETTINGS(0x00, 0x00000002),
    SCENE_CMD_COL_HEADER(&spot02_sceneCollisionHeader_003C54),
    SCENE_CMD_ENTRANCE_LIST(spot02_sceneEntranceList0x006CC0),
    SCENE_CMD_SPECIAL_FILES(0x01, 0x0002),
    SCENE_CMD_PATH_LIST(spot02_scenePathway_006EB0),
    SCENE_CMD_SPAWN_LIST(1, spot02_sceneStartPositionList0x006C80),
    SCENE_CMD_SKYBOX_SETTINGS(1, 1, false),
    SCENE_CMD_EXIT_LIST(spot02_sceneExitList_006CC4),
    SCENE_CMD_ENV_LIGHT_SETTINGS(20, spot02_sceneLightSettings0x006CD4),
    SCENE_CMD_CUTSCENE_DATA(spot02_sceneCutsceneData0x003C80), // instead of spot02_scene_Cs_003C80 !
    SCENE_CMD_END(),
}; 

but

s32 spot02_scene_Cs_003C80[] = {
    CS_BEGIN_CUTSCENE(42, 1637),
    CS_PLAYER_ACTION_LIST(8),
    CS_NPC_ACTION(0x0011, 0, 20, 0x0000, 0x4000, 0x0000, 582, 180, 80, 582, 180, 80, 258413576, 0, 1),
    ...

As a temporary workaround I'll use spot02_sceneCutsceneData0x003C80 instead of spot02_scene_Cs_003C80 in the xml

There are a few objects which have non double word aligned textures. These objects will not match with our current method (u64 arrays for textures).

Full list of OoT objects with this problem:

• object_oF1s. Early goron. Unused.
• object_kw1. Red-Haired Kokiri Girl. Used in ending credits in Lon Lon Ranch.
• object_cne. Young Hylian Women.
• object_bji. Lon Lon credits and NPCs
• object_ahg. Hylian Men I. Lon Lon credits, uncursed HoS people, and NPCs
• object_boj. Hylian Men II. Happy Mask Shop Customer

MM needs to investigate if it has this problem too.

ZAPD is a complex and extensive piece of software now, extracting a diverse and interlinked set of data types. It is extremely difficult to learn to work in it in a nontrivial way: most of the time (over a week of) I spent on my first substantial PR was in understanding how the extraction process worked in the first place. And that was with @AngheloAlf 's help.

This is especially true of what all the ambiguously-named functions actually do, and when. (For example, ExtractFromXML could mean that it's extracting using an XML file, or that it's extracting data from the XML file itself).

Likewise, there are many similarly-named functions that appear to do similar, but not identical things: the various functions with Source and Output in seem to be used in several different ways by different ZResources. This was particularly difficult for writing a new one, since it's not clear what best practice should be.

The extraction loops should be documented (this would fix the first paragraph), every file should have a description of the format of the data it extracts and any peculiarities and caveats required by the specific type noted. Individual functions should be documented where appropriate (which would address the second two paragraphs): this seems like it should especially be true of the top-level classes like ZResource; #178 is a good start on this, but more needs to be done, especially if #14 and #15 are going to be implemented in the coming months. It should be possible to write a new ZResource or similar new functionality based on what is actually intended, rather than guessing based on the imperfect and inconsistent way in which things are currently arranged.

ZAPD should also support PAL and NTSC message systems.

Clang is

• faster
• more efficient
• has more, better and more descriptive warnings and errors
• has more extensive defaults for warnings
• is already the default for Macs

It would be nice to have a configuration file for struct names used by ZAPD to write out assets in C, like SkeletonHeader, CamData, ...

This would make documentation easier by not having to first wait for ZAPD to be updated and merged into decomp repos

Edit: Following discussion on Discord about Jenkins difficulties this would bring, the idea would be:

Have config.xml with <StructName id="FlexSkeletonHeader" name="MyUpdatedFlexSkeletonHeaderStructName_FOR_OOT"> entries somewhere in the oot repo (duplicate of config_oot.xml in the zapd repo).
Have config.xml with <StructName id="FlexSkeletonHeader" name="MyUpdatedFlexSkeletonHeaderStructName_FOR_MM"> entries somewhere in the mm repo (duplicate of config_mm.xml in the zapd repo).
(config.xml would be added on any zapd merge following implementing config)

Whenever oot/mm changes a struct name present in config.xml, update it in the config too.

When zapd uses a new struct name, it will be added to the oot/mm repo on next zapd merge by editing config.xml, and added immediately to zapd's config_oot.xml/config_mm.xml.

Jenkins (for the zapd repo) will overwrite oot/mm's config.xml with config_oot.xml/config_mm.xml when building zapd to allow development builds to update config.xml.

This may also be true for other things than Blob, didn't test it

https://github.com/zeldaret/oot/blob/master/assets/xml/objects/gameplay_field_keep.xml#L58
<Blob Name="gFieldUnaccounted_00CB30" Size="0x800"/>

I would expect this to fail on extraction, or at least warn about it (missing Offset attribute), but ZAPD says nothing.

The Name does end up being used (changing it also changes the extracted .c), so ZAPD must be doing some nice magic to still find the right offset for the data.

Good job ZAPD but I guess this could be messed up by moving nodes around in the xml?
I tried moving the Blob without Offset up in the file and then it got silently ignored

Currently color index textures are virtually impossible to modify since afaik there isn't any easy way to generate a palette for these. To solve this I had the idea of making zapd generate a palette on its own for any indexed texture with the .ci4 or .ci8 extension, so users could them just #include a .ci4.tlut for example that zapd would generate from the .ci4 png, removing the need for users to manually create a palette (or even know how palettes work at all). Another important thing is that zapd, hopefully with libpng should be able to convert a normal png into an indexed png in case the png is not already set up that way, although it should not have to actually make the png be 16 colors etc since that should be up to the user imo. I also think that zapd should be able to extract pngs in this way, and detect if a palette is not being shared with another texture, and just include the palette from the png itself.

Currently we have some assets that are being #included with a ../ at the beginning of the path and other that doesn't have that.

Example:

u16 object_bombiwa_TLUT_000000[] = {
    #include "assets/objects/object_bombiwa/tlut_00000000.rgb5a1.inc.c"
};

u8 object_bombiwa_Tex_000020[] = {
    #include "assets/objects/object_bombiwa/tex_00000020.ci4.inc.c"
};

static Vtx object_bombiwaVtx_000820[] = {
    #include "../assets/objects/object_bombiwa/object_bombiwaVtx_000820.vtx.inc"
};

Also, those #include should not be indented.

This is not meant to be a 100% exhaustive list.

• XML handle may not be closed.
• File handles may not be closed.
• Strings that are not stack variables need to be freed. Some come from XML, and it may handle freeing of those. We need to consult documentation.
• ZFile.resources needs to free and destroy the ZResource on ZFile destructor.
• ZFiles need to be destroyed after the XML file is done being processed. They are allocated memory in a few places, but never freed.
• ZCollision allocates VertexEntry classes into the vertices vector and never frees them.
• ZCollision allocates PolygonEntry classes into the polygons vector and never frees them.
• ZCollision allocates a CameraDataList and never frees it.
• ZCollision allocates WaterBoxHeader classes into the waterBoxes vector and never frees them.
• ZCutscene allocates commands into the commands vector and never frees them.
• ZDisplayList allocates ZDisplayLists into the otherDLists vector and never frees them.
• File.ReadAllBytes allocates a char array that is never freed.
• File.ReadAllText allocates a char array that is never freed.
• SetListActor allocates ActorSpawnEntry to the actors vector and never frees them.
• SetCutscenes allocates ZCutscene to the cutscenes vector and never frees them.
• SetEntranceList allocates EntranceEntry to the entrances vector and never frees them.
• SetLightSettings allocates LightingSettings to the settings vectors and never frees them.
• SetMesh allocates MeshHeader and other objects, but never frees them.
• SetPathways allocates PathwayEntry to the pathways vector and never frees them.
• SetRoomList allocates RoomEntry to the rooms vector and never frees them.
• SetStartPositionList allocates ActorSpawnEntry to the actors vector and never frees them.
• SetTransitionActorList allocates TransitionActorEntry to the transitionActors vector and never frees them.
• ZRoom.ExtractFromXML allocates multiple objects and never frees them.
• ZRoom allocates room commands to the commandList vector and never frees them.
• ZLimbStandard.FromRawData allocates a ZLimbStandard that is never freed or added to the ZResource list in a ZFile.
• ZLimbLOD::FromRawData allocates a ZLimbLOD that is never freed or added to the ZResource list in a ZFile.
• There are ZDisplayList allocations throughout the code that are never freed or added to the ZResource list in a ZFile.
• Final audit of all opened file handles, xml handles, and all uses of the "new" operator for a later delete.

ZResource and it's descendent classes should use more polymorphism. There is not a huge amount of consistency in method names right now for the extraction code entry point. This should be an overridden method from ZResource for each type.
Further, the code in ZFile.ParseXML for each type should be moved to a method in ZResource and overridden methods.

Some files, such as the error messages in MM's sys_initial_check, require that the dimensions and filesizes of the textures are used in the code. To be able to do this most coherently,

• ZAPD ought to expose both the texture's dimensions and its size for external use.
• These could also be used in displaylists that use textures.

This would keep the hardcoding of sizes to a minimum. Exposure will probably be done via a set of defines for each texture, in the asset file's header (or more suitable place in the case of static textures), of the form [...]_WIDTH, [...]_HEIGHT, [...]_SIZE If suitable string manipulation is not available or too costly, appending these to the variable name should be sufficient to avoid clashes.

Currently ZAPD will extract asset files like this:

Gfx sOtyukaPlatformBottomDL[] = {
    gsSPVertex(&sOtyukaPlatformBottomVtx[0], 4, 0),
    gsSP2Triangles(0, 2, 1, 0, 0, 3, 2, 0),
    gsSPEndDisplayList(),
}; 

Assets that are currently in the repo have things like this:

 Gfx sPlatformBottomDList[] = {
    gsSPVertex(sPlatformBottomVerts, ARRAY_COUNT(sPlatformBottomVerts), 0),
    gsSP2Triangles(0, 2, 1, 0, 0, 3, 2, 0),
    gsSPEndDisplayList(),
};

Another example of this

CollisionHeader sOtyukaCollision = { 
    { -60, -60, -60 },
    { 60, 0, 60 },
    8,
    sOtyukaCollision_vtx_00002540,
    10,
    sOtyukaCollision_polygons_000024A0,
    sOtyukaCollision_surfaceType_00002490,
    sOtyukaCollision_camDataList_00002488,
    0,
    NULL
}; 

Is being extract instead of:

static CollisionHeader sColHeader = {
    { -60, -60, -60 }, { 60, 0, 60 },    ARRAY_COUNT(sVtxList), sVtxList, ARRAY_COUNT(sPolyList),
    sPolyList,         sSurfaceTypeList, sCameraDataList,       0,        NULL,
};

for example in oot decomp's assets/scenes/test_levels/test01/test01_room_0.c

Gfx test01_room_0DL_006438[] = {
    gsSPDisplayList(0x03002220),
    gsSPDisplayList(0x03003198),
    gsSPDisplayList(0x03003628),
    ...

should be

Gfx test01_room_0DL_006438[] = {
    gsSPDisplayList(test01_room_0DL_002220),
    gsSPDisplayList(test01_room_0DL_003198),
    gsSPDisplayList(test01_room_0DL_003628),
    ...

find (many) more examples with searching regex gsSPDisplayList\(0x0[0-6] under assets/ (eg with VSCode or grep -r assets -e 'gsSPDisplayList(0x0[0-6]')

@AngheloAlf said on discord this is fixed by #151 , that would need confirming

The link_animetion file contains player animations in a different format to the usual animations found in objects. The headers for these animations are located in gameplay_keep and are already extracted as PlayerAnimations. These contain the frame count and the segment start for the animation data in link_animetion.
At each segment, there is (sizeof(Vec3s) * limbCount + 2) * frameCount worth of s16 limb rotation data. Player has 22 limbs after adding 1 in SkelAnime_InitFlex, so this is 0x86 * frameCount worth of data.
I believe then a suitable form for the link_animetion file's xml is:
<PlayerAnimationData Name="gPlayerAnimData_00D710" FrameCount="51" Offset="0x00D710"/>

ZAPD is currently adding the offset as a comment some extracted arrays (See image).

This should be removed or changed to be a "debug feature", for example, passing a parameter like --debug-offsets when invoking ZAPD.

The following OP codes are in yet supported:
// Opcode 0xE4 unimplemented!
// Opcode 0xE4 unimplemented!
// Opcode 0xF1 unimplemented!

XML to recreate can be found here:
https://github.com/louist103/oot/blob/ObjectDecomp/assets/xml/objects/gameplay_field_keep.xml
commit e659d71

While searching how Impa's working I searched in the scene where the actor is used, and I noticed that the latest alternate header of the spot00 (hyrule field) scene (spot00_room_0ActorList_001D40) has a glitched actor list that extends after what should be the end of the array

Kenix had the idea of allowing new data structures to be defined in a Game.xml. These would take advantage of the existing ZResource types, especially primitives and vectors. This should make ZAP a bit more extendable, allowing it to mirror structs in the game codebase.

For the declaration system, ZAPD needs to make use of the proper functions instead of directly accessing the std::map. There are still a few uses of the latter that need to be removed.

In objects, if data is auto generated it should be static and have an s prefix.
Only the variables specified in the XML need to be accessed externally. This is important for symbol name collisions.

ZAPD used .cfg files when a texture was compiled into a .inc.c and required a different variable name. This is no longer needed since .inc.c files don't contain the variable anymore, so ZAPD should stop generating them when extracting.

The second argument to SCENE_CMD_SPECIAL_FILES is an object id, in vanilla 0 (OBJECT_INVALID), 2 (OBJECT_GAMEPLAY_FIELD_KEEP) or 3 (OBJECT_GAMEPLAY_DANGEON_KEEP)

It could in theory be any object id I think

The issue is ZAPD writes a number instead of the appropriate enum constant, this prevents from shifting the whole object table

Macro definition (include/z64scene.h)

#define SCENE_CMD_SPECIAL_FILES(elfMessageFile, keepObjectId) \
    { SCENE_CMD_ID_SPECIAL_FILES, elfMessageFile, CMD_W(keepObjectId) }

Relevant struct for the command (include/z64scene.h)

typedef struct {
    /* 0x00 */ u8  code;
    /* 0x01 */ u8  cUpElfMsgNum;
    /* 0x04 */ u32 keepObjectId;
} SCmdSpecialFiles;

Relevant scene command handler (src/code/z_scene.c)

void func_8009883C(GlobalContext* globalCtx, SceneCmd* cmd) {
    if (cmd->specialFiles.keepObjectId != 0) {
        globalCtx->objectCtx.subKeepIndex = Object_Spawn(&globalCtx->objectCtx, cmd->specialFiles.keepObjectId);
        gSegments[5] = VIRTUAL_TO_PHYSICAL(globalCtx->objectCtx.status[globalCtx->objectCtx.subKeepIndex].segment);
    }

These are primarily used in pre-rendered rooms.
We should scrub any JFIF metadata before compiling back into a ROM.

Ocarina of Time (and probably Majora's Mask) have many duplicated textures across objects, rooms, and scenes. We need to be able to detect these duplicates and have them all stored to a common location.

Now:

StandardLimb object_daiku_Limb_007900 = { 
    { 1950, 0, 0 }, 0x0F, 0xFF,
    object_daiku_DL_005CF0
};

0xFF is known as LIMB_DONE in oot z64animation.h

More complicated, but eventually we probably want to also use enums for all limb indices. Ideally use user-made ones.
I don't think there's much that can be done on that front before we move to commiting assets headers onto the repo

Ideally:

StandardLimb object_daiku_Limb_007900 = { 
    { 1950, 0, 0 }, LIMB_OBJECT_DAIKU_0F, LIMB_DONE,
    object_daiku_DL_005CF0
};

ZAPD currently is able to detect those in scenes, but they can't be declared in an XML file for objects.

One example is object_death at offsets 0xCB84 and 0xCBC0.

These are things that I wasnt able to do myself, and would ideally be completed asap so we can get the updates into OoT's master. Posted this in discord already but posting here so it doesnt get lost.

• I misspelled Indices as Indicies in the last commit
• use 0x%06X for all segmented address offsets
• remove some more snake case and leading underscores in var names (tex and vtx especially. but can leave it for offsets)
• make data that is not xml defined as static (especially important for symbol collisions)
• clean the git history

Static should have a manual enable for things like overlay assets.

Solution TBD

When outputting vertices, ZAPD should use the Vtx union instead of Vtx_t. This is the type intended for use and it will ensure vertices are 8-bytes aligned for dma.
In OoT, we also use the VTX macro to define vertices, so ZAPD should probably output it as well. It includes the double { needed when using Vtx, and it sets the flag to 0 since it's always 0 as far as we know.

Currently, ZAPD generates many small unaccounted arrays of 0s. Most of these could be omitted when they are just padding for end of file alignment (to 0x10 bytes) or long alignment (to 8 bytes) when the next element is a texture (u64[]), vertices (Vtx[]) or display lists (Gfx[]).

Note this is a non-issue for the vanilla games but it should probably be addressed as this behavior is weird, and should probably result in an error instead

Consider (OoT) object_daiku_TLUT_001AC0 in assets/xml/objects/object_daiku.xml (or any other texture, but TLUTs make more sense here since they aren't necessarily nice rectangles)

With

<Texture Name="object_daiku_TLUT_001AC0" OutName="tlut_00001AC0" Format="rgba16" Width="256" Height="1" Offset="0x1AC0"/>

the ROM builds OK

assets/objects/object_daiku/object_daiku.c excerpt

AnimationHeader object_daiku_Anim_001AB0 = { 
    { 59 }, object_daiku_Anim_001AB0FrameData,
    object_daiku_Anim_001AB0JointIndices, 10
};

u64 object_daiku_TLUT_001AC0[] = {
#include "assets/objects/object_daiku/tlut_00001AC0.rgba16.inc.c"
};

u64 object_daiku_Tex_001CC0[] = {
#include "assets/objects/object_daiku/tex_00001CC0.ci8.inc.c"
};

build/assets/objects/object_daiku/tlut_00001AC0.rgba16.inc.c

    0x0000FFE5A495FCED, 0x41432883F7A3FFFF, 0xC5DB080160478849, 0xDE9FAD1728C31A23,  // 0x000000 
    0x11E309A12AA5EF63, 0xBD99EEE1F6DDFB63, 0xB4C939C1F61911E1, 0xFFA17849E48FEF79,  // 0x000020 
    0x2801FFBDDFBD98CB, 0x09E1FCABFFFD730D, 0x015FF7A5F7BB4003, 0x684739056ACB2083,  // 0x000040 
    0x1801200131053003, 0xE6E1F77B4145D65D, 0x72892A65FFA3ED53, 0x49879411E7211A63,  // 0x000060 
    0xED55CEB72223E739, 0xDCEBB04B838FFBE7, 0x9049008F88C9CE1D, 0x6A4B494594532263,  // 0x000080 
    0x5047A4D5095F30C3, 0x5187B8CB7B0BA991, 0x2A63FA19FF1FFFA5, 0x6A0B48438A8D4803,  // 0x0000A0 
    0x31030915D513F155, 0xCA15DA157C21FB1F, 0xFA9DBDDBD6B5DEF9, 0x1A65BD9B2265FC67,  // 0x0000C0 
    0xCE75D69DD69F011F, 0xADAFDE2143291223, 0x00D9D499B559C557, 0xA4D7CD57FB1D8C11,  // 0x0000E0 
    0x09591159ED95AA59, 0x119DD195D61BD29B, 0xB415C599CD1BC597, 0xBC17F617BD59E29B,  // 0x000100 
    0xB31B8B8FCCD3B35D, 0xCD5DD55D00D5FB9F, 0xC4D3AB93A39319DF, 0xB993EE9DC951C451,  // 0x000120 
    0xEF1FAC51A451824D, 0xF69D00D3E111ABCF, 0xA38DFF61BE31A8CB, 0x9CA5E6E5724BEF25,  // 0x000140 
    0xD0CD4147EF37EF77, 0xC5D7820BA8CD3AE7, 0x69C9A90D7A896249, 0xB10D628BC8CD824B,  // 0x000160 
    0x8B0B32E732A76987, 0x6A07B5F1C631A52B, 0x59C9DD6DB3D5011B, 0xFA5BEF216A4751C9,  // 0x000180 
    0x40C338C369894985, 0xBE3339039CE970C9, 0x50C9FDF3DF396105, 0x0001000100010001,  // 0x0001A0 
    0x0001000100010001, 0x0001000100010001, 0x0001000100010001, 0x0001000100010001,  // 0x0001C0 
    0x0001000100010001, 0x0001000100010001, 0x0001000100010001, 0x0001000100010001,  // 0x0001E0 

build/assets/objects/object_daiku/tex_00001CC0.ci8.inc.c

    0x0101010101010101, 0x0101010101010101, 0x0101010101010101, 0x1C1C1C1C1C1C1C1C,  // 0x000000 
    0x1616161616161616, 0x1A1A1A1A1A1A1A1A, 0x40403B3B403B403B, 0x1E1E1E1E1E1E1E1E,  // 0x000020 

With

<Texture Name="object_daiku_TLUT_001AC0" OutName="tlut_00001AC0" Format="rgba16" Width="255" Height="1" Offset="0x1AC0"/>

(Width="255")

the ROM doesn't build OK

$ ./first_diff.py
First difference at ROM addr 0x176BCBC, object_daiku_TLUT_001AC0 (RAM 0x6001AC0, ROM 0x176BAC0, build/assets/objects/object_daiku/object_daiku.o)
Bytes: 00:01:00:00 vs 00:01:00:01

1 differing word(s).

assets/objects/object_daiku/object_daiku.c excerpt

AnimationHeader object_daiku_Anim_001AB0 = { 
    { 59 }, object_daiku_Anim_001AB0FrameData,
    object_daiku_Anim_001AB0JointIndices, 10
};

u64 object_daiku_TLUT_001AC0[] = {
#include "assets/objects/object_daiku/tlut_00001AC0.rgba16.inc.c"
};

u64 object_daiku_Tex_001CC0[] = {
#include "assets/objects/object_daiku/tex_00001CC0.ci8.inc.c"
};

build/assets/objects/object_daiku/tlut_00001AC0.rgba16.inc.c

    0x0000FFE5A495FCED, 0x41432883F7A3FFFF, 0xC5DB080160478849, 0xDE9FAD1728C31A23,  // 0x000000 
    0x11E309A12AA5EF63, 0xBD99EEE1F6DDFB63, 0xB4C939C1F61911E1, 0xFFA17849E48FEF79,  // 0x000020 
    0x2801FFBDDFBD98CB, 0x09E1FCABFFFD730D, 0x015FF7A5F7BB4003, 0x684739056ACB2083,  // 0x000040 
    0x1801200131053003, 0xE6E1F77B4145D65D, 0x72892A65FFA3ED53, 0x49879411E7211A63,  // 0x000060 
    0xED55CEB72223E739, 0xDCEBB04B838FFBE7, 0x9049008F88C9CE1D, 0x6A4B494594532263,  // 0x000080 
    0x5047A4D5095F30C3, 0x5187B8CB7B0BA991, 0x2A63FA19FF1FFFA5, 0x6A0B48438A8D4803,  // 0x0000A0 
    0x31030915D513F155, 0xCA15DA157C21FB1F, 0xFA9DBDDBD6B5DEF9, 0x1A65BD9B2265FC67,  // 0x0000C0 
    0xCE75D69DD69F011F, 0xADAFDE2143291223, 0x00D9D499B559C557, 0xA4D7CD57FB1D8C11,  // 0x0000E0 
    0x09591159ED95AA59, 0x119DD195D61BD29B, 0xB415C599CD1BC597, 0xBC17F617BD59E29B,  // 0x000100 
    0xB31B8B8FCCD3B35D, 0xCD5DD55D00D5FB9F, 0xC4D3AB93A39319DF, 0xB993EE9DC951C451,  // 0x000120 
    0xEF1FAC51A451824D, 0xF69D00D3E111ABCF, 0xA38DFF61BE31A8CB, 0x9CA5E6E5724BEF25,  // 0x000140 
    0xD0CD4147EF37EF77, 0xC5D7820BA8CD3AE7, 0x69C9A90D7A896249, 0xB10D628BC8CD824B,  // 0x000160 
    0x8B0B32E732A76987, 0x6A07B5F1C631A52B, 0x59C9DD6DB3D5011B, 0xFA5BEF216A4751C9,  // 0x000180 
    0x40C338C369894985, 0xBE3339039CE970C9, 0x50C9FDF3DF396105, 0x0001000100010001,  // 0x0001A0 
    0x0001000100010001, 0x0001000100010001, 0x0001000100010001, 0x0001000100010001,  // 0x0001C0 
    0x0001000100010001, 0x0001000100010001, 0x0001000100010001, 0x0001000100010000,  // 0x0001E0 

build/assets/objects/object_daiku/tex_00001CC0.ci8.inc.c

    0x0101010101010101, 0x0101010101010101, 0x0101010101010101, 0x1C1C1C1C1C1C1C1C,  // 0x000000 
    0x1616161616161616, 0x1A1A1A1A1A1A1A1A, 0x40403B3B403B403B, 0x1E1E1E1E1E1E1E1E,  // 0x000020 

So with both XMLs, the object_daiku is the same size and same organization, the only difference is the last byte in object_daiku_TLUT_001AC0
I suspect the difference is actually two bytes (the last rgba16 value would be two bytes) but it can't be observed here

So the issue is those two bytes should be read from the file instead of being 0 (or garbage that happens to be 0 here idk)
Or ZAPD should fail to extract with a message that a rgba16 texture needs a multiple-of-4 amount of pixels

I assume the same happens with other formats, with other values/amount of data lost.

Returning a string is a legacy feature back before a declaration system was in place. The function would return the string with the C source code, but this is (for the most part) no longer the case, so we shouldn't be returning a string anymore.

Recently MM added ZAPD to their compilation process, so we should be checking we don't break MM support.

Cf title, jabu rooms and boss room use a raw vram pointer instead of a symbol

All the files involved:

assets/scenes/dungeons/bdan/bdan_room_2.c
assets/scenes/dungeons/bdan/bdan_room_8.c
assets/scenes/dungeons/bdan/bdan_room_9.c
assets/scenes/dungeons/bdan/bdan_room_10.c
assets/scenes/dungeons/bdan/bdan_room_11.c
assets/scenes/dungeons/bdan/bdan_room_12.c

assets/scenes/dungeons/bdan_boss/bdan_boss_room_1.c

May explain/play a role in this issue ZeeRoX had ( https://discord.com/channels/688807550715560050/688851337085190255/924427701853511680 ):

This was brought to my attention by zel