Sorry babe, I’m an ambitious girl and you, you’re just small time.

In this part of the series we’ll take a look at how cutscenes are handled. We’ll skip the animation implementation for now, as I haven’t figured out the final details on how I want to import them into Godot yet.

Cutscenes in GTA are all rendered in real time in engine. You might have never seen them, but the locations that are used in cutscenes all exist somewhere on the map.

For example the famous intro cutscene of GTA: III happens around here, cleverly hidden behind the mountains:

And GTA: VC’s intro cutscene actually plays inside the Ocean View Hotel. Hidden by Vice Cities new interior system.

The script

To see how those cutscenes are set up we’ll have to dive into the source of the mission script.

If we take a look at GTA: III intro.sc file, we see the following lines:

// **********************************START OF BANK CUTSCENE****************************
MAKE_PLAYER_SAFE_FOR_CUTSCENE Player

SET_INTRO_IS_PLAYING TRUE

LOAD_COLLISION LEVEL_GENERIC
LOAD_SPECIAL_CHARACTER 1 cat
LOAD_SPECIAL_CHARACTER 2 colrob
LOAD_SPECIAL_CHARACTER 3 miguel
LOAD_SPECIAL_CHARACTER 4 playerx
LOAD_SPECIAL_MODEL cut_obj1	cs_ban
LOAD_SPECIAL_MODEL cut_obj2 bankd
LOAD_SPECIAL_MODEL cut_obj3	cs_loot
LOAD_SPECIAL_MODEL cut_obj4	colt1
LOAD_SPECIAL_MODEL cut_obj5	cath

We’ll go into the details of the scripting language in another post (there is so much stuff to write about for this game!), for now let’s go over these opcodes one by one.

• MAKE_PLAYER_SAFE_FOR_CUTSCENE Player makes sure that nothing happens to the player during the cutscene. This prevents things like the player dying during the cutscene.
• SET_INTRO_IS_PLAYING sets a flag indicating that the intro is playing, we don’t know what this does exactly, but I assume other pieces of the engine use this to determine if the intro is playing.
• LOAD_COLLISION loads collision data.
• LOAD_SPECIAL_CHARACTER 1 cat loads the character model cat into special character slot 1.
• LOAD_SPECIAL_MODEL cut_obj1 cs_ban loads cs_ban into cutscene object slot cut_obj1.

Let’s skip a little bit further in the script to the actual cutscene stuff:

LOAD_CUTSCENE bet
LOAD_SCENE -559.65 1030.56 40.0

SET_CUTSCENE_OFFSET -537.42 1051.204 36.884

CREATE_CUTSCENE_OBJECT PED_SPECIAL4 cs_player
SET_CUTSCENE_ANIM cs_player playerx

// More cutscene object setup

CREATE_CUTSCENE_HEAD cs_cat cut_obj5 cs_cathead
SET_CUTSCENE_HEAD_ANIM cs_cathead cat

// More cutscene object setup

START_CUTSCENE

Again let’s go over the opcodes one by one.

• LOAD_CUTSCENE Loads the cutscene data from anim/cuts.img, this includes:
  • bet.ifp the key frame animation data
  • bet.dat the camera movement data
  • bet_cat.anm Catalina’s head animation data
• LOAD_SCENE Tells the engine to load the scene at a specific location
• SET_CUTSCENE_OFFSET Cutscene data is based on a local coordinate system (0.0, 0.0, 0.0). To play the cutscene at the correct location an offset is applied.
• CREATE_CUTSCENE_OBJECT PED_SPECIAL4 cs_player Creates a new cutscene object with model PED_SPECIAL4 stored in variable cs_player
• SET_CUTSCENE_ANIM cs_player playerx Applies the playerx animation from bet.ifp to the cs_player cutscene object
• CREATE_CUTSCENE_HEAD cs_cat cut_obj5 cs_cathead Creates the cath head object on the cs_cat cutscene object and stores it in cs_cathead
• SET_CUTSCENE_HEAD_ANIM Applies the cat animation from bet_cat.anm to Catalina’s head.
• START_CUTSCENE Starts the cutscene.

That’s quite some setup to play a single cutscene! To replicate the cutscene we have to do the same steps in Godot. In the next few posts I’ll describe the steps to get this cutscene playing.

The map

First of all we have to make sure the map is loaded, the intro cutscene is located at -559.65 1030.56 40.0 which should load if we load the map data from the landsw directory.

Rendering is a bit off, but it’s good enough for now!

In the next post we’ll take a look at camera movement data found in bet.dat.

It’s time to discuss textures. The way materials are handled in the Trilogy are much simpler then materials these days. No fancy normal maps or specular maps. Just a simple texture and perhaps an alpha texture for some translucent models.

Textures are stored in TXD files aka TeXture Dictionary.

ℹ️

Fun fact, GTA IV uses similar files but calls them WTD on Windows (Windows Texture Dictionary) and uses a different first character depending on the platform.

These texture dictionary files use the same format as DFF files as they are RenderWare based. Let’s have a look at a TXD file.

File structure

Because TXD files are just RenderWare files, we can view the file structure in the same tools as DFF files.

The tree of LoadSC0 looks like this:

RwTextureDictionary
├── RwTextureNative
│   └── RwExtension
└── RwExtension

TextureDictionary

This is the root section of TXD files, it contains the number of textures in the archive.

TextureNative

For each texture inside the archive there is a TextureNative section. This section contains info about the format of the texture data and the texture data itself.

Texture format

In this section we will go over some of the texture formats. A detailed description of the possible texture formats can be found at gtamods.com.

Mapping most of the texture formats to their Godot equivalent is quite easy:

⚠️️ Note: This table is incomplete.

• * We’ll get to this one in a later post
• ** See section on palette textures below

With that in mind it’s pretty straightforward to turn TXD entries into Godot textures.

// texData contains our texture native section struct
int mipMapLevel = 0
    
ImageTexture.CreateFromImage(
    Image.CreateFromData(
        (int)texData.width,
        (int)texData.height,
        false, // Set depending on mipmap flags
        texData.getFormat(), // Performs the mapping from the above table
        texData.textureData[mipMapLevel] // Perform this for each mipmap level
    ) 
)

Palette Textures

If we take a look at GTA: III we notice that a lot of textures use a palette. This means that the texture is split into 2 pieces of data. The regular texture data contains an array of indices in a range of 0 - 255. These indices point to the palette texture which contain the actual color data. It’s as simple as paint by number, lets take a look at the following example:

Index texture		Palette texture		Result
	+		=

Godot (and modern graphic cards) don’t support this natively. We have three options to solve this:

1. Transform the texture data at runtime. Create a RGB(A) byte buffer that has the size of the texture, loop through all indices and set the color accordingly.
2. Transform the texture data at first run and store it on the users harddrive in the new format.
3. A custom shader that performs the lookup.

In my implementation I went for option #3.

We’ll have to create a shader that takes 2 textures. One for the index texture and one for the palette texture.

uniform sampler2D index_texture : filter_nearest; // Using linear would mess up the indices
uniform sampler2D palette_texture : source_color;

Ideally our index texture would be of type usampler2D as our index range is an uint between 0 and 255, but due to a bug in Godot this is not possible, instead we’ll need to transform the index to an uint inside the shader.

To look up the index of the color pixel we have to retrieve the value from the index_texture. Due to the previously mentioned bug we then need to multiply the result by 255.0:

uint tex_index = uint(texture(index_texture, UV).r * 255.0);

With the index we can now perform a lookup of the actual color that is used inside the palette texture.

Instead of texture which takes normalized UV coordinates, we can use texelFetch which takes pixel coordinates.

vec3 tex_color = texelFetch(palette_texture, ivec2(int(tex_index), 0), 0).rgb;

And that’s pretty much it. My final shader code:

shader_type spatial;

render_mode cull_front;

uniform sampler2D index_texture : filter_nearest;
uniform sampler2D palette_texture : source_color;
uniform vec3 albedo_color : source_color = vec3(1.0, 1.0, 1.0);

void fragment() {
    uint tex_index = uint(texture(index_texture, UV).r * 255.0);
    vec3 tex_color = texelFetch(palette_texture, ivec2(int(tex_index), 0), 0).rgb;
	
    vec3 material_color = albedo_color.rgb;

    ALBEDO = tex_color * material_color;
}

And the result:

My problem

We got some nice visuals, but when we place our “player” in front of the golf course, he’ll just fall through the map.

To solve this we need to load the collision data for each map object, but how and when is this loaded? When we looked at the map files there was no mention of collision data except for these two lines:

COLFILE 0 MODELS\COLL\VEHICLES.COL
COLFILE 0 MODELS\COLL\WEAPONS.COL

Looking at the name of those collision files I don’t expect them to contain golf collision data except for a Caddy and Golfclub.

My guess

I do have an idea of how collision data is loaded, but that’s all based on the following best guesses: Back in the day when I was into modding, I had to make quite a few custom collisions. There are a couple of steps you had to take to add collision to your model:

1. Create a coll archive
2. Add a coll entry with the same name as your model
3. Add the coll archive to the IMG archive

That seems easy enough, the game just looks for a coll entry with the same name as our model. But, when does the game load these?

Well, back in the day it was also not uncommon to accidentally create a corrupt collision file. When you imported a corrupt coll archive the game would crash during loading with a nice popup similar to this one.

My guess; during loading, all collision archives inside the IMG file are read and loaded into memory or at least put into some kind of file table.

With the how and when figured out we can try to do the same in our implementation.

My solution

After loading an IMG file, I search for all .col entries in the file table and load them into memory, in my case a Dictionary for easy model name to collision lookup.

Something like the following pseudocode.

private Dictionary<string, Coll> _collisionData = new();

public void Load() 
{
    Image img = LoadImg("gta3.img");
    var collisionEntries = img.FindByType("col");
    foreach (var entry in collisionEntries) 
    {
        var collFile = LoadColFile(img, entry);
        // Add all entries in the collFile to the dictionary
    }
}

And as always the result:

⚠️️ A little disclaimer:

Godot currently doesn’t provide streaming capabilities out of the box. But, the games we try to simulate are very old and hardware has improved immensely in the past 25(!) years. So we might actually get away with just rendering the map without implementing any streaming.

For now, we’ll take the most naive, non optimized, approach (remember I am not an expert at Godot nor at Game dev) and might improve our implementation at a later stage.

With that out of the way, let’s get started!

IPL Loading

In our example we’ll be using Vice City’s golf course. The golf course is located near the games center (0.0, 0.0, 0.0) which makes it convenient for quick testing.

As described in my previous post the placement of objects is defined in IPL files. This file contains the location data of an object. Which means we should be able to display a bunch of boxes at the location data and already get a feel of the map.

The easiest way to display a lot of boxes at certain positions is by just adding MeshInstance3D nodes to a scene, but this is a learning experience so we can do it a bit more efficiently. We can use MultiMeshInstance3D, a node specifically created to render repeating meshes efficiently, perfect for our use case!

Some example code, this assumes we have an IPL file loaded into memory that contains an array of instances.

// Create a multi mesh
MultiMesh multiMesh = new MultiMesh
{
    TransformFormat = MultiMesh.TransformFormatEnum.Transform3D,    // Sets up our multimesh to support 3D transforms
    InstanceCount = ipl.Instances.Count,                            // Set the amount of instances to our IPL instance count
    Mesh = new BoxMesh                                              // Set the mesh to a simple box
    {
        Size = new Vector3(2.0f, 2.0f, 2.0f),
    }
};

// Loop through our IPL instances and set the transforms in the multimesh
for (var i = 0; i < ipl.Instances.Count; i++)
{
    var inst = ipl.Instances[i];
    multiMesh.SetInstanceTransform(i, new Transform3D(this.Basis, inst.Position));  // Set the transform to our instance position (Ignoring rotation and scale for now)
}

MultiMeshInstance3D multiMeshNode = new MultiMeshInstance3D();  // Create a MultiMeshInstance3D node
multiMeshNode.Multimesh = multiMesh;                            // Attach our multi mesh to the node
AddChild(multiMeshNode);                                        // Add the MultiMeshInstance3D node to the scene

The result:

Hmm something is up, and it’s not Z! If you didn’t get that joke no worries I’ll explain it.

The coordinate system

Godot and GTA use a different coordinate system! Freya Holmer created an amazing image that explains the difference in coordinate systems and it’s usages.

As you can see in the above image, Godot uses the Right Handed Y-Up coordinate system. But, which one does GTA use? Well the modeling / mapping tool used for the original trilogy was 3DS Max, which matches with the coordinate system used by GTA, the Right Handed Z-Up coordinate system.

There are multiple options to switch between those coordinate systems.

1. Swap the Y and Z values of every Vector3 when loading it into Godot.
2. Rotate every object individually
3. Rotate the camera
4. Rotate the world root node

And to be honest I am not sure what the best approach is. Option #1 sounds easy, but will probably end up causing issues when Z is not used in a Vector3 (IE: By the script) and might not work nicely with rotations. Option #2 sounds like a lot of work. So I guess that leaves us with options #3 and #4. For now, I’ll use option #4 as that is the easiest.

After rotating the world 90 degrees we end up with the following, which already looks a lot better!

All nice and dandy those boxes, but we’d like to actually see the map. To do that we’ll need to load the IDE first as those define which model / textures should be used.

But first we’ll need to refactor our box rendering code. MultiMesh is great for rendering the same mesh multiple times, but in our case we want to render a different mesh for each instance. So let’s rewrite our box rendering code to use regular nodes.

// Create our box mesh
Mesh mesh = new BoxMesh
{
    Size = new Vector3(2.0f, 2.0f, 2.0f)
};
     
// Loop through our IPL instances and create a new MeshInstance3D with our box mesh at the instance position
for (var i = 0; i < ipl.Instances.Count; i++)
{
    var inst = ipl.Instances[i];
    var meshInstance = new MeshInstance3D();
    meshInstance.SetMesh(mesh);
    meshInstance.Position = inst.Position;
    AddChild(meshInstance);
}

The result of this code is exactly the same as the MultiMesh implementation except it uses a few more draw calls, and it allows us to render different meshes for each instance.

IDE loading

To replace our boxes with meshes we have to load the IDE file as those define which model belongs to each ID.

Let’s parse the IDE file and create some kind of look up table of definitions, this could be a simple array where the index is the ID of a definition or some kind of Dictionary. Now that we have an easy way to access the definition we can retrieve the model information, and use that to load our mesh.

The updated code should look something like this pseudocode:

public void _Ready() {
    // Load IDE and create the definition look up table
    LoadIDE("data/maps/golf/golf.ide")
    // Load IPL
    LoadIPL("data/maps/golf/golf.ipl")
}

public void LoadIpl(string path) {
    // Parse the IPL file into an object
    var ipl = IplLoader.Load(path);
    
    // Loop through our IPL instances
    for (var i = 0; i < ipl.Instances.Count; i++)
    {
        var inst = ipl.Instances[i];
        
        // Instead of creating the BoxMesh we load the model according to the definition
        var model = LoadModelForId(inst.Id);
        
        // Set the position of the model
        model.Position = inst.Position;
        
        // Add the model to the scene
        AddChild(model);
    }
}

private RwNode LoadModelForId(int id) 
{
    // Fetch definition from lookup table (Note that some definitions might be missing due to us only loading golf.ide)
    // Load model according to definition (as described in part 1/2 of this series)
}

This should already give us something recognizable.

And with textures. (I have to write about texture loading at some point!)

There are some things not quite right though. If we for example take a look at those bushes (besides the fact that they are textureless) they are also misplaced.

We forgot to apply the rotation! Let’s update the code to also apply the rotation to our model.

// Set the position of the model
model.Position = inst.Position;

// Set the rotation of the model
model.Quaternion = inst.Quaternion

The result (no this didn’t magically fix our missing textures):

That’s it for this post, in the next one we’ll also load collision so we don’t fall through the map.

Now that we have model and collision loading in place. Let’s put it all together and take a look at how the meshes are placed to create our beloved cities.

To get to that we need to understand the structure of the game files and how those contribute to the games map.

ℹ️ A lot of this info is based on the knowledge shared on gtamods.com

DAT

One of the first things the games load are the following dat files; default.dat and gta(game version).dat. These dat files are simple text files that tell the engine which file to load.

Let’s open up Vice City’s default.dat and go through the file step by step.

# 
# Load IDEs first, then the models and after that the IPLs
# 

The first couple of lines start with a #. This could be familiar to Godot devs as these are just comments. The engine ignores any line that starts with a #.

Let’s read a little bit further, the first non comment line is:

IDE DATA\DEFAULT.IDE

Now it starts to get interesting. So what does this line mean? We can split it up into two parts; IDE and DATA\DEFAULT.IDE. In this example IDE is the command that tells the engine to load an IDE file also known as Item DEfinition, DATA\DEFAULT.IDE is the relative path of the ide file that should be loaded. We’ll get back to IDE files a little bit later.

Next up we see the following lines (skipping the comments).

TEXDICTION MODELS\MISC.TXD
MODELFILE MODELS\GENERIC\AIR_VLO.DFF
COLFILE 0 MODELS\COLL\VEHICLES.COL

Again this tells the engine to load specific file types.

• TEXDICTION Loads a texture dictionary aka a collection of Textures.
• MODELFILE Loads a model
• COLFILE Loads a collision file.

That’s it for default.dat. Let’s also open gta.dat or in Vice City’s case gta_vc.dat.

This file is pretty similar to default.dat, it contains a bunch of IDE commands, but it does contain two new commands:

SPLASH loadsc2
IPL DATA\MAP.ZON

• SPLASH Switches the splash screen during loading. Fun fact: this only works on the console versions of the game.
• IPL Loads an IPL file also known as a Item PLacement. Again we’ll take a look at those specific files in a bit.

If we take a look at both GTA: III and SA there are a couple more commands that are not used in Vice City.

• IMG Loads an Image archive file
• MAPZONE Loads a zone file in GTA: III. Seems to have been replaced with a regular IPL file in the later games.

IDE

The Item Definition file contains as it name might give away; definitions of items. In all seriousness, the file defines the ID of a model, the TXD it uses and a bunch of other properties depending on the type of the item.

To clear things up, lets checkout default.ide, which is also the first file that got loaded by default.dat.

Again we start the file with a bunch of comments as indicated by #. We can probably skip this explanation next time ;), but do take a look at the comments in the file they contain useful info left by the devs.

Besides the comments, the structure of the file is slightly different from the .dat files we looked at previously. Instead of each line being a command that gets executed this file is split up into sections. Section starts are marked by specific keywords identifying the type of items in that section and the ending is marked by end.

With that out of the way lets have a look the file:

objs
# wheels
160, wheel_sport, generic, 2, 20, 70, 0
# Other entries
end

The objs keyword is the start marker of a section. In this case the Objects section. This section defines regular object aka models.

If we look at the first entry we see: 160, wheel_sport, generic, 2, 20, 70, 0. Lets break it down:

• 160 is the ID of the model, this is a unique ID that is used by the engine to reference this model
• wheel_sport is the name of the DFF (aka a model)
• generic is the name of the TXD (aka the texture archive)
• 2 the amount of meshes in this model
• 20 the draw distance of the first mesh
• 70 the draw distance of the second mesh
• 0 some flags that set special rendering options

So now if the engine is told to load model 160 it will know which model to use, what texture archive to apply and how to render it.

We can go very in depth into every section but for now I’ll give a short summary.

• hier Cutscene objects
• cars Vehicle definitions
• peds Pedestrian definitions

The next couple of sections are not found in default.ide but are used by the game in other ide files:

• tobj Timed objects, spawned only at specific times
• path Used to define paths for vehicles and pedestrians
• 2dfx 2D effects, like lights
• weap Weapon models
• anim Animated objects
• txdp Texture archive extensions

That’s it for the IDE file, lets see how these definitions are actually referenced by IPL files.

IPL

The Item Placement files uses the exact same format as IDE files. It’s a plain text file split up into sections.

If we open Vice City’s airport.ipl in a text editor you’ll see the following lines.

inst
865, ap_tower, 0, -1685.179443, -923.3638916, 13.48704815, 1, 1, 1, 0, 0, 0, 1
# More entries
end

Let’s break it down again.

The section marker is inst short for instance. This section describes which models should be placed where.

Each entry has a lot of properties: 865, ap_tower, 0, -1685.179443, -923.3638916, 13.48704815, 1, 1, 1, 0, 0, 0, 1

• 865 The ID as defined in IDE files
• ap_tower Model name (Not sure if this has any actual purpose as the model is also defined in the IDE)
• 0 The interior this model belongs to
• -1685.179443 The X position
• -923.3638916 The Y position
• 13.48704815 The Z position
• 1 The X scale
• 1 The Y scale
• 1 The Z scale
• 0 The X rotation (quaternion)
• 0 The Y rotation (quaternion)
• 0 The Z rotation (quaternion)
• 1 The W rotation (quaternion)

This is probably the most useful section of IPL files but lets list them all.

• cull Defines zones with some attributes
• pick Defines a pickup
• path Defines paths for vehicles
• occl Defines occlusion zones

The following sections are San Andreas only:

• grge Defines a garage
• enex Defines entry and exit markers for interiors
• cars Defines a parked car generator
• jump Defines a stunt jump
• tcyc Something related to the timecycle
• auzo Defines an audio zone

Next steps

With this knowledge of the IDE and IPL files we should already be able to create something that resembles a map!

We’ll get to the implementation in the next post.