⚠️️ A little disclaimer:

Godot currently doesn’t provide streaming capabilities out of the box. But, the games we try to simulate are very old and hardware has improved immensely in the past 25(!) years. So we might actually get away with just rendering the map without implementing any streaming.

For now, we’ll take the most naive, non optimized, approach (remember I am not an expert at Godot nor at Game dev) and might improve our implementation at a later stage.

With that out of the way, let’s get started!

IPL Loading

In our example we’ll be using Vice City’s golf course. The golf course is located near the games center (0.0, 0.0, 0.0) which makes it convenient for quick testing.

As described in my previous post the placement of objects is defined in IPL files. This file contains the location data of an object. Which means we should be able to display a bunch of boxes at the location data and already get a feel of the map.

The easiest way to display a lot of boxes at certain positions is by just adding MeshInstance3D nodes to a scene, but this is a learning experience so we can do it a bit more efficiently. We can use MultiMeshInstance3D, a node specifically created to render repeating meshes efficiently, perfect for our use case!

Some example code, this assumes we have an IPL file loaded into memory that contains an array of instances.

// Create a multi mesh
MultiMesh multiMesh = new MultiMesh
{
    TransformFormat = MultiMesh.TransformFormatEnum.Transform3D,    // Sets up our multimesh to support 3D transforms
    InstanceCount = ipl.Instances.Count,                            // Set the amount of instances to our IPL instance count
    Mesh = new BoxMesh                                              // Set the mesh to a simple box
    {
        Size = new Vector3(2.0f, 2.0f, 2.0f),
    }
};

// Loop through our IPL instances and set the transforms in the multimesh
for (var i = 0; i < ipl.Instances.Count; i++)
{
    var inst = ipl.Instances[i];
    multiMesh.SetInstanceTransform(i, new Transform3D(this.Basis, inst.Position));  // Set the transform to our instance position (Ignoring rotation and scale for now)
}

MultiMeshInstance3D multiMeshNode = new MultiMeshInstance3D();  // Create a MultiMeshInstance3D node
multiMeshNode.Multimesh = multiMesh;                            // Attach our multi mesh to the node
AddChild(multiMeshNode);                                        // Add the MultiMeshInstance3D node to the scene

The result:

Hmm something is up, and it’s not Z! If you didn’t get that joke no worries I’ll explain it.

The coordinate system

Godot and GTA use a different coordinate system! Freya Holmer created an amazing image that explains the difference in coordinate systems and it’s usages.

As you can see in the above image, Godot uses the Right Handed Y-Up coordinate system. But, which one does GTA use? Well the modeling / mapping tool used for the original trilogy was 3DS Max, which matches with the coordinate system used by GTA, the Right Handed Z-Up coordinate system.

There are multiple options to switch between those coordinate systems.

1. Swap the Y and Z values of every Vector3 when loading it into Godot.
2. Rotate every object individually
3. Rotate the camera
4. Rotate the world root node

And to be honest I am not sure what the best approach is. Option #1 sounds easy, but will probably end up causing issues when Z is not used in a Vector3 (IE: By the script) and might not work nicely with rotations. Option #2 sounds like a lot of work. So I guess that leaves us with options #3 and #4. For now, I’ll use option #4 as that is the easiest.

After rotating the world 90 degrees we end up with the following, which already looks a lot better!

All nice and dandy those boxes, but we’d like to actually see the map. To do that we’ll need to load the IDE first as those define which model / textures should be used.

But first we’ll need to refactor our box rendering code. MultiMesh is great for rendering the same mesh multiple times, but in our case we want to render a different mesh for each instance. So let’s rewrite our box rendering code to use regular nodes.

// Create our box mesh
Mesh mesh = new BoxMesh
{
    Size = new Vector3(2.0f, 2.0f, 2.0f)
};
     
// Loop through our IPL instances and create a new MeshInstance3D with our box mesh at the instance position
for (var i = 0; i < ipl.Instances.Count; i++)
{
    var inst = ipl.Instances[i];
    var meshInstance = new MeshInstance3D();
    meshInstance.SetMesh(mesh);
    meshInstance.Position = inst.Position;
    AddChild(meshInstance);
}

The result of this code is exactly the same as the MultiMesh implementation except it uses a few more draw calls, and it allows us to render different meshes for each instance.

IDE loading

To replace our boxes with meshes we have to load the IDE file as those define which model belongs to each ID.

Let’s parse the IDE file and create some kind of look up table of definitions, this could be a simple array where the index is the ID of a definition or some kind of Dictionary. Now that we have an easy way to access the definition we can retrieve the model information, and use that to load our mesh.

The updated code should look something like this pseudocode:

public void _Ready() {
    // Load IDE and create the definition look up table
    LoadIDE("data/maps/golf/golf.ide")
    // Load IPL
    LoadIPL("data/maps/golf/golf.ipl")
}

public void LoadIpl(string path) {
    // Parse the IPL file into an object
    var ipl = IplLoader.Load(path);
    
    // Loop through our IPL instances
    for (var i = 0; i < ipl.Instances.Count; i++)
    {
        var inst = ipl.Instances[i];
        
        // Instead of creating the BoxMesh we load the model according to the definition
        var model = LoadModelForId(inst.Id);
        
        // Set the position of the model
        model.Position = inst.Position;
        
        // Add the model to the scene
        AddChild(model);
    }
}

private RwNode LoadModelForId(int id) 
{
    // Fetch definition from lookup table (Note that some definitions might be missing due to us only loading golf.ide)
    // Load model according to definition (as described in part 1/2 of this series)
}

This should already give us something recognizable.

And with textures. (I have to write about texture loading at some point!)

There are some things not quite right though. If we for example take a look at those bushes (besides the fact that they are textureless) they are also misplaced.

We forgot to apply the rotation! Let’s update the code to also apply the rotation to our model.

// Set the position of the model
model.Position = inst.Position;

// Set the rotation of the model
model.Quaternion = inst.Quaternion

The result (no this didn’t magically fix our missing textures):

That’s it for this post, in the next one we’ll also load collision so we don’t fall through the map.

Now that we have model and collision loading in place. Let’s put it all together and take a look at how the meshes are placed to create our beloved cities.

To get to that we need to understand the structure of the game files and how those contribute to the games map.

ℹ️ A lot of this info is based on the knowledge shared on gtamods.com

DAT

One of the first things the games load are the following dat files; default.dat and gta(game version).dat. These dat files are simple text files that tell the engine which file to load.

Let’s open up Vice City’s default.dat and go through the file step by step.

# 
# Load IDEs first, then the models and after that the IPLs
# 

The first couple of lines start with a #. This could be familiar to Godot devs as these are just comments. The engine ignores any line that starts with a #.

Let’s read a little bit further, the first non comment line is:

IDE DATA\DEFAULT.IDE

Now it starts to get interesting. So what does this line mean? We can split it up into two parts; IDE and DATA\DEFAULT.IDE. In this example IDE is the command that tells the engine to load an IDE file also known as Item DEfinition, DATA\DEFAULT.IDE is the relative path of the ide file that should be loaded. We’ll get back to IDE files a little bit later.

Next up we see the following lines (skipping the comments).

TEXDICTION MODELS\MISC.TXD
MODELFILE MODELS\GENERIC\AIR_VLO.DFF
COLFILE 0 MODELS\COLL\VEHICLES.COL

Again this tells the engine to load specific file types.

• TEXDICTION Loads a texture dictionary aka a collection of Textures.
• MODELFILE Loads a model
• COLFILE Loads a collision file.

That’s it for default.dat. Let’s also open gta.dat or in Vice City’s case gta_vc.dat.

This file is pretty similar to default.dat, it contains a bunch of IDE commands, but it does contain two new commands:

SPLASH loadsc2
IPL DATA\MAP.ZON

• SPLASH Switches the splash screen during loading. Fun fact: this only works on the console versions of the game.
• IPL Loads an IPL file also known as a Item PLacement. Again we’ll take a look at those specific files in a bit.

If we take a look at both GTA: III and SA there are a couple more commands that are not used in Vice City.

• IMG Loads an Image archive file
• MAPZONE Loads a zone file in GTA: III. Seems to have been replaced with a regular IPL file in the later games.

IDE

The Item Definition file contains as it name might give away; definitions of items. In all seriousness, the file defines the ID of a model, the TXD it uses and a bunch of other properties depending on the type of the item.

To clear things up, lets checkout default.ide, which is also the first file that got loaded by default.dat.

Again we start the file with a bunch of comments as indicated by #. We can probably skip this explanation next time ;), but do take a look at the comments in the file they contain useful info left by the devs.

Besides the comments, the structure of the file is slightly different from the .dat files we looked at previously. Instead of each line being a command that gets executed this file is split up into sections. Section starts are marked by specific keywords identifying the type of items in that section and the ending is marked by end.

With that out of the way lets have a look the file:

objs
# wheels
160, wheel_sport, generic, 2, 20, 70, 0
# Other entries
end

The objs keyword is the start marker of a section. In this case the Objects section. This section defines regular object aka models.

If we look at the first entry we see: 160, wheel_sport, generic, 2, 20, 70, 0. Lets break it down:

• 160 is the ID of the model, this is a unique ID that is used by the engine to reference this model
• wheel_sport is the name of the DFF (aka a model)
• generic is the name of the TXD (aka the texture archive)
• 2 the amount of meshes in this model
• 20 the draw distance of the first mesh
• 70 the draw distance of the second mesh
• 0 some flags that set special rendering options

So now if the engine is told to load model 160 it will know which model to use, what texture archive to apply and how to render it.

We can go very in depth into every section but for now I’ll give a short summary.

• hier Cutscene objects
• cars Vehicle definitions
• peds Pedestrian definitions

The next couple of sections are not found in default.ide but are used by the game in other ide files:

• tobj Timed objects, spawned only at specific times
• path Used to define paths for vehicles and pedestrians
• 2dfx 2D effects, like lights
• weap Weapon models
• anim Animated objects
• txdp Texture archive extensions

That’s it for the IDE file, lets see how these definitions are actually referenced by IPL files.

IPL

The Item Placement files uses the exact same format as IDE files. It’s a plain text file split up into sections.

If we open Vice City’s airport.ipl in a text editor you’ll see the following lines.

inst
865, ap_tower, 0, -1685.179443, -923.3638916, 13.48704815, 1, 1, 1, 0, 0, 0, 1
# More entries
end

Let’s break it down again.

The section marker is inst short for instance. This section describes which models should be placed where.

Each entry has a lot of properties: 865, ap_tower, 0, -1685.179443, -923.3638916, 13.48704815, 1, 1, 1, 0, 0, 0, 1

• 865 The ID as defined in IDE files
• ap_tower Model name (Not sure if this has any actual purpose as the model is also defined in the IDE)
• 0 The interior this model belongs to
• -1685.179443 The X position
• -923.3638916 The Y position
• 13.48704815 The Z position
• 1 The X scale
• 1 The Y scale
• 1 The Z scale
• 0 The X rotation (quaternion)
• 0 The Y rotation (quaternion)
• 0 The Z rotation (quaternion)
• 1 The W rotation (quaternion)

This is probably the most useful section of IPL files but lets list them all.

• cull Defines zones with some attributes
• pick Defines a pickup
• path Defines paths for vehicles
• occl Defines occlusion zones

The following sections are San Andreas only:

• grge Defines a garage
• enex Defines entry and exit markers for interiors
• cars Defines a parked car generator
• jump Defines a stunt jump
• tcyc Something related to the timecycle
• auzo Defines an audio zone

Next steps

With this knowledge of the IDE and IPL files we should already be able to create something that resembles a map!

We’ll get to the implementation in the next post.

With some basic model loading working (no worries we’ll get to the textures at some point!) it might be nice to look at something completely different; Collision!

What do I mean with collision?

The models we loaded up until now are only used for the visual representation of an object. The chair we loaded might look physical, but if someone would try to sit on it, they would fall through. The reason for this is that it’s lacking a physical model aka a collision model. A collision model defines the shape, and in case of GTA the material, of the model. We use the collision model to define the roads we drive on and the walls we bump into.

Why do we need separate collision models?

Physics are complex, even though we have amazing physics engines and our computers can compute millions of physic interactions per second, simulating everything using their visual representation would be impossible. The visual representation of a model is usually much more detailed than necessary for believable physics simulation.

Instead, physics of video games (and especially in the early 2000) used simplified shapes. Mostly a combination of (ordered by efficiency); spheres, boxes and basic meshes.

The collision file format

The collision format used by the GTA Trilogy has been documented at gtamods.com.

Short summary:

• Collision files contain multiple collision entries.
• Each collision entry can contain an array of Spheres, Boxes and a Mesh shapes.
• The shapes can reference a material which is hardcoded in the engine.

The ramp example

Let’s take a look at generic.col, this file is a collection of multiple col files into one. Also known as a coll archive. In our example we’ll focus on the entry ramp, this is simple collision model that uses all 3 types! A perfect example for this post.

The ramp exists out of 4 shapes:

• 2 Spheres
• 1 Box
• 1 Mesh shape

The spheres are used to cover the top of the pipes on the side. The box is used to cover the long pipe. And the mesh is used for the actual ramp.

Loading the collision data into Godot

Let’s start by creating a StaticBody3D, these are used for static objects like our ramp. As the name implies static bodies are not effected by other physics bodies, perfect for our static ramp.

A physics body like StaticBody3D needs one or multiple CollisionShape3D nodes, which define their shape using a Shape3D. The shapes we are interested in are: SphereShape3D, BoxShape3D and ConcavePolygonShape3D.

To create a StaticBody3D from our coll file all we need to do is convert our Spheres, Boxes and Meshes to Godot equivalents.

Spheres

Creating the SphereShape3D from our collision file data is the easiest. The data contains everything we need; a radius and a position.

Example code:

var collisionShape = new CollisionShape3D();

// Create the shape
var shape = new SphereShape3D();
shape.Radius = sphere.radius;

// Set the shape and update it's position according to the spheres center
collisionShape.Shape = shape;
collisionShape.Position = sphere.center;

As you can see our example uses 2 spheres to cover the top of the pipes:

Boxes

BoxShape3D requires a tiny bit more work to create compared to spheres, as the required data is defined slightly different in the coll file compared to Godots implementation.

The data we got:

• Min (the minimum coordinates of the box). Example: Vector3(-10, -10, 5)
• Max (the maximum coordinates of the box). Example: Vector3(10, 10, 10)

BoxShape3D data:

• Size (the size of the box). Example: Vector3(20, 20, 5)
• Position (center position of the box). Example: Vector3(0, 0, 7.5)

Converting this data is as simple as:

Vector3 size = Vector3.Subtract(max, min)
Vector3 position = (min + max) * 0.5f;

Example code:

var collisionShape = new CollisionShape3D(); 

// Create the shape
var shape = new BoxShape3D();
shape.Size = Vector3.Subtract(box.max, box.min);

// Set the shape and update it's position according to the calculated box center
collisionShape.Shape = shape;
collisionShape.Position = (box.min + box.max) * 0.5f;

A single box collider is used to cover the long pipe:

Mesh

A coll entry can contain a single mesh. The mesh is defined by vertices and faces.

We can convert the mesh to a ConcavePolygonShape3D

Example code:

var collisionShape = new CollisionShape3D();

// Create the mesh shape
var meshShape = new ConcavePolygonShape3D();

// Convert the face / vertex data to a triangle array
var triangleArray = new Vector3[faces.Length * 3];
for (int i = 0; i < faces.Length; i++)
{
    triangleArray[i * 3] = vertices[faces[i].a].ToVector3();
    triangleArray[i * 3 + 1] = vertices[faces[i].b].ToVector3();
    triangleArray[i * 3 + 2] = vertices[faces[i].c].ToVector3();
}

// Apply the triangle data to the meshShape
meshShape.Data = triangleArray;

// Set the shape
collisionShape.Shape = meshShape;

The actual ramp shape is defined using the mesh collider:

Final result

Below the full collision of the ramp:

In part 2 of this series we’ll take a look at slightly more complex models. First up we’ll take a look at a model that uses multiple geometries, this way we can see how the hierarchy is constructed.

Multi geometry model

Let’s take a look at the model SCHAIR.dff. This model is the chair used by Sonny in the intro cutscene of Vice City. This model can be found in the gta3.img archive, you’ll need an image editor to extract the model file from the archive.

If we print the RenderWare structure you can see that the tree is a bit bigger compared to the arrow.dff that we used in the previous part.

RwClump
├── RwFrameList
│   ├── RwExtension
│   │   ├── RwHAnimPlg
│   │   └── RwFrame
│   ├── RwExtension
│   │   ├── RwHAnimPlg
│   │   └── RwFrame
│   ├── RwExtension
│   │   ├── RwHAnimPlg
│   │   └── RwFrame
│   └── RwExtension
│       ├── RwHAnimPlg
│       └── RwFrame
├── RwGeometryList
│   ├── RwGeometry
│   │   ├── RwMaterialList
│   │   │   └── RwMaterial
│   │   │       ├── RwTexture
│   │   │       │   ├── RwString
│   │   │       │   ├── RwString
│   │   │       │   └── RwExtension
│   │   │       └── RwExtension
│   │   └── RwExtension
│   │       └── RwBinMeshPlg
│   └── RwGeometry
│       ├── RwMaterialList
│       │   └── RwMaterial
│       │       ├── RwTexture
│       │       │   ├── RwString
│       │       │   ├── RwString
│       │       │   └── RwExtension
│       │       └── RwExtension
│       └── RwExtension
│           └── RwBinMeshPlg
├── RwAtomic
│   └── RwExtension
├── RwAtomic
│   └── RwExtension
└── RwExtension

The main difference is the amount of frames (4) and geometries (2), accompanied by 2 atomic sections to match the geometry entries to a frame.

Let’s start with taking a look at the frame data inside the FrameList.

Frame 1

Name: "schair_dummy"
ParentIndex: -1

Frame 2

Name: "schair"
ParentIndex: 0

Frame 3

Name: "schrbot"
ParentIndex: 1

Frame 4

Name: "schrtop"
ParentIndex: 1

In this case schair_dummy is the root node which is located at index 0. The parent index is set to -1 which indicates it does not have a parent. schair on the other hand has a parent index of 0, which indicates that it’s parent is schair_dummy. Last but not least we have 2 frames called schrbot and schrtop (Sonny Chair Bottom and Sonny Chair Top) which have a parent index of 1 aka schair.

The hierarchy we end up with is:

schair_dummy
    └── schair
        ├── schrbot
        └── schrtop

If we use the same approach as we used in part 1 of this series, we expect the following, very similar, output hierarchy.

Node3D
└── schair_dummy
    └── schair
        ├── schrbot
        └── schrtop

And here it is schair in Godot.

Fun fact, GTA: III doesn’t make use of skinned meshes for their animated characters, but instead relies on the above technique to construct its models. So here is a little screenshot of Claude:

That’s it for now, we’ll get to Material and Skeleton loading in another part, but first we’ll make a little sidestep and checkout collision loading!

In this first part of the series we discuss model loading. Nothing more visual than rendering our first GTA model.

To load a GTA model we first need to understand the original engine this game used. The original trilogy was based on the RenderWare engine (RW from now on), an engine that was very popular during the time. Many games by different developers where created on this engine, think about titles like Burnout, Tony Hawks Pro Skater 3 and Bully. It was the Unreal Engine of that era.

Lucky for us this means a lot is known about the file formats used by RW based games. Entire tools and libraries where build to inspect the contents of RW files. One of which is called RwAnalyze, this is still one of the best tools (22 years old by now!) to get a grasp of the basic structure of RW files.

As you can see in the above screenshot a dff file is simply 3D data stored in a tree like structure.

An example

Let’s take a closer look at a pretty basic dff file. We can use Vice City’s arrow.dff which can be found in the /models/generic/ directory. This model has only a single mesh, no textures and is pretty much as basic as it gets.

ℹ️ You might notice that there are not that many dff files in the models directory, which might seem odd for a game with hundreds of models. This is because most models are packed into the gta3.img archive file. We’ll get to the details of that file in another part.

The structure

The following tree shows the structure of the dff file.

RwClump
├── RwFrameList
│   └── RwFrame
├── RwGeometryList
│   └── RwGeometry
│       ├── RwMaterialList
│       │   └── RwMaterial
│       ├── RwBinMeshPlg
│       └── RwMorphPlg
└── RwAtomic
    ├── RwParticlesPLG
    └── RwMatEffectsPlg

Let’s have a look at the sections and how they relate to Godot.

RwClump

A container of a frame hierarchy and 3D data. We could create something similar using a Node3D with child nodes.

RwFrameList

List of Frames.

RwFrame

A frame has a name, positioning data and a parent ID. Frames are used to build up the hierarchy of the model. Frames can be used to position geometry, dummies or bones from a skeleton. In Godot this is again similar to a Node3D.

In our case of arrow.dff this frame contains the name “arrow”.

RwGeometryList

List of Geometry

RwGeometry

The visual representation of the model. Contains the vertex, polygon and material data. The Godot equivalent of a Mesh.

RwMaterialList

List of materials

RwMaterial

Material definition, contains the texture name, colors, alpha name etc.

RwBinMeshPlg

Plugin that defines how the geometry is split by materials. Which vertices / polygons use which material. Similar to a Surface on a Mesh in Godot.

RwMorphPlg

I currently don’t support this in my parser, if we need it we’ll probably get to it at some point.

RwAtomic

An atomic matches geometries to a frame. Making it possible to create a hierarchy of geometry. We’ll need this info to setup the mesh / node hierarchy in Godot.

RwParticlesPLG I currently don’t support this in my parser, if we need it we’ll probably get to it at some point.

RwMatEffectsPlg

I currently don’t support this in my parser, if we need it we’ll probably get to it at some point.

Godot

If we take the above arrow.dff example and map that to Godot we can imagine this would end up something like:

Node3D
└── MeshInstance3D (arrow)

Meshes

The first step to create this scene is by converting all RwGeometry sections to a Godot Mesh. Godot offers multiple API’s to create meshes at runtime. In my case I ended up using ArrayMesh. By going through all material splits we can create Surfaces using the AddSurfaceFromArrays method and assign a Material to the Surface by using SurfaceSetMaterial

Hierarchy

Once all meshes and materials are created we have to reconstruct the hierarchy. For this we’ll use the RwFrameList and RwAtomic sections. When a frame is connected to a geometry as indicated by the RwAtomic section we can create a new MeshInstance3D and assign it the Mesh that we created from the geometry. In case no geometry is connected to the frame we’ll just create a simple Node3D node. We’ll hook those nodes up according to the parent / child information of the RwFrame.

The result

With some trial and error this is the result:

I hope you enjoyed this first part of the series. In the next part we’ll take a look at loading more complex models.