Upgrading the DM rendering engine - now with source code

Lesser known clone projects or isolated news items about rare or unusual clones.
Forum rules
Please read the Forum rules and policies before posting.
Post Reply
babebridou
Novice
Posts: 25
Joined: Mon May 17, 2010 6:47 am

Upgrading the DM rendering engine - now with source code

Post by babebridou »

I'm a big fan of the original game, and I'd like to enhance it visually. My plan is to use Unity3d (http://www.unity3d.com) strictly as a rendering engine. Game mechanics won't change - Theron will be free to change the camera angle however he sees fit, but otherwise the environment will still be segmented in tiles with 4 positions, no jumping, etc. And make it free & open the source code asap so other devs can play with it too. That's the plan anyway. As with all "fun" projects, it's easy to get started, and soon it gets crazy tedious :S

Here's how it looks so far: only layout, haven't bothered rendering pits and stuff, doors are only stubs, etc. But the whole dungeon layout is parsed and instanced in a single level.

http://www.fairyteller.com/dungeon/Fair ... ngeon.html (right-click to go full-screen)

Sorry about the clumsy controls: it's a french keyboard (zsqd rather than wsad, but you can still use the arrows), and it's locked on inverted mouse Y axis (because I learned to play flight sims with the mouse on my amiga, and it somehow stuck). I haven't bothered yet with those controls: I'm currently learning how the original game manages its data in memory so it uses the same "specs" as CSBwin (reading dungeon.dat format instead of xmls) - I hope to be able to use "native DSA transitions" instead of "having to rewrite all game mechanics and hope it works with older dungeons". I just hope I'm going somewhere with this sentence :D Right now I'm using SCK to export an XML, parse it at the start. Now that the display engine for tiles is stubbed and "functional", it's time to refactor what's under the hood and do some more low level modifications for the original gamestate engine to work properly. And it will take me some time, because, well, I'm still in the learning phase for all that stuff :)

PS sorry if the above makes no sense :)

PS2, it works with CSB, too :)
http://www.fairyteller.com/dungeon/Fair ... ngeon.html

PS3 Please report if you see or experience issues (performance, crashes etc). It should work on mac & pc.

PS4 Huge thanks to everyone for keeping the DM spirit alive and kicking.

Update may 28:
woah, I tried to port the CSBWin code from c++ to c# - it was horrible. I stopped halfway through the databases mostly because I had no idea of what I was doing and whether it would work at all.
The good part is, I now have a semi decent idea of how things work "under the hood" with actuators and with DSA.

So, I'm back to the drawing board :)
Last edited by babebridou on Fri Jun 11, 2010 8:43 am, edited 1 time in total.
myjean17
Novice
Posts: 28
Joined: Thu Apr 09, 2009 1:50 am

Re: Upgrading the DM rendering engine

Post by myjean17 »

Looks pretty good! How did you do the levels?
User avatar
Bit
Arch Master
Posts: 1064
Joined: Mon Mar 03, 2008 10:53 am
Location: Nuts trees

Re: Upgrading the DM rendering engine

Post by Bit »

I'm busy with the code for more than a year now, and I still got a decent decent idea... :P
babebridou
Novice
Posts: 25
Joined: Mon May 17, 2010 6:47 am

Re: Upgrading the DM rendering engine

Post by babebridou »

Looks pretty good! How did you do the levels?
I made "tiles" for walls, floors, stairs and doors as "prefabs" in unity - I used a cut & paste of the original DM wall graphic as a texture. Then I parsed the dungeon.dat binary file using the encyclopedia's dungeon file definition as a spec.
I'm busy with the code for more than a year now, and I still got a decent decent idea...
hehe :) yeah I kept it on hold for the moment, mostly to learn new stuff about the unity3d engine. Who knows, some day maybe I'll get back to it?

In the mean time...
here's a link to the unity project with all the source code (8MB) : http://www.fairyteller.com/dungeon/UnityDMtoExport.rar
You can get unity3d there for free: http://unity3d.com/ (and I mean free, the pro version is awesome, but the free version is just insanely good)

As I said, pardon my french and my clumsy source code, I haven't written much javascript in the last decade and I couldn't be arsed to document a lot :( But eh, I share :D

Entry point is the DungeonItemManagerTester script, which creates a DungeonDatabase object (with 15 kinds of readers, one for each object type weapon, actuator, tile, monster etc) out of a dungeon.dat file, then iterates through each level and build the corresponding tiles. Each level *should* have code to activate/deactivate its rendering components whether the player is a few levels above or below it to save memory, but I'm not completely sure if I broke that with the project cleaning & export.

I'm not sure if it can be improved directly as there are glaring design mistakes, so many I forgot which actually :D, but at least, eh, it's stuff that works, runs, and while not really fun yet, it might be helpful to others!

Please tell if you have any issues importing the project in unity or if I messed up anything badly.

Edit: the reason for me posting the source is, I'd need some coding architecture input - I'm having troubles figuring out a clean process to grab, say, Iaido's portrait or the first "hall of champions" wall text, something that definitely shouldn't be hard, not that early in development. I think most of my issues come from the definition differences between the CSBWin memory and its dungeon.dat init file so I'm actually hardcoding the wrong data slots here, but to be honest I don't know exactly where to start from here.
User avatar
Bit
Arch Master
Posts: 1064
Joined: Mon Mar 03, 2008 10:53 am
Location: Nuts trees

Re: Upgrading the DM rendering engine - now with source code

Post by Bit »

Unfortunately I can't help with js, and can't be bothered to learn now too.
All I can give is a little feedback about the demo. I did download the unity-engine, perhaps that's really interesting.
Did install the webplayer too to watch the demo (usually I really don't like to install any 3rd-party stuff that I don't know!)
So - first impression was, a very very odd feeling with that old walls moving in 3D now. Then - maybe it's just the adjustment of the mouse sensivity, switching mouse behaviour for up/down and so on to get a common feeling for recent games that I play. (Also the viewing height - are we crawling?) - but - so as it is, movement along stairs is more evil than movement in the elder scrolls 1... Basically that's surely something that can be fixed quickly, but, taking a quick look into your source, I just hope that you can find support here - I'm just lost with that.
Just want to ask - you did manage to read the original dungeon.dat and the demo part is from that?
That would already be a beginning.
Checking out all things about the object database is a thing I'm close to puzzle around soon too. But - I'm not quick anymore!
babebridou
Novice
Posts: 25
Joined: Mon May 17, 2010 6:47 am

Re: Upgrading the DM rendering engine - now with source code

Post by babebridou »

Funny you point that out. There's actually a story about the viewing height and the stairs.

How it all ended up playing like this (height of the point of view, speed, whatever) was all because of Stairs. You're totally right about stairs. The thing is, stairs are evil. I spent a dozen hours trying to solve the "stairs problem", browsing architectural websites all over the internet just to get it right. Stairs decided of all the proportions you see in the first person perspective.

Here is how I went:
1st pass, parsing the dungeon.dat file. Here the main difficulty was getting my mindset on playing with byte arrays and doing learning to code those binary operations something I never did before. I ended up with a cubic layout, with dimensions 3m/3m/3m for each "square". The feeling was very close to the one you get from original DM game, with a more or less central viewport and a familiar perspective. Then I had only walls and floor tiles.

2nd pass, I tried connecting levels together, and the nightmare began. For one, stairs are the result of a programming trick in DM/CSB. Their definition should be on 3 bits (2 bits for the direction (N/S/E/W) on the level, one bit for up/down), but unless I miss something they are only defined on 2 bits, one for "NS" or "EW", and one for Up/Down. So I had to either parse the surrounding walls to check which direction it really was. I tried that solution for a while, but I ran in another critical issue:

3rd pass, actually making the stairs. FTLs are a bunch of cheaters (and so is everyone). For them stairs are simply teleporters. I didn't want to do it their way at that point, so I tried to model actual stairs that feel right. And to achieve that, I remembered my old optional Architecture class at school, and the almighty golden rule of stair design:
60cm<2*(step height)+1*(step depth)<65cm
I tried straight stairs first. This would screw the level offsets and pit layouts badly and make the dungeon impossible to render, because the stairs would need to be like 6-7 tiles in horizontal length instead of just one. Simply impossible considering some stairs are in the middle of some levels.

I tried spiral stairs, but in order to build them you actually have to make sure that you can climb them, that is, that there is enough distance between the first step (0° rotation) and the ~270° step so you can fit more than a human body in height. It actually requires a lot of horizontal space to achieve that. in order to get enough height increments per rotation unit to be able to do a full circle, you need much, much more than 1 meters in width for your stair case. Besides, I needed a way to deal with all alignment possibilities, you could enter upstairs east and exit downstairs from east, west, north and south, so that was a no go.

So I hit my head on my keyboard a bit until I found out that I could get away with that kind of stair layout an by doubling the dimension of all tiles (6m/6m/6m). Which made some sense, considering I need to fit 4 characters/monsters on each tile. The cool thing about this stair tile is that you can also enter them from any direction and exit them from any other direction. So that solution made sense for my prototype.

Now imagine yourself in a 6m*6m*6m hallway, and you should probably get that sort of "shorty" feeling. I did all my tests using a small window from inside the unity editor.

Looking back at the result and reading your observations I guess I really should go back and scrap those daggerfall staircases of doom altogether, and go with a visual shader trick and a quick fade to black effect as a transition or something like that. But the whole "stairs" experiment was quite instructive in retrospect! The main lesson I learned here would be: "Dungeon Master Is Not A Game About Stairs" :D
----

Now about the dungeon.dat thing, yeah. In the "ressources" folder you will find a dungeondat.txt which is nothing but a renamed dungeon.dat file that Unity can open as a byte array on load. 44kb instead of 4mb for the xml version, and much, much faster. The code is in javascript and only the rendering part requires unity. It should be possible to run just the parsing of the file in a web browser without any need for unity, though I didn't actually try.
Sorry for the long post but let's explain that part:
This is the unity script: it's attached to a unity object

Code: Select all

function Start () {                                          //the function called when the object is instanciated (game starts)
	var dungeondat:TextAsset = Resources.Load("dungeondat"); // here we load the dungeon.dat file and turn it into bytes
	db = new DungeonDatabase(dungeondat.bytes);              //here we parse the bytes into a 
                                                             //new DungeonDatabase object, which is described elsewhere
	MoveToStartPosition();                                   //moves the camera to the start position
	for(k = 0;k<db.header.GetNumberOfMaps();k++){            //for each level in the database
		BuildLevel(k);                                       //create the unity3d objects for the level
	}
Here's the DungeonDatabase class

Code: Select all

class DungeonDatabase{
	var bytes : byte[];
	var header : HeaderReader;
	var mapDefs : MapDefinitionReader;
	var indexOfTilesWithObjectsOnThemReader : IndexOfTilesWithObjectsOnThemReader;
	var listOfObjectIdsOfFirstObjectsOnTilesReader : ListOfObjectIdsOfFirstObjectsOnTilesReader;
	var textDataReader : TextDataReader;
	var doorReader : DoorReader;
	var teleporterReader : TeleporterReader;
	var textReader : TextReader;
	var actuatorReader : ActuatorReader;
	var creatureReader : CreatureReader;
	var weaponReader : WeaponReader;
	var clothReader : ClothReader;
	var scrollReader : ScrollReader;
	var potionReader : PotionReader;
	var containerReader : ContainerReader;
	var miscReader : MiscReader;
	var missileReader : MissileReader;
	var cloudReader : CloudReader;
	var mapDataReader : MapDataReader;
function DungeonDatabase(bytes:byte[]){
	   ReadDungeon(bytes);
//..and do debug & testing stuff if needed
}
function ReadDungeon(byte[] bytes){
// bunch of code that initialises all the "readers"
}
}
The engine doesn't create "objects" for each elements in the dungeon. I use "readers" which parse the byte array of data in real time as fast as they can using methods that are expected. For instance to get the type of tile for coordinates i, j on level k, assuming your DungeonDatabase is "db", you just type this:

Code: Select all

var buffer:byte[] = db.mapDataReader.GetTilesBuffer(k);             //gets the buffer that corresponds to the level k tile definition
var tile = db.mapDataReader.GetBufferedTile(buffer, i, j, k);       // grabs the tile at coordinates i, j
var tileType = db.mapDataReader.GetBufferedTileType(buffer, i,j,k); // grabs the type of the tile (wall, floor, door, pit, stairs...)
if(DungeonStatics.TILE_TYPE_WALL == tileType){                      //if it's a wall...
                                                                    //...do whatever you want with it!
}
User avatar
zoom
Grand Master
Posts: 1819
Joined: Tue Sep 23, 2003 1:27 am
Location: far away but close enough

Re: Upgrading the DM rendering engine - now with source code

Post by zoom »

quite a story! wow
User avatar
Bit
Arch Master
Posts: 1064
Joined: Mon Mar 03, 2008 10:53 am
Location: Nuts trees

Re: Upgrading the DM rendering engine - now with source code

Post by Bit »

yes, there's much passion in it. I like that and those are things that should be supported - if one only could.
I can only say: because of the 'teleportic nature' of DM-stairs, the game rule is, that nothing happens on stais.
You can even abuse them to run in safety, regenerate and then slap the poor monster again. So I'd say, keep that simple idea before you get crazy. Somehow you may need to have a break between levels and different wallsets or whatever anyway.
babebridou
Novice
Posts: 25
Joined: Mon May 17, 2010 6:47 am

Re: Upgrading the DM rendering engine - now with source code

Post by babebridou »

Thanks Bit, it's super easy to get lost on technical stuff when you work on an indie project so don't beat on yourself, Game Design input & user feedback are extremely valuable! You're contributing to crucial elements of this little project (which I had put on hold for roughly a month :P). Coding is easy when you're a coder, but it's impossible to code when you don't know what to do :)

I spent about an hour and a half this morning thinking this through with a big cup of coffee. I say scrap stairs, and use ladders instead. It makes some sense actually:
1) climbing on a ladder is not really exciting, it makes sense to skip the action and the player will appreciate the fade-to-black/teleport instead of having to go through a boring & repetitive animation
2) it's pretty much the only way apart from lifts to change levels without changing the position offset
3) it leaves design space open in case there's an enthusiastic DM who wants to design a dungeon around realistic/beautiful stairs of sorts.
4) dealing with orientation is cheap with ladders and they are relatively easy to model. I'm currently making an abstract "single-cell-stairs" tile that would solve the orientation problem anyway.

RL is a bitch but I'll try to publish a new version one of these days :(
alex chan
Neophyte
Posts: 1
Joined: Tue Jun 19, 2012 6:17 am

Re: Upgrading the DM rendering engine - now with source code

Post by alex chan »

hi, bababridoe,
I hope to develop this game in Unity 3D and iPhone. .
that's why, I want to develop it with you together .
But I can't your like "www.fairyteller.com".
what's wrong?
Post Reply