[Fixed for V0.41] DM2 demo V0.40 crashes

[Lunever]

After I had played through CSB yesterday I wanted to test the new DM2 demo V0.40. I just imported characters from CSB to DM2 an then quit the game to proceed later. When I resumed the game I first got a crash after the "loading dungeon" message, with the following diags:

RTC (V0.40) Diagnostic file - Mon Jul 03 11:37:54 2006

Error:
an access violation exception.

Stack Dump:
RTCMediaGraphics.DrawTransShadow(pointer 0x02B07800, pointer 0x02B048D0, int 0, int 0, pointer 0x0012FE88, int 0);
RTC.DrawBitmap(pointer 0x02B048D0, int 3353, int 0, int 0, int 0, word 4, float 0);
RTC.MakeWallMask(pointer 0x01F34A74, int 3353);
RTC.InitSecondary();
RTC.ProgStateIntroContinue2();
RTC.DoFrame(int 7);
RTC.WinMain();

When I loaded ot again however, at first it worked normally. On my very old laptop I can play RTC/DM/CSB fluidly without problems (at 22 FPS). The V0.39 DM2 demo had been lagging a bit (6 FPS), but since George had announced that the new DM2 demo would be designed more efficiently and has according to the V0.40 feature list also improved the general RTC performance (and indeed it seems to me that DM/CSB did run slightly smoother than before), I expected to be able to play a bit DM2.

The game does start at 13 FPS, which might not be fast, but ok. But, half a minute after the game starts, basically right after walking around once in the sleeping caverns and ascending the ladder, it suddenly drops to 0 FPS and everything, even mouse pointer movement drops to a virtually unusable slow speed. Does anyone else have this problem? Is there anything that can be done about it (except for buying a newer PC)?

My config specifications are:

; Return To Chaos
; (c)2006 George Gilbert
;
; Primary configuration file.
;
; IMPORTANT - THIS IS A TAB DELIMITED FILE; BE CAREFUL HOW YOU EDIT IT.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Game set up ;
; ;
; SCREEN_MODE - Screen mode to use ;
; SCREENSHOTS - Save just the game area (NORMAL) or include borders too (ALL) ;
; KEY_ - Define the keys for movement and spell casting ;
; ;
; The screen mode must either be set to AUTO or a valid, x-delimited, screen mode ;
; (for example 640x480x24). ;
; ;
; The keys for RTC are defined as follows: ;
; - Step Forward: KEY_STEP_FORWARD or Up Arrow ;
; - Step Backward: KEY_STEP_BACKWARD or Down Arrow ;
; - Turn Left: KEY_TURN_LEFT or Left Arrow ;
; - Turn Right: KEY_TURN_RIGHT or Right Arrow ;
; - Step Left: KEY_STEP_LEFT ;
; - Step Right: KEY_STEP_RIGHT ;
; - Cycle Character: KEY_VIEW_CYCLE ;
; - Change Character: KEY_VIEW_CHARn ;
; - Change Caster: KEY_RUNE_CHARn ;
; - Cast Rune: KEY_RUNE_n ;
; - Delete Rune: KEY_RUNE_DELETE ;
; - Cast Spell: KEY_RUNE_CAST ;
; - Pause Game: ESCAPE ;
; - Save Game: CTRL+S ;
; - Quit Game: CTRL+Q ;
; - Save screenshot: CTRL+P ;
; ;
; Allowed values for keys are as follows: ;
; - A to Z ;
; - 0 to 9 ;
; - F1 to F12 ;
; - NUMPAD0 to NUMPAD9 ;
; - INSERT, HOME, END, PAGEUP, PAGEDOWN, DELETE, BACKSPACE, RETURN, TAB, SPACEBAR ;
; - ,./;'#[]-=\` ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

SCREEN_MODE 640x480x24
SCREENSHOTS ALL

KEY_STEP_FORWARD S
KEY_STEP_BACKWARD X
KEY_TURN_LEFT A
KEY_TURN_RIGHT D
KEY_STEP_LEFT Y
KEY_STEP_RIGHT C

KEY_VIEW_CYCLE F5
KEY_VIEW_CHAR_1 F1
KEY_VIEW_CHAR_2 F2
KEY_VIEW_CHAR_3 F3
KEY_VIEW_CHAR_4 F4

KEY_RUNE_CHAR_1 Q
KEY_RUNE_CHAR_2 W
KEY_RUNE_CHAR_3 E
KEY_RUNE_CHAR_4 R
KEY_RUNE_1 1
KEY_RUNE_2 2
KEY_RUNE_3 3
KEY_RUNE_4 4
KEY_RUNE_5 5
KEY_RUNE_6 6
KEY_RUNE_DELETE 7
KEY_RUNE_CAST TAB

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Additional features ;
; ;
; DIFFICULTY - difficulty rating (0 to 15) ;
; EXTRA_MONSTERS - the number of extra monsters, per existing one, to add ;
; ;
; N.B. The game mechanics (damage done/taken, food depreciation rate etc) will be ;
; scaled if you import characters which are too strong for the dungeon as it was ;
; originally designed. ;
; On top of this you may increase the difficulty still further; Level 0 is normal, ;
; Level 5 will provide a good challenge for the seasoned player, Level 15 is nigh ;
; on impossible without Archmaster characters in every skill... ;
; ;
; An extra monster setting of 1 will double the number of monsters; 2 will treble ;
; them etc etc. ;
; ;
; The difficulty scaling and additional monsters will only applied when you first ;
; enter the dungeon, it is ignored if you resume a saved game. ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

DIFFICULTY 0
EXTRA_MONSTERS 0

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Performance options ;
; ;
; CACHE - Cache graphics (YES or NO) ;
; ANTI_ALIAS - Anti-alias scaling of the graphics (YES or NO) ;
; SCALE_CLOUDS - Smoothly scale cloud size according to its power (YES or NO) ;
; ;
; N.B. Caching only recommended for 64MB+ machines ;
; Switching anti-alias on without using the graphics cache kills the frame rate! ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

CACHE YES
ANTI_ALIAS NO
SCALE_CLOUDS NO

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Sound ;
; ;
; SOUND_LEVEL - Volume for sound effects (0 to 7) ;
; MUSIC_LEVEL - Volume for background music (0 to 7) ;
; FOOTSTEPS - Play monster foot(?)steps (YES or NO) ;
; ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

SOUND_LEVEL 1
MUSIC_LEVEL 1
FOOTSTEPS YES

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Aesthetics ;
; ;
; LOW_RESOLUTION - Use low resolution graphics like DM (YES) or best possible (NO) ;
; TRANSPARENCY - Alpha blend transparancy for some objects (YES or NO) ;
; STEPLIGHTING - Change lighting in descrete steps like DM (YES) or smoothly (NO);
; SLOW_WAKING - Change lighting as eyes adjust after sleeping (YES or NO) ;
; FIREBALL_LIGHT - Fireballs and explosions light up the local area (YES or NO) ;
; FLIGHT_ARC - Objects travel along a parabolic flight path (YES or NO) ;
; SCALE_PORTRAITS - View character portraits from a distance (YES or NO) ;
; POINTER - Mouse pointer style (PC, ST or AMIGA) ;
; ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

LOW_RESOLUTION NO
TRANSPARENCY NO
STEPLIGHTING YES
SLOW_WAKING NO
FIREBALL_LIGHT NO
FLIGHT_ARC NO
SCALE_PORTRAITS NO
POINTER AMIGA

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Diagnostic Tests ;
; ;
; BOOTTEXT - display progress through the boot-up sequence (YES or NO) ;
; FRAMERATE - display the fps when drawing the dungeon (YES or NO) ;
; ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

BOOTTEXT YES
FRAMERATE YES

[George Gilbert]

half a minute after the game starts, basically right after walking around once in the sleeping caverns and ascending the ladder, it suddenly drops to 0 FPS and everything, even mouse pointer movement drops to a virtually unusable slow speed. Does anyone else have this problem? Is there anything that can be done about it (except for buying a newer PC)?

Well, as previously discussed, you're running with an OS and laptop combination that aren't even supported by Microsoft because it's too slow, let alone me...

More helpfully, can you let me know *exactly* when it ground to a halt. Specifically, did you go out of the front door, or were you just milling about inside that initial house?

If you can send me a save game file from the point where everything is virtually stopped (use CTRL+S to save you some mouse moving!) then I can see what it's trying to do. Also, if after saving you exit, restart RTC and then load the save game again, does that put the speed back to normal again (for at least a bit)?

BTW - the config.txt file is fine; once you've got caching set to yes, then pretty much everything else has no effect on performance.

[Lunever]

Well, we had that OS discussion before. Yet, while I accept that it is responsible for the game only running at 22 FPS for DM/CSB and 6-13 FPS for DM2, I don't think it is responsible for the drop to 0 FPS. Remind - DM/CSB is fine and the V0.39 DM2 demo also did work.

If I reload the savegame it only takes a couple of seconds and the game hangs at 0 FPS. I'll send you a savegame.

[Lunever]

I did a bit further testing and after that I am even more convinced that something is going wrong in the engine. A couple of numbers:

On my very old laptop with its 300 MHz and 256 MB RAM if I shut down everything, including network connections and security software, CPU usage is below 10%, usually about 5%, despite WinXPpro. If I play DM or CSB in RTC it runs at 22 FPS most of the time. If I load the DM2 demo, no matter whether with default Torham or imported veterans, it runs as 13 FPS as long as I stay in the cellar, no matter how long, but within 10 seconds of stepping up the ladder, it drops to 0 FPS and takes about 40 seconds to react to any input. If I quit the game by <CTRL+Q> it takes even more than a minute to click away the credits.

For comparison:

On my old desktop with its 1GHz and 768 MB RAM I usually do not shut down security and have at least 1 instance of IE6 open in the background while playing RTC and the DM or CSB dungeon usually runs at 36 FPS. If I play the DM2 dungeon it often drops down to 5 FPS, especially in the rain, even if I shut down everything else (even security) before starting RTC.

So something in RTC-DM2 is eating up vast ressources beyond any comparison to other dungeons. I guess you won't notice it much on a 3GHz machine, but I think RTC-DM2 should run at least at a clean 1GHz machine. If it is not some bug that is producing this lag but just overall performance problems: George, can you please allow a config option to disable wheather and ambient light? I know the latter might be not compatible with very specific custom dungeons (but for them you can still simply turn it on again), but for DM2 it might just make the difference that keeps me from having more fun with RTC-DM2 than with FTL-DM2. 5 FPS is just to slow, and while 1GHz might not be state of the art, I think it should allow playing RTC.

[Lunever]

I got more crashes, caused by corrupt items:

There have been a couple of strange things happening in the wolves' area. Since I had read that they don't attack you much if you don't attack them, and that you can distract them by throwing the bone, and that you can make them fetch the gold pieces across the pits if you throw the bone there. Well, albeit I had not attacked them, they attacked me relentlessly, they did not show much interest in the bone, but they did fetch the gold pieces while I've been invisible, even without being motivated by the bone.

However, they brought more gold pieces than there were supposed to be across the pits. If I try to spend or change any of it, the game crashes with the following diags:

RTC (V0.40) Diagnostic file - Tue Jul 04 19:08:25 2006

Error:
an access violation exception.

Stack Dump:
RTC.UpdateItem(int 1011, pointer 0x00000012, char 10, char 39, char 2, pointer 0x05289C12);
RTC.UpdateItemStdSpell(int 1035, int 3980, char 10, char 39, char 2, pointer 0x00000000);
RTC.UpdateItem(int 1035, pointer 0x051E46B8, char 10, char 39, char 2, pointer 0x00000000);
RTC.UpdateItems();
RTC.UpdateDungeon();
RTC.ProgStateInDungeon();
RTC.DoFrame(int 50);
RTC.WinMain();

If I put one of these 10 gold pieces in a moneybox, positioning 1 gold piece automtically also repositions a second gold piece to the exact same spot in the box, so the box looks as if only 5 gold pieces were in there. Somehow these coins have been duplicated and thus become unusable. I'll send you a savegame where you can reproduce this by using the money in the box that is held by the party leader as a mouse pointer (the other moneybox in Lor's inventory is ok and only contains proper coins).

After having collected all treasure in that area I started killing wolves because they've been just hindering my movement (since I couldn't lure them with the bone), and then there was a different strange event: A wolf bit Kazai for 413 points of damage and a critical wound (which is ok since Kazai has 999 health). I let Kazai drink a healing potion and then killed the wolf. Suddenly the entirety of Kazai's inventory was dumbed to the floor, as if he had died, just that he didn't have died.

[George Gilbert]

It's not the weather or light that's causing the problems in your case - I can be absolutely certain of this because the weather isn't even started until you get out of the door of the first house...and you havn't even got that far.

When you go up the ladder, the only thing of significance that occurs is that the engine loads up the graphics required in the local area (which will all be different to those in the entrance cave area). This will mean that the total memory usage will climb significantly at that point and will be different from other RTC dungeons.

I therefore imagine that it's nothing to do with the CPU but that your laptop has just run out of "real" memory and is starting to use swap space - I'll keep looking, but that's the most likely source of the problem as the engine doesn't do anything else.

BTW - the %age CPU reported by Task Manager in Windows is only the %age of the available CPU that isn't being used by the OS, not the absolute %age of the available clock cycles on the CPU. For example if a processor is rated at 1 GHz, it means that it can handle 1 billion instructions per second; if the OS uses 800 million of those cycles and an application uses 100 million, then the Task Manager will report it as being 50% of the CPU, not 10% (which is what you might expect it to be). My point is that quoting values from the Task Manager is completely meaningless as it doesn't give any indication whatsoever of how many clock cycles are available to run RTC (or any other application).

In fact, saying that "CPU usage is below 10%, usually about 5%, despite WinXPpro" just shows how much the OS is draining from your computer, most background applications run at about 1% or less in Task Manager (i.e. in your system, there are so few clock cycles left for applications, even background ones appear as a high percentage of those available).

[Lunever]

K, just forget the CPU report.

Sure, the wheather is not freezing the very old laptop, but it might have its part in making the old desktop lag somewhat, so making it optional might help some older machines, maybe the same might be valid for ambient light and more important, wallset animations instead of static wallsets.

Since the small V0.39 demo did run on the very old laptop, but the bigger V0.40 demo does not run there, the problem (aside from that being able to afford a more current PC would be nice) is probably to be found in that very difference (since DM/CSB V0.40 runs actually smoother even there than V0.39 I don't think the freeze is caused directly by the engine).
So, am I correct if I assume that the engine is simultaneously loading ALL wallsets of 1 COMPLETE LEVEL? So a small cavern level does not cause a freeze, nor does a small single demo village level, but a big landscape level does? Would it help if a future release would cut the landscape into smaller, separate level pieces?

[George Gilbert]

Sure, the wheather is not freezing the very old laptop, but it might have its part in making the old desktop lag somewhat, so making it optional might help some older machines, maybe the same might be valid for ambient light and more important, wallset animations instead of static wallsets.

As I've mentioned many times before, the ambient light takes virtually no time to process (a few hundred clock cycles, so about 0.00001% of the time).

Interestingly, the animations take even fewer, about 20 clock cycles, so about 0.000001% of the time - cutting them out really wouldn't make any difference to the CPU.

You need to bear in mind just how graphically intensive RTC is - for example, consider a modest sized monster of about 200x150 pixels, at 32bpp that equates to 1MB per bitmap. If it's got a front / side / back and a couple of attack bitmaps then that's 5MB in bitmaps alone. A complete wallset is about 10MB of graphics (depending on how big each wall bitmap is).

<aside>
Just in case anyone's interested, the total amount of graphics used in the DMII dungeon comes to 23MB *OF COMPRESSED PNG FILES*. RTCs compression algorithms manages to get this down to 8MB in the graphics.dat files (surprisingly, PNG isn't that compressed and it's quite easy to beat it). When those graphics have to be displayed to screen though, they have to be decompressed - I've no idea what DirectX (or more importantly the memory on your graphics card) does under the covers to be able to draw them to screen, but it's probably stores the bitmaps decompressed taking up 10-100 times more memory.

By way of comparison, the memory required by RTC itself (specifically not including the bitmaps) is only about 1.5MB.
</aside>

With your old laptops memory being sucked up by the innapropriate OS, you've probably only got about 100MB spare; possibly less and it's totally unsurprising that you run out of physical memory very quickly and have to use swap space. With a slow CPU (or at least a very low number of spare clock cycles not being used by the OS), it's obvious it's going to run slowly from having to even load the graphics let alone any consideration of the engine...

Since the small V0.39 demo did run on the very old laptop, but the bigger V0.40 demo does not run there, the problem (aside from that being able to afford a more current PC would be nice) is probably to be found in that very difference (since DM/CSB V0.40 runs actually smoother even there than V0.39 I don't think the freeze is caused directly by the engine).

As you say, it's nothing to do with the engine - it's because you've run out of memory on your old laptop...

So, am I correct if I assume that the engine is simultaneously loading ALL wallsets of 1 COMPLETE LEVEL? So a small cavern level does not cause a freeze, nor does a small single demo village level, but a big landscape level does? Would it help if a future release would cut the landscape into smaller, separate level pieces?

No - as I said above, it's the local area (only about 4 tiles radius - i.e. the area that's immediately visible), so it's already as efficient as possible.

[Lunever]

Right, so forget about my idea about more config options, since that wouldn't make a difference.

Yet, what I still don't get (sorry if it should be me being a bit slow-witted here), even with much capacity being taken by the OS and too little RAM, why is it that the V0.39 demo does run, if only a 4 tile radius is relevant? I mean, the guard house does use exactly the same grafix in V0.39 as in V0.40, right? And the V0.40 engine is even more efficient than V0.39, right? Why then does V0.39 work with the DM2 demo?

[George Gilbert]

The V0.40 demo has all the graphics for the rest of the level stored in memory compressed; i.e. so the total memory required is more.

Whilst the engine only decompresses the bitmaps to their full size when they're in the local area, the compressed versions still take up a finite space - I assume that it's this that's pushing it over the edge.

Technically what I could do is for every few steps the party takes do a scan and recompress the graphics that are no longer needed so only keeping in memory the bare minimum full sized images. I very strongly suspect though that this would make it run far too slowly for CPU reasons!

[Lunever]

Hmm, right, so if you don't have some other idea, I'd either have to buy a new laptop (no cash atm), install a different OS (yet since the laptop is not really mine probably not a good option either), or simply accept that I cannot play advanced RTC custom dungeons like DM2 when not at home.

[Lunever]

Well, although I'd prefered RTC, at least I found I can play FTL-DM2 on the very old laptop by - hard to believe - running it in DosBox on WinXP. I would have thought DoxBox to take too much ressources, but, to my surprise, it does work, with only a slight lag, even during combat.

[George Gilbert]

That's because the original graphics are about 1/16th the size of those used in RTC (the images are 1/4 the size and 1/4 the bit depth) and so require much less memory to hold them all.

Given that the performance on your laptop is limited by memory usage, then that's hardly surprising

[George Gilbert]

BTW - all the crashes / problems mixed in with this thread are now all fixed.

[George Gilbert]

BTW2 - I've now made quite a few improvements to the performance and memory usage in V0.41.

This should make playing DM / CSB on your old laptop much smoother, but I'm not sure it will help with DMII as the size of the graphics is fixed and there isn't much I can do about that. At the very worst though, it will allow you to play for a little longer until you run out of memory!

[Lunever]

Well, no matter whether my old laptop will run the DM2 demo V0.41 or not, it's certainly not bad if it gave cause for RTC becoming even more efficient. Thank you anyway.