Page 1 of 1

reDM2 - Verified Reverse Engineering of Dungeon Master II (SKULL.EXE)

Posted: Tue Jul 21, 2026 9:14 am
by Cycl0ne
Hi,

Sphenx asked me some days ago in Discord if i was planning a pyDM2. And to be honest, i had a copy of DM2 for Amiga, but never played it. I think at that time i was more into PSX. But when I started it here in DosBox-x, I remembered, the setup was odd. Lots of animation on screen, the walking was different and the "brown cave" with high tech reincarnation was not the kind of "feeling" i had with the original DM.

Nevertheless, i had a look at SKProject, and since I have more time now, my pyDM runs stable, I took the challenge and looked at the DM2 code. Then I thought, maybe i should take a look at the disassembly of the game, and then i thought, ok, how could we create a reDM2 Source, since DM2 seems to be not this high quality on the data / code on Wiki Side.

So, i created a new project on my ssd and thought about how I could achieve this disassembly project with the use of maybe some AI Agents. The first try was a complete failure. I just threw some "ideas" into the agent + "have a look". After some (3-4h) crunching on the assembly, i had an executable which showed the main menu, the about (staff) and new game was working only to see just a dungeon view in brown. I noticed I couldnt get further here, this way, how i approached this wouldnt work. so i changed my complete way and created some harnesses (against the halucinations, ai peeking at SKProject or at the website to cheat about the code).

Im not sure, and here i have to say: "RESPECT Christophe for reDMCSB!", if this task is really achievable, but I think with the possibilities we have now in LLMs, it could be. So here is my share on this topic:

The target

Dungeon Master II — The Legend of Skullkeep, FTL/Interplay 1995, DOS release. SKULL.EXE is 522,637 bytes: a DOS/4GW linear executable, 32-bit protected mode, compiled with Watcom C, statically linked, no symbols. The code object splits into three one-way regions established by call-direction analysis:

Code: Select all

0x10000-0x5c97a   313,722 B   1090 fn   FTL game code
0x5c97a-0x62284    22,794 B     86 fn   HMI SOS audio library (self-contained)
0x62284-0x695AC    29,480 B    204 fn   Watcom CRT + HMI driver loaders
The game is launched through IBMIOP.EXE, a 12 KB parent process that hooks the timer, keyboard and critical-error interrupts, installs an int 0xfc service handler, then EXECs the game and services its requests for the child's lifetime. Video output never touches hardware directly — it goes through that interrupt. This matters later.

Why byte-matching failed
The original plan was matching decompilation: write C, compile it, and prove correctness by byte-identical output. That requires the original compiler.

The oldest obtainable Watcom is 11.0c (1998); the game is from 1995. Calibrating 11.0c against its own shipped clib3r.lib — roughly 150 flag combinations plus two hill-climbs — plateaued on a residual where every instruction matched in opcode, immediate and position, and only the register allocation differed: every ebx was ecx and vice versa. No command-line option controls that. The library objects in the archive were themselves built by an earlier compiler than the shipped binaries, which is also why zignatures from those objects match a 1995 game that 11.0c's own code generator cannot reproduce.

Byte identity was therefore unreachable, and the project goal became functional equivalence — which needed a different kind of evidence.

What I built

Tier 1 — differential testing (tools/difftest.py)
Unicorn-based. Both the original function and the C port run in separate emulator instances, from identical initial state, driven with the same inputs. Compared: EAX exactly, the complete set of memory writes (captured via UC_HOOK_MEM_WRITE), write ordering, stack balance, and callee-saved register preservation.
The write-set comparison is the important half. A port that returns the right value but writes one byte too many fails, naming the buffer and offset. Side effects cannot hide.
Ports are compiled with native Open Watcom against the original's calling convention (__watcall: EAX, EDX, EBX, ECX) and linked as a DOS/4G LE with a map file, so build-side addresses come from symbol lookup rather than being assumed identical to the original's.

Tier 2 — state snapshots (tools/objstate.py)
Most functions read global state, which made them untestable in isolation. Rather than synthesising that state, the fixture runs the original's own load path in the emulator: savegame_load → dungeon_load → seven packed-bitstream records → object_state_load, then the startup stretch through the first gameplay draw. Only five file-I/O leaves are stubbed — functions whose entire body is an int 21h.

The result is a real heap with real globals, dumped as a snapshot both difftest sides preload identically. Functions testable: 20 → 77.

Getting there required serving int 0xfc. IBMIOP.EXE was unpacked by loading its MZ image in 16-bit Unicorn and letting its own LZEXE stub decompress itself — no algorithm reimplemented. Its dispatcher accepts 28 codes, 21 of which are no-ops; the implemented handlers are 7–23 byte wrappers. The driver parks bare data pointers in unused interrupt vectors, which is why the game reads a mode-info struct through what looks like a null pointer.

Tier 3 — fixture runs
For functions too entangled to isolate, the original runs over a reconstructed fixture and the resulting heap is inspected. This establishes format understanding, not equivalence of any port — stated as a limitation rather than implied.


Discipline that turned out to matter
Four rules, each written after the corresponding failure.

Known answers. Original-versus-port proves agreement, not correctness — a systematic fault in the snapshot makes both sides wrong identically and the test stays green. Every spec carries at least one case whose expected value was derived independently of the emulation, ranked by independence: a data-file value beats a hand walk over snapshot bytes, which beats a second tool. Reading either emulated side counts for nothing. Adding this immediately exposed a test case that had been mislabelled since the harness's first day and had passed its entire life on agreement alone.

Exact-EAX policy. Watcom frequently leaves garbage in the upper half of return registers because callers only read AX. When those bits track program state — a pointer half, a count — reproducing them is a free assertion the return type doesn't expose, and it caught 31 of 569 cases in one break test. When they're a register-allocation artifact, no source-level C can match them. The discriminator is empirical: sweep and see whether the bits track anything.

Write order. Inverted default: order is artifact until shown observable, because nothing can observe the state between two stores in a non-interruptible function. Values and the final write set are never relaxed.

The analysis scripts themselves. All three tiers verify the original's behaviour; none verifies the script that summarises it. A tie-break in an aggregation script once attributed the wrong category on a single-element set, was recorded as an anomaly, and the next task was framed around explaining it. Nothing crashed, no test failed, no number looked wrong. Countermeasures: aggregation must report ambiguity rather than resolve it, and a surprising result gets checked against raw data before it becomes a finding. Retractions happen in place, where the claim was recorded — a finding that quietly disappears is worse than one never made, because the next reader can't tell which claims were checked.

-----

Crunching now for around 20h.... I found a first bug in the code, it seems FTL copied their blitter routines and modfied the copies, not chcking all paths:

Bug in blit4_copy_mirror_masked (fcn.00011a72)
The function performs a horizontally mirrored, masked 4bpp surface copy — one of four blit variants forming a {plain, mirrored} × {opaque, masked} quartet:

Code: Select all

Address	Function		Variant
0x11504	blit4_copy		plain, opaque
0x116b0	blit4_copy_mirror	mirrored, opaque
0x1185e	blit4_copy_masked	plain, 	masked
0x11a72	blit4_copy_mirror_masked	mirrored, masked
The defect. On the same-parity path, the trailing (odd) pixel is drawn only when the source nibble equals the transparency key — the branch condition is inverted relative to all three sibling functions. The instruction is a jne where the siblings have a je.

Effect. The last odd pixel of a mirrored masked blit writes the key colour instead of leaving the destination untouched. Transparency fails exactly where it should apply, and applies exactly where it shouldn't.

Why it survived. It requires same parity and an odd trailing pixel count, so it manifests as a single incorrect pixel at one edge of a flipped sprite. The crossed-parity path is unaffected — it jumps into 0x1185e's correct je, so only one of the two paths carries the defect.

-----

So here I stop with so much text. Sorry, Ill keep you informed.

Re: reDM2 - Verified Reverse Engineering of Dungeon Master II (SKULL.EXE)

Posted: Tue Jul 21, 2026 12:34 pm
by ChristopheF
I would say FTL probably compiled SKULL.EXE with Watcom C version 10.0a or 10.5, both available at: https://winworldpc.com/product/watcom-c-c/70
Version 10.6 was released in 1996, so after DM2 for PC release date in August 1995.

There is a DOS program called UNLZEXE.EXE that can uncompress executables compressed with LZEXE.EXE (LZ91). You can find both it in the ReDMCSB archive, and of course need to run them in Dosbox.

Re: reDM2 - Verified Reverse Engineering of Dungeon Master II (SKULL.EXE)

Posted: Wed Jul 22, 2026 7:27 pm
by Cycl0ne
Hey Christophe, no problem. since the IBMIO.exe is packed and needs to unpack itself, claude picked the unpacker out of its own source. ;-)
But the compiler will be a good aproach later to check against Binary.

I created a repo now in codeberg. There you can see the progress
https://codeberg.org/CyCl0ne/reDM2

I gave sphenx my init script to create the harness for the AI ..

The progress after 3rd day (im not allways on the computer.. but since the harness says: Ask Human, sometimes the Agent stalls):
From 1090 Functions: 146 in C Files, 194 Difftests (TOML) and 292 Named Functions.

If it continues like this, then im in 15-18 Days through. Fascinating.. for 300kb compiled Code.

Thanks, i gave the Agent the link to the compiler and hes now trying against it:

Code: Select all

The 10.0a ISO is 138 MB with two mirrors. Let me resolve the actual file URL (HEAD request following redirects).
While that downloads, let me extract the exact byte-sequences of the confirmed crt_* functions from SKULL.EXE — the comparison targets for the library-match test. Let me get the strongest byte-matched ones from the memory table.
Download finished. Let me verify it and read the CRT stripping methodology (the exact procedure used for 11.0c) so I replicate it correctly for 10.0a.
Download verified (138 MB, valid). The method is clear: extract OMF modules with wlib, byte-compare specific functions (strcmp_, strcpy_, filelength_, …) against SKULL.EXE's crt_* addresses. Let me see the archive's structure.
The 7z contains WATCOM_C10A.ISO (303 MB). Let me extract it, then pull the library files from the ISO (no compiler needed).
Found WATCOM/LIB386/DOS/CLIB3R.LIB (187392 B, dated 1994-09-01) — the 32-bit DOS register-convention C library. Let me extract it and the math libs.
All four 10.0a libs extracted (CLIB3R.LIB is the key one, dated 1994-09-01). Now I'll extract the OMF modules with wlib and compare against SKULL.EXE — exactly the 11.0c method. Let me first confirm wlib works and list the modules.

strcmp matches byte-for-byte: cmp386.o's LEDATA (5351 89c3 39d0 746c 8b03…) is identical to SKULL.EXE @ 0x62d10 — 10.0a's library reproduces the shipped code. Now I need the systematic comparison across all
  functions (with relocation masking). Let me write an OMF parser and compare all 10 modules against SKULL.EXE.

Library-match result: 10/10 byte-exact (all reloc-masked) — every tested crt_* in SKULL.EXE is byte-identical to 10.0a's CLIB3R.LIB, and 11.0c gave identical results on shared modules. But these are all
  ASM/syscall-wrapper functions that match any Watcom version — they don't discriminate 10.0a from 11.0c. The register-allocator question needs a C-compiled function. Let me find one where the two libraries
  actually differ (still no compiler needed).

Discrimination found! The C-compiled library functions differ between generations: atoi (87B vs 71B), bsearch (107 vs 118), memcpy/memcmp/strchr/strncmp all differ; only strlen is identical. Now the decisive
  test — which generation does SKULL.EXE contain? Let me get the byte patterns and search SKULL.EXE for each.

Decisive: SKULL.EXE contains the 10.0a register order (89c7 89d6 89d9), NOT the 11.0c order — the 11.0c pattern isn't found anywhere. Let me confirm this is the actual CRT memcpy (not inlined game code) by
  finding the full function near the memcpy tail at 0x68c96/0x6415e, and check the whole function matches 10.0a.

The CRT memcpy variants in SKULL.EXE (0x64147, 0x68c88) all use the eax/edi-first ordering (89c7 89de, 89c8 c1e902 f2a5) — the 10.0a fingerprint — and the tail matches. Let me confirm across multiple
  discriminating functions (memcmp, atoi, bsearch, strchr) with a batch search of each version's distinctive bytes.