Gfx Extract from Lands of Lore and .shp file format how to

DK · Post by DK » Sat May 03, 2008 7:38 pm

Yep!
Lots of post recently

So, a very quick reply.

Yes, changing palette for SHP work only for current frame because the palette for ALL frames is applied only when clicking the first time on SHP.
So, when you find the correct palette, just select it, then click again on SHP and the selected palette is applied correctly to all frames.

About taking screenshot, yes, maybe it's a faster solution, and yes, DosBox 8bit mode screenshot are correct (but do not use any scaling system).

And FIREBALL.SHP is my nightmare

I'll try to fix some bug those days...

Post by **Gambit37** » Sat May 03, 2008 8:26 pm

I did a quick test of grabbing screenshots from DosBox and it works -- but of course it's a standard 8-bit palette, while VGA only used 6-bits.

I'm just in the process of writing a little routine to convert this to something usable for Westpak.

ShaggyDowns · Post by **ShaggyDowns** » Sat May 03, 2008 8:36 pm

Yeah,

I got the cave dweller pal working as well. After messing around some more I had a few others work also. Still, sometimes parts of the monsters bodies are not correct. For instance, I found the ratman color pal, but the claws, toenails and mouth were incorrect! They were like a bright green, while the rest of the image was colored correctly. Another one would be the amazon, everything is correct except the torso, etc. etc.

Since DK added the extra zoom feature the LOL monster images are scaled correctly as is and work perfectly when placed into RTC. I don't export to the sheet using WestPak because then the images become small again. What I do is use a free program called SnagIt, which allows me to cut an image right from any window on my desktop or a webpage and save it. I love SnagIt, it works wonders!

Lately I have been running LOL in DosBox window mode and turning the game speed down all the way. I then cut images of the monsters using SnagIt from different angles. I also do this with wallobjects, items, etc. You can also take a screenshot using DosBox with ctrl-F5.

As far as scaling goes, the images I cut using SnagIt are already perfectly scaled to be used with RTC. I haven't had any problems with having to enlarge or shrink the monster images yet. I have yet to try the Dungeon Unpacker program out, but I will give it a shot and let you know what I come up with.

~SD~

Post by **Gambit37** » Sat May 03, 2008 9:14 pm

Dragon Unpacker won't help with colours or anything, it's just a useful way of getting the SHP and CPS files out of the PAK files -- Westpak tends to be too buggy to do that without restarting again and again.

Anyway... I did a test and took a screenshot of Yvel from the game. Then I opened that screenshot in PhotoShop, saved it's palette as an ACT file, then ran my program on it which converts it to a PAL file WestPak can use.

So I then open WestPak, browse to my folder where I've extracted all the SHP and CPS files. Click on YVEL.SHP and load the palette and hey presto -- it's (more-or-less) perfect!

For scaling issues, that doesn't bother me as I resize everything in PhotoShop as necessary.

DK · Post by DK » Sat May 03, 2008 9:39 pm

I did a quick test of grabbing screenshots from DosBox and it works -- but of course it's a standard 8-bit palette, while VGA only used 6-bits.

Import palette from PNG should already work in Westpak... I'll check it better....

Post by **Gambit37** » Sat May 03, 2008 9:55 pm

Oh right! I didn't know that, it does say that on the palette load screen, just *.COL or *.PAL.

I just tried it -- yes, it works!

Post by **Gambit37** » Sun May 04, 2008 1:59 pm

Well, I've managed to extract a bunch of stuff, but some palettes definitely don't work correctly, no matter what you do.

For example, The Amazon, Thug, Bandit and Orc Leader never look right. And even after I got the right palette for Yvel and applied it to the beasts from that level, those are still not right.

Others are also wrong, but not massively so -- wouldn't take too long to fix up in PhotoShop.

There is obviously a lot of internal palette mapping going on in the game, so I don't think we can get these perfect using just extraction tools.

DK: Do you know how to extract the wall graphics? I can't seem to find these anywhere, or they are in a format that is not viewable?

ShaggyDowns · Post by **ShaggyDowns** » Sun May 04, 2008 6:00 pm

Yeah,

That's what I kept on finding out trying to match up the pal colors with the monsters. As far as the walls go, I can't seem to find those either! I was wondering what the .VCN files were? Again, it has been quite a lot easier for me to use DosBox and grab the images that way.

~SD~

Post by **Gambit37** » Sun May 04, 2008 6:12 pm

The animation extract of WSA files works pretty well, once you have the right palettes:

DK · Post by DK » Sun May 04, 2008 7:10 pm

Well, I've managed to extract a bunch of stuff, but some palettes definitely don't work correctly, no matter what you do.
For example, The Amazon, Thug, Bandit and Orc Leader never look right. And even after I got the right palette for Yvel and applied it to the beasts from that level, those are still not right.
..
There is obviously a lot of internal palette mapping going on in the game, so I don't think we can get these perfect using just extraction tools.

You're right! The decompression code include a list of pointer to palette, so if this list is decoded incorrectly, you do not have the right color, also with a correct palette.
Can you send me the screenshot from the game that you have found incorrect? So I can try to check better the mapping table.

DK: Do you know how to extract the wall graphics? I can't seem to find these anywhere, or they are in a format that is not viewable?

Yes, I've the docs to decompress them (they are similar to old EoB but a little more advanced) but no time to implement.

Indeed, I've made some try, but they do not show up correctly, need more work.

Post by **Gambit37** » Sun May 04, 2008 8:31 pm

OK, here's an example -- 8 bit screenshots from dosbox:

The same creature has different palettes applied from the current palette to make it look like there are more varieties.

If you look at the palette between Yvel and the Woods, above, you can see that the palettes are different, yet the colours in the creatures remain the same due to palette index mapping.

Here is the exported sheet from WestPak of the same creature, using the correct palette applied in WestPak -- you can see it's messed up:

However, in the extracted image, you can see they included a little swatch of the colours to be used in re-mapping, to the right of the last frame of the Orc. I guess this is where the problem is -- WestPak doesn't know about these extra colours for some creatures right?

I don't know if these shots help you at all?

DK · Post by DK » Sun May 04, 2008 10:58 pm

Thank you for the samples; they have been very useful!
Now remember, in the palette table for each SHP frame, I've applied only first 16 colors (in fact some are correct, and others not), because some SHP will not show up correctly.
Now, that I have enabled full palette, the image show up correctly.

For the color swatch, it can be useful to change color for monster, but it can be an interesting bonus function

Just replace the second column and you have new skin color.

Here, if you wanna try, I've put the Westpak2 with full palette for SHP (not limited to first 16 colors). Unfortunately, now, some SHP will not decompress anymore.
Let me know what they are, because I forgot them.

Thank for your collaboration.

P.S. To moderators of this forum: it's ok continue this discussion here?
I can use the forum on sf.net but it's empty

so, if I do not bother too much, I'd like continue here. Thanks.

ShaggyDowns · Post by **ShaggyDowns** » Sun May 04, 2008 11:30 pm

Nice,

It's working a lot better now! For me, most of the monsters that I had trouble with before are working correctly. Such as the amazon, minotaur, and bandit to name a few. Awesome work guys, thanks!!!

~SD~

Post by **Gambit37** » Sun May 04, 2008 11:49 pm

It's fine to continue discussing this here (I'm a mod).

I'll check out the new version, thank you for the very quick amendments.

ShaggyDowns · Post by **ShaggyDowns** » Mon May 05, 2008 12:18 am

Like DK said,

Some of the .SHP files have errors and will not open now. Although, some of the ones I was really interested in are working fine now! I have yet to try them all, but will put a list together of the ones that don't work when I get a chance.

~SD~

Post by **Gambit37** » Mon May 05, 2008 12:48 am

This is really cool -- I now have all the monsters in their correct palettes!

The only ones that still don't look right are:

TCABAL.SHP
WRAITH.SHP
APARITIN.SHP

...but I think this is because they use a lookup table to make them appear to be semi-transparent in the levels.

Monsters that crash WestPak are:

GUARD.SHP

As you said DK, lots of other SHPs now don't work. These seem to be the ones that are chapter decorations. I was able to extract these all before OK in the previous version.

I shall post up a zip file with them soon.

Post by **Gambit37** » Mon May 05, 2008 1:20 am

OK, here's a zip of the monsters, in all their correct palettes -- note that these are saved as sheets in their original size (not doubled).

If you haven't played the game yet, I'd suggest not to download this as it will spoil it for you!!!!

http://dmweb.free.fr/Stuff/lol/lol-creatures.zip

ShaggyDowns · Post by **ShaggyDowns** » Mon May 05, 2008 1:33 am

Yes,

I haven't been able to even access guard.shp with the new version where DK added the zoom feature or this other version from today. However, I could access it with the very first version I d/led prior to the zoom feature version. The colors were still incorrect, but it was accessible. You are definately right about the wraith & apparition shp. I haven't been able to get them correct either.

Gambit37 wrote:OK, here's a zip of the monsters, in all their correct palettes

SWEET!!!

Thanks again guys!

~SD~

Post by **Gambit37** » Mon May 05, 2008 1:46 am

If you look in the level .PAK files, there is an .INI file that specifies which file to load the palette from. I used this information to apply the correct palettes, so WRAITH etc. do in fact have the correct palettes applied in my zip -- they just aren't used in a regular way by the game engine.

DK · Post by DK » Mon May 05, 2008 10:01 am

The only ones that still don't look right are:
TCABAL.SHP
WRAITH.SHP
APARITIN.SHP
...but I think this is because they use a lookup table to make them appear to be semi-transparent in the levels.

Do you have screenshot from the game for these? They can be useful.

Monsters that crash WestPak are: GUARD.SHP

I'll check

OK, here's a zip of the monsters, in all their correct palettes.
If you haven't played the game yet, I'd suggest not to download this as it will spoil it for you!!!!

Great!
No problem for the spoiler, I've already played LoL1 over ten years ago

If you look in the level .PAK files, there is an .INI file that specifies which file to load the palette from. I used this information to apply the correct palettes, so WRAITH etc. do in fact have the correct palettes applied in my zip -- they just aren't used in a regular way by the game engine.

Thanks, very interesting!
I still haven't look better at the other data files, so, I can use those informations from .INI to apply the correct palette, good!

P.S.

It's fine to continue discussing this here (I'm a mod).

Thanks.

Post by **Gambit37** » Mon May 05, 2008 3:17 pm

DK wrote:Do you have screenshot from the game for these? They can be useful.

No, I don't think I have any save games from near there. I would have to play it all again...

DK wrote:Thanks, very interesting! I still haven't look better at the other data files, so, I can use those informations from .INI to apply the correct palette, good!

The INI file doesn't always specify a palette (*.COL) but it usually specifies a level/data file to use (FOREST or TOWER for example) -- from that information, I knew which screenshot to load a palette from.

DK · Post by DK » Mon May 05, 2008 4:14 pm

No, I don't think I have any save games from near there. I would have to play it all again...

ok, I'll try to find something on the web as a reference.

The INI file doesn't always specify a palette (*.COL) but it usually specifies a level/data file to use (FOREST or TOWER for example) -- from that information, I knew which screenshot to load a palette from.

I see... but can be useful

I've a little update the 0.70exp version (the link is the same as above), now almost the SHP work again, with correct palette.
Still GUARD.SHP give a memory allocation error...
The problem is:
for each SHP frame, there is a palette index (I've called ActorPalette because first time I've found it is in Kyrandia3 sprites).
This "ActorPalette" is index to main palette, so to find out the real color you must check the byte value of the image with this table and not directly with the palette table.

For example:
Color 14 of ActorPalette is equal 56.
A pixel color of the image with value 14 is not the color 14 of the main palette, but is the color 56 of the main palette, because the color is referred to ActorPalette.
The problem is, if the pixel color is greater of the ActorPalette range (and for some SHP will be), which is the right color?
For ex. if the ActorPalette is max 16 values, a color 229, cannot exist into ActorPalette.

In this version, I've made this solution.
If the color is in the range of ActorPalette, then use it for conversion, if the color is out of range, use it directly with main palette.
So, some color, can be incorrect.
But, I've tried STARTUP.PAK -> GAMESHP.SHP and seems ok. (The ActorPal is 16 index, but a pixel color into frame 89, the last, is 229, so...).
Hope you can understand those technically informations...

Post by **Gambit37** » Mon May 05, 2008 4:52 pm

I found a save game in the White Tower -- here's some screenshots of the wraith, apparition and snake spell (a WSA):

I can't remember where TCABAL shows up though!

Post by **Gambit37** » Mon May 05, 2008 5:09 pm

Here is the VIPER.WSA as an animated GIF -- you can use this to get the unmapped palette -- this might help you work out the transparency mapping table?

Here it is as a PNG sheet too:

http://dmweb.free.fr/Stuff/lol/viper.png

VIPER.WSA is in GENERAL.PAK (as are all of the spell effects)

Post by **Gambit37** » Mon May 05, 2008 5:37 pm

Aha --- I just realised my mistake for WRAITH, APPARITION and TCABAL. Like the animated spells, these need to have the FXPAL.COL palette applied to them!

In fact, I think the WRAITH uses a combination of palettes -- the one for the WHITETOWER + FXPAL -- because part of it is shown full colour and part of it semi-transparent:

zoom · Post by **zoom** » Mon May 05, 2008 5:37 pm

cool, some screenshots !

Post by **Gambit37** » Mon May 05, 2008 6:09 pm

Have you never seen LoL before? It's pretty good and works very well under DOSBOX. You can get the full game from here:

http://www.the-underdogs.info/game.php?gameid=1324

By the way to others reading this thread for the images: I've updated the ZIP file above with the latest palette corrections

ian_scho · Post by **ian_scho** » Mon May 05, 2008 6:32 pm

Gambit37 wrote:By the way to others reading this thread for the images: I've updated the ZIP file above with the latest palette corrections

You are psychic, thanks!

Post by **Gambit37** » Mon May 05, 2008 6:53 pm

No problem

Once I've got the full game graphics extracted, I'll put a full zip in Joramun's resources thread in the Custom Dungeons forum.

I was supposed to be doing somehting entirely different today, but never mind -- it's good to get some progress on this, it's been bugging me for years!

zoom · Post by **zoom** » Mon May 05, 2008 9:08 pm

I know LoL..
been rather some time since
I have played it however.
will try out with dosbox, thanx.

btw, the party does not look
very well at the screenies