Forums > The Library > Worlds.com
Figuring out the CMP/MOV format
Created by Wirlaburla
Mar 28th, 2022 at 8:30 pm (edited)
#487
Worlds has many interesting file formats and specs surrounding itself, some not properly documented, some we know too much about. But if there was one format we truly do not have any information for despite it's common and required usage in Worlds, it is the CMP/MOV format. Most people think these file-types differ due to the extension, but they are entirely the same and interchangeable. Try it out, change the extension and use it. This thread serves to be a community effort to figure out the true origins and purpose of this file format, as well as reverse engineer it for expanding upon it. I will update this post with additional info when more becomes available.

The general idea of the format is that it's a multi-frame/multi-layer compressed image format. The format allows for more than one frame in a single file, used for animations, 2D rotatable avatars, etc.

What we know so far is that the "magic" or header of the format always starts the same:
4C 7A 48 32 or in ASCII: "LvH2"
The '2' could possibly be used as a version or compression-type byte, but we know every single CMP file uses these as it's first 4 bytes and will refuse to be read as otherwise. Additionally, the 5th byte is required: C6. The 5th byte in Windows is C6 while in Mac format, it's 22. Refer to Sypha's post below:

Spoiler: Syphas work
So I did a little work on this a while back. I didn't get very far, probably because of a lack of experience with image formats, but this is what I managed to figure out. Please correct me if I'm wrong with anything, these notes are at least two years old, probably closer to four.

First 4 bytes are constant, like Wirla said. 4C 7A 48 32. Next two bytes are either 0680, 0280, 4680, and according to Wirla with some very old CMP files use C680 and 2280. I don't have notes on what this means.

Next two bytes are the number of vertical pixels. After that are the number of horizontal pixels.

Then what seems to be the loss factor, from 0 to 99. Again, two bytes. I have a question mark by this one.

Next two bytes are the number of colors.

Finally is the reduced color palette number. Again, two bytes. I really don't remember what this means. A note says 1C if pallette is standard, 2X if not. Not sure what X was.

Not so verbose:

σ - Signature (sigma)
φ - ??? (phi)
Pv - Vertical Pixels
Ph - Horizontal Pixels
κ - Loss Factor? (kappa)
C - Number of Colors
δ - Reduced Color Palette Number (delta)
http://kensmom.net/beam1.png
From the CompImg help, we can determine a multitude of information.
Spoiler: CompImg Help
CompImg Version 0.68 (Win32)
Copyright (C) 1993-95 Knowledge Adventure, Inc. All Rights Reserved

Usage: compimg <image_filespec or +image_list> {...} {options}
Compress image file(s) (any format) to paletted compressed image file(s).
Output name, by default, will be input file name, with .im# extension,
where # is lossiness factor. Pal files will ALWAYS be named with
.pa# extension, where # is 0 through 8, for 1, 2, 4, 8, ..., 256 colors.
Options [defaults shown in brackets]:
-eExt Output extension (without leading '.').
-pairs Use first filename as input name, next as output name, and so on.
-c# Colors in full resolution palette (between 2 and 256) [64].
-r# Colors in reduced resolution palette (power of 2, <= 256) [8].
-r0 Do not include reduced resolution palette.
-fName Name of fixed palette file [fixed.pa5].
-f0 Do not include fixed resolution palette.
-mName Create a movie file from the specified images, output to Name.
-MName Create a mouse-movie from a single .rel file, or several images
and no .rel file (internal .rel file); output to Name.
-ow Compress to Microsoft Windows orientation output file(s).
-om Compress to Apple Macintosh orientation output file(s).
-po{Name} Compute common palette for all images {write to Name}.
-pOName Compute common palette, write to Name, then stop.
-piName Use specified palette file for all images.
-pt Temporary palettes; don't read/write palettes from/to disk.
-pk Keep unused colors in the palette.
-px Don't include any palette information in output file(s).
-pc Don't include cuts (for synthesizing 2X zooms) in output file(s).
-pf Force palette generation, even if input image is paletted.
-pm# Memory & time used by palette generation: 0 (small & fast) to 3 [2].
-pn Instead of palette, include name of -pi palette file in image or movie.
-pw Use the 20 Windows fixed colors for the entire palette.
-d{#} Dither image (Sierra Lite), optional # is maximum dither error [8].
-da{#} Use alternate ditherer: 1=Floyd, 2=Stucki, 3=Jarvis, 4=Burkes [1].
-Xd# Scale image width to specified # of pixels.
-Yd# Scale image height to specified # of pixels.
-Go# Gamma correct image (# is floating point new gamma value).
-C#,#,#,# Crop: specify left,top,right, and bottom # of pixels to remove.
-CLName Crop images using left,top,right,bottom in named listing file.
-s#,#,# Set scale of image; x,y pixels/meter, exponent (base 2).
-l#{,#} Loss factors, higher numbers -> more loss; range 0 to 99 [8,6].
-L Allow loss-less blocks. Combine with -l0,0 for total losslessness.
-LL1 Use loss-less compression for first image of a movie or animation.
-t{#,#} Optional x,y of transparent pixel; -1 -> right/bottom [0,0].
-tc#,#,# Use transparency, but directly specify R,G,B of transparent pixel.
-td# Allow pixels to vary by # (sq. distance) and still be transparent [0].
-tC{Name} Auto. crop transparent pixels from edges of images/movies.
Optional name is output file to save cropping information.
-D Conserve disk space by reading/converting images, when necessary, twice.
-h#,# X and Y coordinate of hot-spot (only used when making a cursor).
-fps# Frames per second (movies only) [6].
-mpf# Milliseconds per frame (instead of -fps) [166].
-delay# Sound delay (in frames) (movies only) [0].
-flip Make a page-flipped movie.
-vc{#} Vary movie loss so no more than % of CPU used by pixels [50].
-vr{#} Drop images so no more than % of CPU used by pixels [50].
-vs# Set video speed (in 1000's pixels/sec) for -vc/-vr options [1000].
-vl#,# Increase loss factors by #'s when too many pixels change [1.00,1.00]
-io Also produce 6 intermediate movie files (.mi0-5) for merging.
-iO Only produce 6 intermediate movie files (.mi0-5) for merging.
-ii Merge a list of intermediate movie files (extensions ignored).
-T{Dir} Use TMP then TEMP environment vars {or Dir} for temporary files.
-qBits Inverse palette precision for movie/mouse movie (0, 5, 6, or 7) [0].
-ace Use common Accomplish options: -ow -pk -pf -pc -f0 -ecmp/-emov.
-tape Read tar files from ASPI/SCSI tape drive (use TAPEID variable).
-tapeid#:#:# Tape drive cntrlr:scsiid:unit instead of TAPEID environ. var.
-tar# Position tape to tar archive set # [1].
-tile#,# Break images into pieces no bigger than X by Y; produce .til file.
-a{#} Use animation compression for movies, # is change threshold [0].
-wca Always close window when finished (Windows version).
-wcs Close window when finished, if successful (Windows version).
-@OptionFile Read additional options from the specified file.
  • It was made in-house by Knowledge Adventure Inc, which makes sense since Worlds Inc originally came from them, even being known as "Knowledge Adventure Worlds Inc."
  • This tool has existed or atleast been copyrighted since 1993.
  • Ties to the Accomplish engine?
    • Accomplish was the name of the engine Worlds Chat and Worlds Chat Gold used, and it was developed by Worlds for Starbright.
  • Possible different types due to a Windows argument and a Mac argument?
  • Either supports being a "movie" or is just a term they use to signify more than one frame.
  • A few tape arguments?
There is a whole lot to digest in that help menu, some things we just don't understand, others confusing. Most users only use the generic spread around options they can understand.
Additionally, we know that it's very likely to be a compressed variant of the BMP format, which none is documented. The references in the documentation of Worlds to "Compressed Bitmap" and the requirement for the input files to be a BMP really says all there can be to make this conclusion. Perhaps some digging into that file format can lead down a path.

The start year interests me though. Worlds Inc. was founded in 1994, and the history between Starbright Worlds and then is not quite told in full. So much information is just full of gaps in the time frame and it makes it hard to understand. Perhaps if we got the entire story and history, we could find a clue that could bring us closer to understanding the format. This year though might mean it was developed before the engine, or the project was in the works for a year prior than what was thought. Possibly apart of some Knowledge Adventure games?

There is also a tool called CmpView, which is used to display the files. This has little new information, but it's helpful to look at still.
Spoiler: CmpView About
CmpView 1.2
By Lee Hasiuk (leeh@worlds.net)
Copyright (c) 1996 Worlds, Inc.
All Rights Reserved.
I have attempted to find Lee Hasiuk, but could not get any concrete details.

Last, but certainly not least, we have a BBC documentary.
https://youtu.be/j7f370kVRUU?t=794At the time set above, Worlds is talked about. Among a few details that are like WOW, we see Starbright Worlds a little later in the video. Starbright Worlds... has walking animations for the avatars. Now this could be a neat trick utilizing some cool programming to make it seem like it's got actual animations, but remember that this is BEFORE Worlds Chat. There was no Shaper, or Animate Actions at the time. The custom tools and utilities for world creation had to be made CUSTOM and from scratch. That means they had to specifically program these avatars to have walk animations with this trick if so, which would be a tad too much effort necessary. I am not sure if the footage is actually using real users too, or if they are "NPCs". The fact is, walking animations DO EXIST or are possible. The argument descriptions in CompImg seem to suggest it may be possible to create this, but we need much experimenting and it might not even be the case. What's interesting is that one of the "woody" avatars from a variation of WCJ has what seems to be a single "animation-frame" of woody walking, something not seen before.

Hopefully, we get the history, the information, and details about this mysterious file format. Maybe someday, you'll be exporting avatars in GIMP. Please, feel free to share ANY of your findings, your discoveries, or anything at all that you know of that'll help. And ask questions, as questions give us goals to answer!

Download links:
CompImg v0.58 (Win16)
CompImg v0.68 (Win32)
CmpView 1.2

NOTE: We currently have CompImg 0.68, which is in English, but there exists an older version in the Japanese language for Win16. I know I saw this on a Japanese Worlds page at some point but I can no longer find it. If anybody has it or know where to find it, let me know!
Mar 29th, 2022 at 1:22 am
#489
Personally, I really want to know who (or what) Shassif! is when you run the old version and why he thought that it was better the program output that instead of parameter hints.


Mar 29th, 2022 at 2:04 am
#491
Personally, I really want to know who (or what) Shassif! is when you run the old version and why he thought that it was better the program output that instead of parameter hints.


https://i.gyazo.com/2117fa80cf6b116f56ef72dcde79c5c9.png
I don't have this older version! Do you happen to still have it and if you could send it to me? We now know who actually worked on the format too because Shassif is one of the developers of Worlds Chat.
Joined06/07/21
Posts13
Mar 29th, 2022 at 2:18 am (edited)
#492
So I did a little work on this a while back. I didn't get very far, probably because of a lack of experience with image formats, but this is what I managed to figure out. Please correct me if I'm wrong with anything, these notes are at least two years old, probably closer to four.

First 4 bytes are constant, like Wirla said. 4C 7A 48 32. Next two bytes are either 0680, 0280, 4680, and according to Wirla with some very old CMP files use C680 and 2280. I don't have notes on what this means.

Next two bytes are the number of vertical pixels. After that are the number of horizontal pixels.

Then what seems to be the loss factor, from 0 to 99. Again, two bytes. I have a question mark by this one.

Next two bytes are the number of colors.

Finally is the reduced color palette number. Again, two bytes. I really don't remember what this means. A note says 1C if pallette is standard, 2X if not. Not sure what X was.

Not so verbose:

σ - Signature (sigma)
φ - ??? (phi)
Pv - Vertical Pixels
Ph - Horizontal Pixels
κ - Loss Factor? (kappa)
C - Number of Colors
δ - Reduced Color Palette Number (delta)


[edit]I can't spell[/edit]
Mar 29th, 2022 at 2:27 am (edited)
#494
I don't have this older version! Do you happen to still have it and if you could send it to me? We now know who actually worked on the format too because Shassif is one of the developers of Worlds Chat.
I think I posted this link in my misc worlds websites but the ZIP can be found here http://iso.dk.netbsd.org/pub/windows/95/winsock/Games_Fun/Alphaworld/ along with a README tutorial.
Mar 29th, 2022 at 10:50 am
#495
I would say, try and make some images that are extremely small to start with. Maybe 32x32 or something. We know what the color palette and depth will be, so using what Sypha163 gave us, we can try to edit certain bits in the middle of the image and see if it changes a color or something. Reimport that into Worlds and observe any changes in behavior. That's really generic reverse engineering advice, really, but it's the best I've got.

We have made contact with former Worlds developers in the past, so we could ask Shassif about this if he's willing to help. I wouldn't know how to do that, but I know a couple of employees have contributed to the Wikipedia article, so they do have Wikimedia accounts.
Correct me if I'm wrong, but that looks to me like an error message asking for assistance from another staff member. Kinda like that red message in Source 1 that straight-up says "grab a programmer!", hence why someone's name would be used. That, or I'm overthinking it.
Joined06/07/21
Posts13
Mar 29th, 2022 at 2:50 pm
#498
That was one of my earlier thoughts back when I was playing with this. I made different CMP files of extremely low color depth and resolution and looked at what changed. I got results I really didn't understand, though. I was expecting/hoping for a sort of bitmap RGB thing and it isn't like that. Sometimes I'd edit something and it would get darker or lighter, maybe change imperceptibly or wildly. My next thought was maybe it was using YUV and YCbCr, but I was honestly out of my depth. I started looking into it but had other priorities at the time.
Mar 29th, 2022 at 3:27 pm (edited)
#499
That was one of my earlier thoughts back when I was playing with this. I made different CMP files of extremely low color depth and resolution and looked at what changed. I got results I really didn't understand, though. I was expecting/hoping for a sort of bitmap RGB thing and it isn't like that. Sometimes I'd edit something and it would get darker or lighter, maybe change imperceptibly or wildly. My next thought was maybe it was using YUV and YCbCr, but I was honestly out of my depth. I started looking into it but had other priorities at the time.
IIRC the format might use a palette system which was common place for image formats at the time (such as GIF).

There are advantages and disadvantages to using palettes. Notably, they can reduce the file size by assigning a color to a slot, and then each pixel refers to that slot instead of repeating the entire color over and over. For images that use only a handful of colors, this saves a lot of memory and storage space, though this theoretically becomes less significant the larger the size of the palette as this would require bigger numbers to keep track of the colors.

CompImg performs a color reduction process during the conversion: https://en.wikipedia.org/wiki/Color_quantization

I think what we could try doing is creating a set of BMP files that all use the exact same set of colors, then converting them all to CMP with the same parameters. Read those in the HEX editor and see if you find any patterns that are common among all of them, that could give us hints to where the palette data is if it exists.
Mar 29th, 2022 at 4:12 pm
#500
IIRC the format might use a palette system which was common place for image formats at the time (such as GIF).
yeah isn't that what the .pa# files that compimg generates are? There's a lot of parameters for dealing with the palette files. You can use -pi to force the conversion to use a specific palette instead of generating a new one, etc.
Mar 29th, 2022 at 7:53 pm
#501
IIRC the format might use a palette system which was common place for image formats at the time (such as GIF).
yeah isn't that what the .pa# files that compimg generates are? There's a lot of parameters for dealing with the palette files. You can use -pi to force the conversion to use a specific palette instead of generating a new one, etc.
Actually a good thing to note. It makes sense, considering this file format is trying to compress the image. Perhaps it's only using one palette for the entire thing, which makes multiple images combined into one a much smaller and better space saver. You aren't storing the palettes separately.

We have made contact with former Worlds developers in the past, so we could ask Shassif about this if he's willing to help. I wouldn't know how to do that, but I know a couple of employees have contributed to the Wikipedia article, so they do have Wikimedia accounts.
The big problem is a lot of them don't have places where we can contact them or they just don't respond. I have attempted reaching Shassif in the past but have had no luck. Bo Adler and Jeff Robinson were exceptions, as Bo was very active on LinkedIn and Jeff has a twitter I was able to message him on. The rest either have so little history that tracking them down is trouble, or just don't want to talk.
Mar 29th, 2022 at 8:07 pm (edited)
#502
I do have a friend who could potentially help or knows some people who are good at this, but they're all busy with things. He's got work stuff to worry about and this other guy he knows is busy taking a break to study Japanese. I did run into a similar situation with another game engine that was completely proprietary and never properly modded before, and also ran into a dead end there.

We might be on our own here, but if you do know anyone who has experience with reversing image containers, then let us know.

Doing this ourselves, we should focus on order of difficulty first. Like suggested before, start with a very simple and one-frame CMP, until we understand how to manipulate colors and how it corresponds to the pixels inside the image itself, before we begin doing things like multiple frames. I'm away from home so I don't have access to all the tools necessary immediately, but I could try experimenting a little bit sometime this week. Or I could start now, if I can get AnyDesk to work. My internet here isn't great so sometimes streaming WorldsPlayer and all my VMs to my laptop doesn't go so smoothly, but we have plenty of time and plenty of people onboard who are interested in trying. I'm planning to create a bunch of equal images with the same colors, the only difference being the actual content inside, until I find a consistent pattern in the file they all share (presumably the palette data). Theoretically the palette shouldn't be affected by resolution, so I can then use a bigger image and test that theory.
Mar 29th, 2022 at 11:04 pm
#503
Added Sypha's post to the OP, as well as added download links to the tools on the bottom of the post.
I'll work on small CMP files and try to check out palette stuff. I'll see what I can get.
Bawoosette on the Virtual Talks discord says we could reverse engineer CompImg and CmpView instead. I personally prefer the file approach but the task is up for grabs if anyone wants to do that.
Joined06/07/21
Posts13
Mar 30th, 2022 at 1:51 am
#504
There is definitely a palette system, and an option for an external one when you make the files with COMPIMG.EXE. Like Sl0nderman said, there are a lot of options for that. It would probably behoove us to generate external palettes and compare the file with what's generated when it's included inside the file.

Something to consider with former employees is that there is very possibly a legal aspect, even if the software is archaic and no longer in use. Worlds owns the format and all aspects of the work they did while employed. They might not even be able to talk with us.
Joined06/07/21
Posts13
Mar 30th, 2022 at 2:02 am
#505
Ope. Didn't realize there was a page two already. Anyway, I started playing around with a disassembler today, IDA, but this isn't really something I have much experience with beyond just reading overviews in the past. Might be the route to go about this if someone wants to go about this from the angle Wirla mentioned, going after the executable files.
Mar 30th, 2022 at 3:06 am (edited)
#506
Something to consider with former employees is that there is very possibly a legal aspect, even if the software is archaic and no longer in use. Worlds owns the format and all aspects of the work they did while employed. They might not even be able to talk with us.
As long as you aren't looking at any leaked source code and are using a strictly clean-room approach, using only reverse engineering, and you aren't bypassing any deliberate DRM measures (of which none currently exist here), it's legal here in the States. It's what the ReactOS/WINE projects have been doing for years, as well as several console emulators.

Ope. Didn't realize there was a page two already. Anyway, I started playing around with a disassembler today, IDA, but this isn't really something I have much experience with beyond just reading overviews in the past. Might be the route to go about this if someone wants to go about this from the angle Wirla mentioned, going after the executable files.
Yeah, again I have friends who use IDA and are really good at this sort of thing, but they seem busy or uninterested, and I can't do it myself (I've tried before). Decompiling into x86 assembly and then building readable C++ code from that is a lot of work. Wirla might be able to do it, but I don't have the expertise required to do so myself.
Mar 30th, 2022 at 3:36 am
#507
Wirla might be able to do it, but I don't have the expertise required to do so myself.
I could probably do it, but I don't want to. I think reverse engineering the file format itself is the best bet. Even if I could get the decompiled C++ code from the programs, it would be a mess to read even understanding C++, and we would need to make our own version anyway due to legal issues. Reverse engineering the file format ourselves and then making our own software based on that knowledge would be a better way of doing it. We are already progressing on that road.
Mar 30th, 2022 at 11:11 am (edited)
#508
Ran into a bit of a hiccup trying to get this working on my laptop.

I don't have room for an extra VM on here and I don't wanna have to waste my phone bill streaming my main rig over the internet all day, is it possible to get this working on Arch or am I out of luck?
wine compimg.exe -c11 -r0 -f0 -ow -mtestA -t +a.lst
Arch btw

Using this image and limiting the palette to 11 colors at 16px squared:


EDIT: just saw the warning about needing to run in DOSBox, I was afraid that might be the problem.
compimg.ovl only has ever worked for me on Windows 98. XP x64, 10 x64 and now Arch have all failed to run it. Not sure how other people are doing it then

I could try skipping over the OVL by just targeting full 8-bit color, but I wanted to select the absolute minimum palette size for the colors I was using on each image since I didn't want any extra stuff crammed in there creating noise when I go to inspect the hex.
Mar 30th, 2022 at 2:55 pm
#509
Ran into a bit of a hiccup trying to get this working on my laptop.

I don't have room for an extra VM on here and I don't wanna have to waste my phone bill streaming my main rig over the internet all day, is it possible to get this working on Arch or am I out of luck?
wine compimg.exe -c11 -r0 -f0 -ow -mtestA -t +a.lst
https://media.discordapp.net/attachments/939981536093171812/958742675920650260/Screenshot_20220330_100039.pngUsing this image and limiting the palette to 11 colors at 16px squared:
https://cdn.discordapp.com/attachments/699015866519650377/958744942560292884/a.bmpEDIT: just saw the warning about needing to run in DOSBox, I was afraid that might be the problem.
compimg.ovl only has ever worked for me on Windows 98. XP x64, 10 x64 and now Arch have all failed to run it. Not sure how other people are doing it then

I could try skipping over the OVL by just targeting full 8-bit color, but I wanted to select the absolute minimum palette size for the colors I was using on each image since I didn't want any extra stuff crammed in there creating noise when I go to inspect the hex.
You shouldn't need OVL and 0.68 runs in 32-bit. Try setting another prefix with WINEARCH=win32 and run it from there. I have no issues and DOSbox does not even try to run. Additionally, make sure your BMP files are made into BMP3 instead of BMP95 using ImageMagick:
convert my.bmp -define bmp:format=bmp3 -compress none my.bmp

Actually, now thinking about it, specific functions may still be in 16-bit.
Mar 30th, 2022 at 5:21 pm (edited)
#511
You shouldn't need OVL and 0.68 runs in 32-bit. Try setting another prefix with WINEARCH=win32 and run it from there. I have no issues and DOSbox does not even try to run. Additionally, make sure your BMP files are made into BMP3 instead of BMP95 using ImageMagick:
convert my.bmp -define bmp:format=bmp3 -compress none my.bmp

Actually, now thinking about it, specific functions may still be in 16-bit.
Made a new prefix and set it as the default, didn't seem to fix the issue with the OVL.

Tried using BMP3, it allowed compimg to function but it gave me a file that isn't compatible with WorldsPlayer nor cmpview.

I thought maybe the fact I was generating with -m instead of -M was the problem, but using -M causes the program to fail to even read the file since it wants multiple frames, and I'm trying to create a single frame CMP to test with, so that won't do.

This was the output of -m (lowercase), it works fine but will not open so I can't really do anything with the file it made.
Warning: making a movie containing a single frame!
Making a non-page-flipping movie.
Making a 11 color movie.
Warning: generating a common palette for one image.
Extracting transparent color from a.bmp at 0,0.
Transparent color is Red 0, Green 0, Blue 0.
Analyzing colors in a.bmp.
Computing common palette 100%
Reading and quantizing a.bmp.
Warning: palette has no cuts; image won't be zoomable.
Compressing a.bmp (frame 1 of 1) 100%
Compression is 1.054010 bits/pixel; estimated final size 33 bytes.
Writing movie to file testA.cmp.
Compressed 822 bytes to 449 (46%).
CompImg Version 0.68 (Win32)
Copyright (C) 1993-95 Knowledge Adventure, Inc. All Rights Reserved


So far we've mostly been using this to make avatars, tiled HD textures or some kind of texture set, so I've never really run into this issue.

I don't want to distract from the subject of the thread, so if I should post this somewhere else and come back once I have it working let me know.
Mar 30th, 2022 at 5:32 pm (edited)
#512
I thought maybe the fact I was generating with -m instead of -M was the problem, but using -M causes the program to fail to even read the file since it wants multiple frames, and I'm trying to create a single frame CMP to test with, so that won't do.
yeah i never had success with -m, if i'm gonna make a single cmp file i usually use -pairs file.bmp file.cmp
i believe that -M looks for a +list.txt file input to process
Guest posting not allowed
Please log in to post a reply.