Forums > The Library > Worlds.com
Figuring out the CMP/MOV format
Created by Wirlaburla
Mar 30th, 2022 at 6:01 pm (edited)
#514
Using the header information of 3 files I generated, according to Sypha, there are some inconsistencies.

The first 3 bytes are all the same, but strangely, the filesize, the number of colors and the reduced color palette number all vary slightly between images despite them all using the exact same set of colors and the same color limit:

08 00 28 18


0B 00 29 18


08 00 29 18

The blue one was missing a couple pixels on the crosses, which could be lossy compression messing with me.

Unfortunately I couldn't exactly find where the palette data lived in each image, but I have a feeling it's somehow connected to this stuff at the end.

It's a different length in each file, I'm trying to find where it starts and ends but the only clue I can seem to find is that it consistently starts around the 0000:00F0 range in a 16x16 image. It seems to end wherever it wants to, and I can't find any common patterns or anything in common between them.

The settings I used which sl0nderman suggested (they worked by the way, thank you for the tip) generated a separate palette file, that might be worth looking at.

A copy of any important files just so we are on the same page:
https://files.worlio.com/users/bonkmaykr/a.cmp
https://files.worlio.com/users/bonkmaykr/b.cmp
https://files.worlio.com/users/bonkmaykr/c.cmp
https://files.worlio.com/users/bonkmaykr/a.p11
https://files.worlio.com/users/bonkmaykr/b.p11
https://files.worlio.com/users/bonkmaykr/c.p11
Mar 31st, 2022 at 7:39 pm
#519
Using the header information of 3 files I generated, according to Sypha, there are some inconsistencies.

The first 3 bytes are all the same, but strangely, the filesize, the number of colors and the reduced color palette number all vary slightly between images despite them all using the exact same set of colors and the same color limit:
https://cdn.discordapp.com/attachments/699015866519650377/958846456423407676/a.bmp08 00 28 18

https://cdn.discordapp.com/attachments/699015866519650377/958846456247230514/b.bmp0B 00 29 18

https://cdn.discordapp.com/attachments/699015866519650377/958846456041734214/c.bmp08 00 29 18

The blue one was missing a couple pixels on the crosses, which could be lossy compression messing with me.

Unfortunately I couldn't exactly find where the palette data lived in each image, but I have a feeling it's somehow connected to this stuff at the end.

It's a different length in each file, I'm trying to find where it starts and ends but the only clue I can seem to find is that it consistently starts around the 0000:00F0 range in a 16x16 image. It seems to end wherever it wants to, and I can't find any common patterns or anything in common between them.

The settings I used which sl0nderman suggested (they worked by the way, thank you for the tip) generated a separate palette file, that might be worth looking at.

A copy of any important files just so we are on the same page:
https://files.worlio.com/users/bonkmaykr/a.cmp
https://files.worlio.com/users/bonkmaykr/b.cmp
https://files.worlio.com/users/bonkmaykr/c.cmp
https://files.worlio.com/users/bonkmaykr/a.p11
https://files.worlio.com/users/bonkmaykr/b.p11
https://files.worlio.com/users/bonkmaykr/c.p11
I've been busy with other projects as of late but I did experiment and I had the same situation. I did not test with the loss quality argument.
Joined06/07/21
Posts12
Apr 4th, 2022 at 12:33 am
#523
Just to be crystal clear, what is the exact command everyone was using to test? There are a few default things that might be cutting out unused colors or increasing compression, confounding attempts to find patterns. There are also a LOT of options for palettes.

I'm going to start playing around a little with this again too. I probably didn't fully explore the header, FWIW. There are a LOT of options in the man page and it would make sense for it to be larger than what I posted.

From man page - -l#{,#} Loss factors, higher numbers -> more loss; range 0 to 99 [8,6].

Default is some loss.
Apr 4th, 2022 at 2:05 pm (edited)
#524
Just to be crystal clear, what is the exact command everyone was using to test? There are a few default things that might be cutting out unused colors or increasing compression, confounding attempts to find patterns. There are also a LOT of options for palettes.

I'm going to start playing around a little with this again too. I probably didn't fully explore the header, FWIW. There are a LOT of options in the man page and it would make sense for it to be larger than what I posted.

From man page - -l#{,#} Loss factors, higher numbers -> more loss; range 0 to 99 [8,6].

Default is some loss.
Using a fixed loss value should remove the randomness factor that causes discrepancies between the images.

Here is the bash script I was using:
#!/bin/bash
wine compimg.exe -c11 -r0 -f0 -ow -ecmp -pairs a.bmp a.cmp
wine compimg.exe -c11 -r0 -f0 -ow -ecmp -pairs b.bmp b.cmp
wine compimg.exe -c11 -r0 -f0 -ow -ecmp -pairs c.bmp c.cmp
wine compimg.exe -c11 -r0 -f0 -ow -ecmp -pairs d.bmp d.cmp


If you're on Windows then you can remove the first line and remove wine before the rest, it should work just the same.
Apr 4th, 2022 at 3:23 pm
#525
Using a fixed loss value should remove the randomness factor that causes discrepancies between the images.

Here is the bash script I was using:
#!/bin/bash
wine compimg.exe -c11 -r0 -f0 -ow -ecmp -pairs a.bmp a.cmp
wine compimg.exe -c11 -r0 -f0 -ow -ecmp -pairs b.bmp b.cmp
wine compimg.exe -c11 -r0 -f0 -ow -ecmp -pairs c.bmp c.cmp
wine compimg.exe -c11 -r0 -f0 -ow -ecmp -pairs d.bmp d.cmp


If you're on Windows then you can remove the first line and remove wine before the rest, it should work just the same.
Be aware that I've noticed some issues using compimg on Wine. Not entirely sure what the deal is, but I've gotten pretty inconsistent results comparing it with a Windows VM.
Apr 27th, 2022 at 11:53 am (edited)
#617
Sorry for not working on this more, been busy doing a million things as usual. Thanks ADHD.

Well, I found documentation for textures in Renderware 2. This isn't the MOV/CMP container, but I figured I should go ahead and link it here anyway incase it has a few clues. The WorldsPlayer might still honor some of these requirements listed in the documentation. A lot of custom file formats are just variations of existing ones, after all.

This is from part of a manual designed for programmers working with the Renderware libraries for Windows, DOS, and Macintosh. It's not as comprehensive as the Renderware 3 docs I also found, but that shouldn't matter as most of it is related to the rendering process and not so much the format of assets on disk. RW3 is also from the 6th console generation and definitely not what Worlds is using.

Scroll down to the section "The Texture File Formats", it goes into very brief detail on what image formats RW2 natively supports out of the box.
http://www.tnlc.com/rw/api/rwdoc.htm

Not sure if this helps, but I found this interesting in particular:
RenderWare’s own texture file format is based on the format used for Sun Microsystems’s rasterfiles. RenderWare will read any legal Sun Rasterfile. For a description of Sun’s Rasterfile format see:

[1] Rasterfile (5) in Sun Microsystems, Sun OS 4.0 Programmer’s Manual, 1990

[2] Pat McGee, Format for byte encoded rasterfiles in Sun-Spots Digest, Volume 6, Issue 84.

[3] David. C. Kay and John R. Levine, Graphics File Formats, Windcrest/McGraw-Hill, 1992.

For a description of MS Windows bitmap (.bmp) file format see:

[4] Bitmap-File Formats, Microsoft Windows 3.1 Programmer’s Reference Volume 4: Resources, Microsoft, 1987-1992. Part Number: 30211.

[5] David. C. Kay and John R. Levine, Graphics File Formats, Windcrest/McGraw-Hill, 1992.
I have an idea. Could someone get a copy of a Renderware game from the mid or late 90s and maybe take a look at it's .ras files, if they aren't using custom formats too? We might find some container similarities between Worlds CMP and RW RAS. Scorched Planet looks like a good start.
Apr 27th, 2022 at 4:16 pm
#618
Well, I found documentation for textures in Renderware 2.
I have doubts, mostly because Worlds Chat didn't use RenderWare at all and used it's own engine named "Accomplish". A lot of custom texture formats of the day also used variations of BMP because it was a good base and damn simple to convert from considering how basic it is. Am interested in seeing how this goes though. Won't hurt to try.
May 8th, 2022 at 7:26 pm
#650
I don't know if this adds anything to all this, but I just noticed that if I set another value to -f, the cmp image gets smoother.

compimg.exe -c255 -r0 -f50 -ow -ecmp -pairs example.bmp example.cmp
May 9th, 2022 at 8:55 pm (edited)
#652
I don't know if this adds anything to all this, but I just noticed that if I set another value to -f, the cmp image gets smoother.
https://cdn.discordapp.com/attachments/904111766068412427/973001469269770300/unknown.pngcompimg.exe -c255 -r0 -f50 -ow -ecmp -pairs example.bmp example.cmp
The help output already tells us what -f does.
-f0 excludes the "fixed resolution palette". The image is blurrier because the color detail has been reduced.

I probably should have taken note of this sooner, but I figured it was already known since "-f0" is written and people are using it to make CMPs already.
May 10th, 2022 at 1:46 am
#653
Not a whole lot of information but I wanted to add that during my restoration of some worlds, I've noticed QT files (.qt). Looking at them in a hex editor shows they are exactly the same as the MOV/CMP format, meaning we have a trio of extensions to work with.

What's interesting is that the QT and MOV extensions are used by Apples QuickTime for video purposes, but QuickTime also had picture functionality. Infact, some products at the time also had the functionality using Apples formats to have multi-frame images and really flexed the format, one example being HyperCard. It's possible a product from this time brought us this file format that ended up in the hands of Worlds, but it's difficult to begin to figure out what it was and where to find it. It's possible it was such an obscure product, or never released.

Just a theory though. The file extensions could be just a coincidence.
May 10th, 2022 at 10:09 am (edited)
#654
Not a whole lot of information but I wanted to add that during my restoration of some worlds, I've noticed QT files (.qt). Looking at them in a hex editor shows they are exactly the same as the MOV/CMP format, meaning we have a trio of extensions to work with.

What's interesting is that the QT and MOV extensions are used by Apples QuickTime for video purposes, but QuickTime also had picture functionality. Infact, some products at the time also had the functionality using Apples formats to have multi-frame images and really flexed the format, one example being HyperCard. It's possible a product from this time brought us this file format that ended up in the hands of Worlds, but it's difficult to begin to figure out what it was and where to find it. It's possible it was such an obscure product, or never released.

Just a theory though. The file extensions could be just a coincidence.
I'm sure it's just coincidental. It seems weird that an image format would stick so closely to video game engine standards rather than simply image format tradition. Things like color-replacement transparency, etc. It wouldn't hurt to try opening the QT in an older QuickTime version, but I'm going to go out on a limb here and say there's a 99.9% chance it will not work.

As far as we do know, CMP seems to take Microsoft's BMP format as a template and apply lossy compression to it as well as some other standard features common on 3D engines at the time (e.g. GoldSRC). It doesn't seem like something the QuickTime player would be compatible with.

I found a copy of Scorched Planet on some abandonware websites and could possibly take a look at it later, to see if RW2's format has any similarities to CMP. I don't have an old copy of QuickTime yet though. I'm typing this while in class so I'm juggling research and schoolwork
May 21st, 2022 at 4:19 am (edited)
#663
i was using sgeo's rwx exporter mod tonight and realized that upon exporting a world, it exports all the textures in 128x128.bmp but my world was comprised entirely of .cmp textures. it even breaks apart .mov textures and holograms into separate images. it's very late right now (and i don't know how to read rust), but he has to have figured something out in there, right?

i can't find the original github repo for it, but the source files are here on worlio: https://files.worlio.com/files/WorldsPlayer/mods/RWX-Exporter/src/
i'm guessing this is the relevant code clock in src/lib.rs:

if !texture.is_null() {
let raster = RwGetTextureRaster(texture);
let width = RwGetRasterWidth(raster) as usize;
let height = RwGetRasterHeight(raster) as usize;
let stride = RwGetRasterStride(raster) as usize;
let pixels = RwGetRasterPixels(raster);
let pixels_slice = slice::from_raw_parts(pixels, (height * stride) as usize);
let pixel_chunks = pixels_slice.chunks(2);
let mut img = bmp::Image::new(width as u32, height as u32);
for (index, pixel) in pixel_chunks.enumerate() {
let rgb565 = NativeEndian::read_u16(pixel);
let rgb888 = rgb565_to_rgb888(rgb565);
img.set_pixel((index % width) as u32, (index / width) as u32, px!(rgb888.0, rgb888.1, rgb888.2));
}
May 21st, 2022 at 5:57 am
#664
i was using sgeo's rwx exporter mod tonight and realized that upon exporting a world, it exports all the textures in 128x128.bmp but my world was comprised entirely of .cmp textures. it even breaks apart .mov textures and holograms into separate images. it's very late right now (and i don't know how to read rust), but he has to have figured something out in there, right?

i can't find the original github repo for it, but the source files are here on worlio: https://files.worlio.com/files/WorldsPlayer/mods/RWX-Exporter/src/
i'm guessing this is the relevant code clock in src/lib.rs:

if !texture.is_null() {
let raster = RwGetTextureRaster(texture);
let width = RwGetRasterWidth(raster) as usize;
let height = RwGetRasterHeight(raster) as usize;
let stride = RwGetRasterStride(raster) as usize;
let pixels = RwGetRasterPixels(raster);
let pixels_slice = slice::from_raw_parts(pixels, (height * stride) as usize);
let pixel_chunks = pixels_slice.chunks(2);
let mut img = bmp::Image::new(width as u32, height as u32);
for (index, pixel) in pixel_chunks.enumerate() {
let rgb565 = NativeEndian::read_u16(pixel);
let rgb888 = rgb565_to_rgb888(rgb565);
img.set_pixel((index % width) as u32, (index / width) as u32, px!(rgb888.0, rgb888.1, rgb888.2));
}
I don't believe it's a converter or anything. When you render an image on screen, you don't really render the image in it's original format. I believe this is just taking the raw bytes from textures loaded and exporting it to a familiar format. This isn't a format converter. If someone could prove me wrong, that'd be great.
Joined09/12/21
Posts4
May 23rd, 2022 at 4:14 pm
#667
The code does not do anything with .cmp/.mov directly.

It uses RWL21.DLL's RwGetTextureRaster to retrieve the texture data from memory, long after some Worlds code somewhere has read in and processed the .cmp/.mov texture into Renderware's format.
Jul 26th, 2022 at 5:47 am (edited)
#729
This is most likely a coincidence or a dead end, but I ran into CMP files yet again when dealing with a completely different program from the same time period and wanted to mention it incase CMP stuff was lying under our nose this whole time. I was getting into the wipEout games on the PS lately and in a Discord server I overheard somebody talking about getting custom textures into that game on the original PS1. It turns out the extention of the texture files are... CMP. As far as I'm aware no PS1 game uses renderware, but as Wirla had mentioned earlier its possible this isn't a renderware format. My goal is to get my hands on one of these files and compare them with the CMPs used in WorldsPlayer. Pretty good chance it won't work but I'd like to at least try it. Unfortunately I'm extremely sick right now due to a tonsilectomy so I probably won't be messing with any texture tools at this time.

Unfortunately I do still believe we may have to go the brute force hex-editor approach. It's common for files to have the same extension but be completely different formats. This just seemed like the closest thing to a 90s 3D texture that shared the same naming and it seemed odd so I'm sharing it here. All I can really say for sure is that whoever named these file types really had trouble picking more distinct ones, we already had like two matches with the QuickTime player despite actually being different file formats.

EDIT: dead end as suspected. Moving on.
Jul 27th, 2022 at 2:29 pm
#730
but as Wirla had mentioned earlier its possible this isn't a renderware format.
No, it isn't. It's an Accomplish format, a texture filetype for the engine Worlds Inc had made for their Worlds Chat. File extension doesn't matter, it's the header that identifies what it actually is. I know CMP has been used in other formats before.
Jul 28th, 2022 at 11:53 pm (edited)
#733
Yeah I know, but since nobody appears to be doing any reversing right now I figured I would try to find any connections to existing formats just incase of the slim chance I find something there. I'm not really good at this sort of thing.
Jul 29th, 2022 at 12:28 am
#734
Yeah I know, but since nobody appears to be doing any reversing right now I figured I would try to find any connections to existing formats just incase of the slim chance I find something there. I'm not really good at this sort of thing.
Look in Knowledge Adventure games. That's the only place you'll probably see it if it's anywhere else. It's an in-house format, given it's limited tools and only connections being from Knowledge Adventure. I'm doing reversal right now but I don't have anything to report.

Actually, I do have one thing. CompImg has two options. We Worldsers usually use -ow for our output, but -om also exists. These arguments are defined as Microsoft orientation and Macintosh orientation values respectively.
Next two bytes are either 0680, 0280, 4680, and according to Wirla with some very old CMP files use C680 and 2280.
That 80 only applied to the Microsoft orientation. Macintosh Orientation changed that to a 40, and CmpView will refuse to open Macintosh orientated files. What these orientation bytes do? Not a clue.
Dec 3rd, 2022 at 6:38 am
#1248
Did some messing around recently. Lot of weirdness.

Offset 0:013F, I am sure a palette of sorts starts, but I think it's written in a CMYK format? I am not sure, but I am able to change the colors of the palette using it, and it seems to match more with the values of CMYK than with RGB, HSV, or anything else I can think of. I also noticed it's not perfectly 255-color, or the conversion process is broken, and I'm not sure how to fix that. The max values it will hold for RGB is 252.

Next two bytes are either 0680, 0280, 4680, and according to Wirla with some very old CMP files use C680 and 2280.
That 80 only applied to the Microsoft orientation. Macintosh Orientation changed that to a 40, and CmpView will refuse to open Macintosh orientated files. What these orientation bytes do? Not a clue.
I also discovered that if I run CompImg with neither, the value will be 00
Dec 3rd, 2022 at 3:36 pm
#1249
Messing around and made some new discoveries.
So I did a little work on this a while back. I didn't get very far, probably because of a lack of experience with image formats, but this is what I managed to figure out. Please correct me if I'm wrong with anything, these notes are at least two years old, probably closer to four.

First 4 bytes are constant, like Wirla said. 4C 7A 48 32. Next two bytes are either 0680, 0280, 4680, and according to Wirla with some very old CMP files use C680 and 2280. I don't have notes on what this means.

Next two bytes are the number of vertical pixels. After that are the number of horizontal pixels.

Then what seems to be the loss factor, from 0 to 99. Again, two bytes. I have a question mark by this one.

Next two bytes are the number of colors.

Finally is the reduced color palette number. Again, two bytes. I really don't remember what this means. A note says 1C if pallette is standard, 2X if not. Not sure what X was.

Not so verbose:

σ - Signature (sigma)
φ - ??? (phi)
Pv - Vertical Pixels
Ph - Horizontal Pixels
κ - Loss Factor? (kappa)
C - Number of Colors
δ - Reduced Color Palette Number (delta)
http://kensmom.net/beam1.png[edit]I can't spell[/edit]
I believe the Color values are wrong. I think those 16-bytes you mentioned for the 'Number of Colors' is actually both that and the reduced color value.

This produced 0x8010, which doesn't quite add up to a 128 total number of colors at all, but if read as two 8-bit uints, it's 128 and 16.
compimg -pt -c128 -r16 -L -l0,0 -ow -Mt t1.bmp t2.bmp t3.bmp t4.bmp
This means those other 16-bits are a mystery again, and I've not yet been able to figure out what they could be.

The offset I mentioned previously seems to definitely be tied to a palette count. I realized all that empty space I've been seeing is empty spaces for the colors, or atleast I guess it's related. It changes depending on how many colors I have selected.
Guest posting not allowed
Please log in to post a reply.