<aside> ⚠️ Sorry this one isn’t formatted at all, it was written before I was using notion. This page is due to be reformatted / rewritten when I have time.
</aside>
This morning, I got in an argument (whats new) about texture formats, after somebody mentioned this format, KTX2, and how greatly it would benefit VRChat, etc. Which I (and others) did not believe in the slightest.
I thought that it was weird VRC would even load these textures at runtime, as KTX2 is supposed to be a format used for supercompression, very similar to crunch compression. The benefit of this format was to uncompress textures into different formats for different platforms, in a seriously compressed way, while remaining as lossless as possible. It would also be able to swap (compression) formats on the fly, i.e. swapping to BC7, ETC2, DXT1, etc, based on the platform and use case, along with parameters set at compression time.
One thing that is important to understand, is that KTX2 is a container format. This is similar to PNG or JPG, they aren’t the actual image data, rather the container that the data is stored in. The data format would usually be whatever the compression type used was, i.e. BC7, DXT1, BC4u, etc. The weird thing about KTX2 is that the container format has specific compression formats that it uses. I guess this is similar to PNG not supporting supercompressed formats.
What I learned was a few things, firstly this is really only slightly better than crunch compression, if even that. You basically crunch-compress the textures manually, although it wont uncompress at runtime, like crunch compression does, because VRChat does not support this format or have the runtime loader for it. Instead what happens is the image is converted to raw DXT1/BC7 data and then packed into the bundle.
It compresses using two formats, on two different levels. UATSC and ETC1s. These formats are similar to their predecessors of similar name, except they are specific to KTX2 as a container format. They both support KTX2 “Supercompression” and RDO (which is kinda like VBR but not?).
ETC1s is mostly responsible for the colour data of the image. When using it alone, Unity will decompress the image in the editor to DXT1 , making it very close to “Medium” compression. This means it has around 4BPP but in my testing it was closer to 3-3.5BPP. It’s similar to ETC1/2 which is a format for mobile devices.
UATSC is similar to the tegra/mobile-specific format, used only on mobile devices, except it has been adapted here to work on all platforms. This will be uncompressed as BC7 data, similar to “High” compression quality, except including that supercompression and RDO described before. It is supposed to be near-lossless but I could tell the difference personally. Nvidia describes this as “putting the image in a zip file, and then RDO moves and optimizes the contents of said zip file to make it even smaller”. I have seen a lot of conflicting stuff, but UATSC is supposed to hold the extra information needed to decompress into BC7 or “near-lossless” quality, while using ETC1s at the same time to hold the core data. I do not know how this works as I didnt really look much further into things that could not be used in my use cases.
It has a few benefits, mainly being disk space, and not having to uncompress on load, however it does look worse with sensitive colour data (i.e. skin) as the bitrate varies a bit, and is usually lower than standard for that format (i.e. 8BPP for BC7 or 4BPP for DXT4), along with being a more “GPU-friendly” format similar to DDS as a container.
It also happens to be pretty bad for maps, as edges and lines can get pixelated / blurry. This is a shame because a format like this would be great for masks and maps if it were able to support it properly.
The downsides are pretty killer though, it will not adjust the VRAM usage at all, as Unity has no way of properly adjusting this data to fit platforms, unless the correct runtime component was available. It also makes workflow pretty rough, as you’d be converting back and fourth constantly.
Essentially here you’d only be using half of the features of KTX2, which would be the container format the image is saved in. In my opinion, you would get much better results using crunch compression, or just compressing your image more than currently. We don’t get any runtime benefits, so the memory usage decreases, etc, aren’t applicable to us sadly. I may still experiment with it, however the only benefit I can find at the moment is disk usage.
Regardless, I just thought this stuff was cool, as I’ve been researching compression formats a bit more. I added an image showing my comparisons in sizes / container formats.