There are not many places where 95% compression of UTF-8 plain text could outweigh needing a 30GB model in memory and a significant fraction of current LLM inference cost to decompress it − and good luck convincing librarians to adopt it.

[–] Natanael@infosec.pub 2 points 2 days ago* (last edited 2 days ago)

There's a 300 MB library for audio compression using it. If you have large audio libraries it could eventually become worth the tradeoff.

https://huggingface.co/facebook/encodec_32khz

The image versions are probably more useful though.

The important part for these codes is that it has pre-LLM functions for quality metrics to judge if the output is close enough to indistinguishable (preserves detail, doesn't add any).

Although there is also variants deriving a neural net from the media to recreate it from the smaller model.