Use Gray Code (or other?) to store any variable bit-width value, without storing its width

Maybe my google-fu is just lame, but I recall 15 years ago reading an article that described how a certain compression algorithm assigned dictionary keys of fewer bits to the most oft-repeated or common longer redundant items it was compressing. As it ran of of room in narrower-bit values, it added bits to the lesser-used dictionary items.

It then replaced the items in the source with these dictionary keys, but, as gray codes (If memory serves me correctly), because, supposedly, when converting a grey-code encoded number bit-by-bit, you s开发者_开发技巧upposedly know when you have the whole number without having had to store somewhere how many bits you need to read.

Problem is, I don't see how this would work, moreover, all the documents gray code I see (e.g., wikipedia) emphasize its advantages when decoding digital positional sensors. I obviously don't need that that for my application.

Is this a different type of encoding I am thinking of, or, am I missing something really obvious?

My application is an trie-based index where the hits are serialized as 3-byte keys to a file table. A leaf could have thousands of hits, but often, since the indeces have from 10K to 100K files, this results in lots of wasted space.

I've thought of other hacks, but my memory keeps going back to this, which would be ideal. Can somebody post a link to an example, or drop some keywords for me? Or samples in .net/java/c*? thanks!

---

It could have been Arithmetic/Range coding (which are academically the same thing by most people).

7zip uses range coding after the LZ* pass; so you could just use the SDK, which is public domain (and includes C# code for the whole compression routine, not just a wrapper).