Technology

You know how these AIs output tokens? I'm going to explain the concept with words, because that makes it easier to understand. But it means that the explanation is quite right.

An AI has a vocabulary. The words in that vocabulary are assigned to 2 groups.

Sometimes when the AI is outputting something, it could use different words equally well. It's not quite the same as having synonyms, cause this isn't really about words. But let's say you have synonyms in different groups. At those points in the text, you can pick from one group or the other to embed a hidden pattern in the text.

Limitations are obvious. To embed the watermark, you need enough opportunities to pick "synonyms". It won't work for very short texts, or if the word choices are very constrained.

I'm curious if the negative effects are really as minor as they say.

• LLMs don’t predict just 1 word at a time. The predict a field of probabilities for the next possible word, eg if you feed a raw model “my favorite animal is a” then the probabilities of the next possible word is a huge array of animals:

(Sorry, afk so can’t post specific animal example image).

• This “distribution of possibilities” is already influenced by something called sampling to actually pick what word the LLM outputs pseudo randomly. It doesn’t just output the most likely word (unless the temperature setting is at zero).

So, what Claude is likely doing is biasing these word probabilities. Take our example. Let’s say the top 4 words are:

• “Cat”

• “Dog”

• “Horse”

• “Fox”

Claude can bias “Dog” and “Fox” to be slightly more likely than “Horse” and “Cat.”

It doesn’t seem like much. But do this for ALL of Claude’s vocabulary, for many thousands of possible words, and you make an embedded “fingerprint” for an LLM, where it slightly prefers a random half of the dictionary so subtly it doesn’t color output to humans, but is detectable if you know which half it prefers.

I believe one key with this is that the “dictionary” is a secret, otherwise bad actors could theoretically compensate for the bias.

This is just to catch the lowest common denominator, though, so I think Claude should publicize it.

If you’re still curious, I’d suggest visualizing sampling yourself with Mikupad: https://github.com/lmg-anon/mikupad

…You can’t actually use Claude with it though, as they hide the output probabilities of their models because they’re jerks who think LLMs should be black boxes their users don’t understand.

They were being pretty vague, but it could be something like "25 characters after every comma used, put a vowel. 25 characters after that vowel, put a space. 25 characters after that space, put a period." (Except much more complex than what I made up.) The point is to string together a text pattern that is so exact that it can't be just a coincidence.

The "watermark" is the text, so when you copy the text, you're also copying the watermark. It's also why they say it might not be conclusive if the text output is manually edited, or the output is too short.

Not for nothing, I think these types of endeavors are going to have unintended consequences. Despite them flat out saying that the lack of a watermark doesn't mean it wasn't AI generated, I think it's very plausible that the proliferation of watermarks will lend credibility to anything without a watermark.