Watermarking AI Output: What Does That Even Mean?

Been wrapping my head around Anthropic's watermarking that indicates AI generation. None of the explanations I've seen depicted it accurately, so I wanted to share.

If you're not familiar, I'm referring to Anthropic's watermarking framework that attempts to make AI-generated text detectable. To understand it, we need to step back to how LLMs select words (tokens) in the first place.

When LLMs select the next word (token) in a response, a random seed will influence that selection where there are multiple options close in probability. So if there's a 37% chance the next word is "start" and a 40% chance it's "begin", you could end up with either. That random-seeded selection then feeds forward to the next word, which is why you can end up with such drastically different responses to the same question.

It's important to note that when the next token is a clear winner, this randomness doesn't come into play. It only matters in scenarios where there are multiple viable options for the next word. Aside from that, this is another one of those mechanisms, like weights not adjusting on the fly, that is inherent to how LLMs work. Different responses to the same question is a feature, not a bug.

With watermarking, those low-stakes selections are handled differently. Instead of using the random seed, the logic instead uses a key that's been salted with a numeric representation of the words leading up to it. If you have that key, you can then reverse engineer word selections to see that they are not truly random. One phrase detected doesn't cinch AI use, but a pattern across longer text is a strong indication.

Anthropic says there's no practical impact on readability, and there really shouldn't be: a cryptographic hash produces numbers statistically indistinguishable from random.

What does change is more subtle. The randomness stops being fresh. A true random draw is different every time; a keyed draw is the same every time for the same phrase. In practice, context is never quite identical, so you still get variety, but where the same phrasing recurs, so will the same word choice.

That shouldn't be a quality problem in any single response. Where I think it gets interesting is in agentic workflows, where the same planning files get re-read across sessions and agents pick up each other's vocabulary and develop a form of twin speak. But that's a different post.