Let's talk a little about what Claude recently started doing to watermark text.

It's based on the same mathematical foundations as Google's SynthId-Text ( https://www.nature.com/articles/s41586-024-08025-4 ) that Google published on github ( https://github.com/google-deepmind/synthid-text ). The difference is basically that they created their own proprietary implementation of it, but the core mathematical framework is the same.

Basically the way it works (and I'm going to butcher this explanation slightly for simplicity) is that let's say you are doing your word generation thing:

"The pencil is on the {next-token}"

It basically then creates a probability cloud covering what {next-token} is going to encompass. This is just standard to how these systems work.

This then perturbs that probability cloud _very slightly_ but in a predictable way that can be analyzed.

So think of it like this:

I generate a random number 1 to 6. I do this a bunch of times. To perturb the output I say:

* The probability of two consecutive numbers is increased by 5%
* The probability of the opposite face is decreased by 5%.

You can reformat this output, you can break it up differently or reformat it, and because it is probabilistic you can even change the output or alter some of the values and—with enough text—we can still detect the signal. "Humanizers" and such do not consistently protect from this kind of analysis, especially when it is done well.

This is an _extremely_ clever way of doing this. It doesn't affect the quality (for real world systems), but it allows for reliable detection if you have enough text.

It is strongest where you have a bunch of related tokens that are basically free choice (e.g., essays). It is weakest when you have highly structured output with only limited possible expressions (e.g., code, sentences that are already heavily structured, etc). It's not impossible here, but it does require more text to do the analysis.

Also because it is probabilistic and rooted in how token selection works it is basically immune from detecting, e.g., Jane Austin as being written by AI.