<
https://www.techdirt.com/2026/08/20/the-eu-wanted-a-deepfake-detector-it-got-an-ai-scarlet-letter/>
"I spoke a little about this on last week’s
Ctrl-Alt-Speech, but now that
Anthropic has come out with more details about how its text watermarking works,
the debate has shifted into overdrive. Many people are upset about it, even
though it appears that Google has already been doing something similar with the
output from Gemini. I think that people are right to be upset, but for the
wrong reasons, aimed at the wrong target.
The real problem is that the EU’s
AI Act is aimed at a threat that never
really materialized, and the end result will fall hardest on the people who get
the most genuine benefit from these tools. Also, it doesn’t help that Anthropic
chose to comply with the law in a manner that looks broader than what the law
requires — though it did so for reasons that are more understandable than some
of its critics suggest. Either way, though, the costs will fall most heavily on
people who are using the tech properly.
Let’s take a few steps back first. If Anthropic’s explanation of how they
watermark text isn’t clear enough for you, I think this explanation of how text
watermarking works is much better. The fundamental thing to understand about
generative AI is that it’s always trying to generate the next token, and it
does so probabilistically, not deterministically, meaning that each time you’ll
get something slightly different. The results already have biases in them
(that’s part of the weights part of LLMs), but the companies can deliberately
bias them in a manner that the tool is ever so slightly more likely to choose
certain words based on a prompt than without that bias.
Think of it in the way that most random generators are not, in fact, “random.”
With a little bit of effort, people can often figure out the “bias” of a random
number generator, giving them an advantage in determining what number will be
generated. Here, it’s the same sort of thing, but the “bias” is impacting the
randomness of which word the tool will choose next.
With a long enough text, and a key regarding the bias, you can look at the text
and see that enough of the very slight changes match the “watermark” bias, as
to suggest the text was likely generated by a model carrying that watermark.
Such a system is hardly foolproof, but it can absolutely call out text likely
generated with that specific bias. Editing text after the fact may or may not
get rid of the watermark, depending on whether or not the edits remove/change
enough of the “biased” words.
Many of the people who are upset are so because they think they won’t be able
to cheat any more, and… I don’t care one bit about them. There is some concern
that the biasing will make you choose worse words, but I’m also not all that
concerned about that. As far as I’m concerned, if you’re using the AI to write
for you, in some cases you’re already having it choose words, and it should be
on you to know when and how to choose better words. The stronger version of
that objection is about how the watermark will apply when merely editing
content, in which case the watermark bias may nudge your prose towards specific
synonyms. But even there, that seems much more like an argument to use the
tools differently, not as a fully damning issue.
My larger concern is in how this will almost certainly be used to simply attack
and denigrate people who actually are using the technology in a reasonable
manner, but will be falsely accused of “cheating.” Just this week I wrote about
the ways in which I use AI tools to help with my work, not as a tool for
writing, but helping me review and edit stories. In that article, I mentioned
the REAL Rating site, which offers a one-to-five scale to make clear that not
all AI use is the same.
But a watermark reduces all of that to a simple binary: “did this use AI at
all.”
And I worry about the fallout from that."
Cheers,
*** Xanni ***
--
mailto:xanni@xanadu.net Andrew Pam
http://xanadu.com.au/ Chief Scientist, Xanadu
https://glasswings.com.au/ Partner, Glass Wings
https://sericyb.com.au/ Manager, Serious Cybernetics
