8/17/2026 at 9:19:41 AM
My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ...So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illicit means.
Any university using AI detection in their submission pipeline, or lawyers, editorialists, proofreaders that check for AI marks will be sending significant amounts of text like unpublished research, books, potentially internal documents and more, most of which is high quality human written, to dozens of AI companies, blindly trusting they won't train on any of that.
by ghrl
8/17/2026 at 11:10:34 AM
Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over the text and destroy the watermark. I predict an interesting cat and mouse game to develop.by ThePhysicist
8/17/2026 at 11:51:20 AM
The paper for it is open. The technique isn't really hiding information in the text itself, but by forcing some of the rolls to follow a specific pattern. LLMs work by estimating the most likely next token, so there's sometimes a list of possible candidates that would all work in the text (e.g. synonyms). At low "temperature", the output is a bit more deterministic and otherwise it's a weighted dice roll of which token/word to pick. An LLM can loop over existing text and figure out if the output matches something it would do, similar to checking chess moves against the best computed move for detecting cheating. But the LLM purposefully creates a pattern of alternating weighted rolls that are highly unlikely to appear in normal text, and that becomes the watermarking.The upside is that this has very low false positive detection rate, but the downsides are many. It only works on longer pieces of text. The system is fragile, and small edits (or rewrites by a local model) can fool the detection. Only the owner of the model is able to re-run inference at this level, so data must be sent to them for evaluation. And sometimes the token output is basically 100% deterministic because the input asks for the straight answer to a fact, or to recite a quote verbatim. That leaves no room for watermarking at all, unless the model is able to lie.
by unbelievr
8/17/2026 at 12:16:41 PM
A way to think of it is that any time the model faces a choice, it leaks some fractional bits of information. Instead of making the choice randomly, you can put information in those bits.But unless you know the prompt, you don't fully know which choice the model faces. Surely a lot of coding space is wasted compensating for that uncertainty. Exotic prompts ("Use no more than five E's in four consecutive words anywhere in the text") seem like they will confuse the hell out of attempts to extract the bits from the output text alone.
by vintermann
8/17/2026 at 1:04:28 PM
> But unless you know the prompt, you don't fully know which choice the model faces. Surely a lot of coding space is wasted compensating for that uncertainty.When you only need to encode one bit, the signal to noise ratio can be very low. If I try to write my own human words under the policy of "try a little bit to avoid the letter 'e' in every fifth word," then a sufficiently long text would still be 'watermarked' even if I only succeed in this dictum (e.g.) 10% more often than the baseline.
by Majromax
8/17/2026 at 1:14:51 PM
Whereas if you fully avoid the letter e, everyone will know you are George Perecby ttoinou
8/17/2026 at 2:54:56 PM
Who is Gorg Prc?by r2_pilot
8/17/2026 at 4:45:47 PM
And if you lowercase everything, you're e e cummings.by cratermoon
8/17/2026 at 5:10:35 PM
most people have at least two, but generally around four to six (or more, thanks internet) interactional styles (not "selves", just things which vary depending on context, subject, and people they are interacting with). probably more. some of it is due to simple physical comfort levels (right now I am in a physical position where capitalizing is more difficult); some might be due to talking to a peer group instead of a group of kids or a priest or boss at a job, etc. unsure how that will shake out with AI but it unnerves me.by jambalaya8
8/17/2026 at 6:03:40 PM
There's more to it than encoding one bit. You also want to avoid false positives. You can encode a single bit by XORing all the bits in the UTF-8 encoding. But then you get a lot of incorrect hits. The lower your tolerance for false positives, the more it acts like you're actually requiring more bits in your payload.by recursive
8/17/2026 at 5:55:50 PM
The scheme that Scott Aaronson describes essentially uses a specific prng, and you can then check a certain function with relatively few tokens to get a sense of whether or not a model using that scheme generated the textA few caveats: you need to know the key to the function (used when generating the text) and you need to know the bias it would introduce
The point is that you do not need to know the full prefix, just a modest sample set of contiguous tokens
by wrsh07
8/17/2026 at 1:36:50 PM
In fact such ‘arbitrary’ constraints uniformly improve composition. Thus eg if I force a - largely arbitrary - technical glossary to be unrelentingly applied to a translation, every single sentence improves in quality.by applicative
8/17/2026 at 4:23:09 PM
Oulipo vindicated.by katatue
8/17/2026 at 3:33:41 PM
> The system is fragile, and small edits (or rewrites by a local model) can fool the detection.Well no, small edits wouldn’t fool the detection as long as the seeding only uses a small run of previous tokens.
And yeah full rewrites breaking it is by design. The watermark is just meant to tell you whether the text was generated by a watermarked model, not whether the ideas came from AI or something like that.
by demibabs
8/17/2026 at 8:03:48 PM
Does that also mean owner must re-run very model ever released by them to detect anything?by out_of_protocol
8/17/2026 at 12:10:52 PM
In practice, it is theater. Are they going to do this with the code output too? This is nonsense security theater for the low thinkers to have a sense that someone is in charge. When we all know nobody is in charge, anywhere.by bsenftner
8/17/2026 at 12:24:03 PM
Their AI model tends to write a lot of lengthy comment blocks.That's a fine place to put the watermark to track those users who accept the code blindly and don't delete/edit the comments.
by pavlov
8/17/2026 at 2:26:41 PM
The model doesn't particularly "know" which tokens within its output are "comments", it's not so easyby kfse
8/17/2026 at 6:05:20 PM
It knows when several tokens are about equal vs. times where one token is vastly preferred. In the latter case, that’s usually code or math or something similar and so it won’t alter those tokens.by alasdair_
8/17/2026 at 1:24:53 PM
EU regulations are going to force it, friend. The tsunami is coming and cannot be stopped and Anthropic has jack to do with it.by applicative
8/17/2026 at 2:41:12 PM
I wonder if that is entirely true. It is also in the AI companies' own best interest to be able to detect AI generated text so that they can avoid training on it ("Habsburg AI").In the case of Anthropic it would also be entirely unsurprising if they've been lobbying the government to force everyone to do something in their (Anthropic's) own best interest.
by HarHarVeryFunny
8/17/2026 at 3:15:47 PM
> It is also in the AI companies' own best interest to be able to detect AI generated text so that they can avoid training on itIt’s also in their interest to demonstrate that they can be trusted and to show that they at least pay lip service to limit the obvious downsides of the tools they are selling. The use cases they sell to mainstream audiences are not affected by detection tools. The point of having a LLM do the work for you is that the work is done, and reliably. It does not matter if it is done by a LLM, and most of the time it is obvious anyway.
by kergonath
8/17/2026 at 6:13:34 PM
The EU mandate for AI watermarking is likely the first step in the direction of prohibiting AI for specific use cases. The pretext for outlawing (or at the very least controlling) the use of AI when the time comes will be something along the lines of data integrity or just general compliance legalese.So, the use cases that are being sold to mainstream audiences actually will be affected by detection tools, especially if the output is intended to be monetized in some way. In the near future the EU will likely come down with heavy intervention to prevent AI from impacting employment rates across Europe. The number of legitimate use cases for costly frontier models drops significantly once eliminating professional jobs is off the table. This is all conjecture at this point though.
by 2_foos_in_a_bar
8/17/2026 at 7:14:19 PM
I’m sorry, this is going to be a bit long but you made good points.> The EU mandate for AI watermarking is likely the first step in the direction of prohibiting AI for specific use cases.
I am not sure how practical that would be. The cat’s already out of the bag and they won’t prevent companies in the whole world from releasing open weight models. Playing catch up by distilling flagship models is also relatively cheap; we’d see smaller companies setting up shop in friendly regimes. And I don’t see any appetite to go full child porn and criminalise the possession of a LLM. So we’d end up with a similar situation as with illegal downloads, i.e., everyone will do it and nobody will care.
> The pretext for outlawing (or at the very least controlling) the use of AI when the time comes will be something along the lines of data integrity or just general compliance legalese.
They could forbid using LLM for hacking, but hacking is already illegal. They could make it a factor when determining punishment, but I don’t think that would work terribly well. Most of the dangerous stuff we can do with LLMs is already illegal, or should become so. Things like propaganda, identity theft, harassment, scams. We need enforcement with teeth on these, not pointless feel-good legislation. Again, there are parallels with cryptocurrencies and torrenting software. These things have illegal uses, but it’s also really difficult to make them illegal, at least in semi-functioning democracies.
> So, the use cases that are being sold to mainstream audiences actually will be affected by detection tools, especially if the output is intended to be monetized in some way.
I don’t know that mainstream audiences are really against LLMs. They are mostly against AI in a nebulous sense, but even non-technical people use ChatGPT or equivalent. I think that the critical mass is already there and the tools are convenient enough that they couldn’t outlaw them without a massive uproar.
Detection tools don’t seem all that relevant to mainstream audiences’ use of LLMs. AI companies will sell this as a safeguard against misuse, and everyone will be happy about it. The politicians will say they accomplished something, the AI companies will slowly turn public opinion, and the public will have shiny toys.
> The number of legitimate use cases for costly frontier models drops significantly once eliminating professional jobs is off the table.
I don’t know. They can open possibilities that we don’t necessarily consider.
One example I have is a friend who is getting his house refurbished. He’s not an engineer or a material scientist. He does not have enough free time to read thoroughly on the many subjects involved. With a decent LLM, he could untangle the technical documents sent by the architect and the contractors to really understand what was going on and be involved, rather than passively follow the architect’s advice. For starters, the LLM was very useful in finding issues in the quotes he received when he was looking for an architect. Those were long, technical documents, with no really standardised structure and full of jargon. I don’t think that person is going to want to stop using LLMs now. Many people are having this sort of moments right now.
> In the near future the EU will likely come down with heavy intervention to prevent AI from impacting employment rates across Europe.
Maybe. But i don’t believe the EU is well equipped for that. Labour laws are largely local and different in each member state. The EU regulations are basically the common denominator, ore or less, and it is easy to see why: for regulations to get adopted, they need a strong enough majority in the Commission, in the Parliament, and in the Council. It is very difficult to get anything controversial that affect the sovereignty of member states passed.
The angle of the current AI regulations is that they set the rules for the single market, which is where the EU is the most legitimate. It is difficult to see market angle for the effect of AI in labour, and I think enough member states would be keen to kill the project.
Also, there are many influences at play, but the EU is fundamentally an economically liberal institution. It very rarely goes in the direction that reduces economic activity. Look at how clumsy it is at fighting against cheap Chinese imports. I don’t think the institutions themselves would really want to make AI illegal. Companies have too much to lose.
by kergonath
8/18/2026 at 6:00:49 PM
Great points all around, lots to think about. It's totally possible that I am way off in my assessment of the future.I think your comment about torrenting touches on an interestingly relevant case study, specifically media piracy. The seed of that technology was sown when the internet was still just clusters of machines passing files around and then exploded with the PC, Napster, and TPB. Napster tried to be a legitimate commercial enterprise with a business model of undermining the ability of copyright holders to rent-seek on the consumption of the material they 'owned'. At the core it was a novel technology (P2P) that revealed an economic arrangement to be out of date (if a distributor no longer has to manufacture a copy of the media for each individual consumer, their business model boils down to rent-seeking). Western legal systems were quick to rule on the matter (in favor of copyright holders), and I have no doubt that they were looking ahead to a future wherein media creation was totally disincentivized by said novel technology.
Now a novel technology (the cloud inference-backed LLM) is challenging another economic arrangement. This time around, the arrangement being challenged is the higher education->professional job pipeline. All advertising and media messaging aside, it really does seem like the frontier labs are only economically viable if they get massive enterprise deals across a broad spectrum of industry. There is fundamentally one chunk of capital organizations are going to spend either supporting their talent pipeline or padding it (to put it gently) with enterprise LLM deals. If the latter path is taken too far, consumer spending plummets (due to lack of middle-class incomes), assets backed by consumer debt/spending fail longterm, and we will have to deal with a deluge of socio-political issues stemming from the absence of real social mobility (we are in the early stages of this now, incidentally).
All this to say, I think parallel situations from the past can guide our thinking re: AI regulation and the forces shaping it. Reasoning based on regular political/economic incentive structures (e.g. economic liberalism not wanting to reduce economic activity) will fly out the window at lightspeed once fear becomes a factor. I personally think its great that LLMs empower individuals such as your friend to increase the control they have over real issues in their life; that is what technology should be doing for us. Cloud inference-backed LLMs are doing the opposite: drastically reducing the power that the everyman has over his socio-economic future by throwing high-paying career paths for a whirl and incentivizing powerful organizations to destabilize the labor market. Only time will tell how this plays out.
by 2_foos_in_a_bar
8/17/2026 at 6:03:19 PM
They say in the paper that they will not do this for code but it may happen for comments.by alasdair_
8/17/2026 at 4:53:33 PM
What happens if we train models (GPT or human students) using the outputs of a model with text havingbthose watermarks? Is there something preventing the watermark from being learnable?by estebarb
8/17/2026 at 6:52:40 PM
no, and in fact they should be extremely easy to detect if they are substituting a token that is a least-probable synonymby ludicrousdispla
8/17/2026 at 5:16:28 PM
[dead]by slickytail
8/17/2026 at 4:44:54 PM
I realize that short attention spans are pervasive now, but the link to the explanation is only eight paragraphs in https://declaude.org/watermarking/by cratermoon
8/18/2026 at 6:20:45 PM
In addition to holding the key, wouldn't you additionally need to know exactly which model to check against? So for passive detection to happen, I think each company would need to check every message against every model version? Also would need to spend resources re-invoking the each model version against each message.by notfed
8/17/2026 at 8:11:14 PM
Yeah kind of surprised that people didn’t have enough patience for Gruber’s writing.It’s quite fun and engaging.
by ozozozd
8/17/2026 at 11:46:19 AM
Probably they bias the RNG for selecting the next token. This can be done practically in a lot of ways, including during training.I suspect the signal will be significantly under the noise floor, so it's not detectable if you don't know exactly what to look for, but certainly you can submit more information then the textual contents.
by ddalex
8/17/2026 at 12:21:35 PM
But if the user's prompt is in the context, you don't know exactly what the RNG chooses between. I don't know what trick they use to get past that, but it seems impossible to get by it in the general case (i.e. if the prompt can be anything) and you'll probably quickly compromise quality if you try.by vintermann
8/17/2026 at 1:11:59 PM
A reasonable guess about the algorithm is 'A Watermark for Large Language Models' (https://arxiv.org/abs/2301.10226). The idea is that each generated token (or bigram) seeds a strong PRNG that splits the vocabulary into a 'green' and 'red' set. The sampler then tries to select a 'green' next-token for generation.After-the-fact checking only needs the vocabulary splitter, which is independent of the LLM. Over a sufficiently large text non-watermarked text would expect to use green and red tokens with the baseline probability, and that difference can easily become statistically significant over sufficiently long texts.
The basic algorithm has obvious knobs to tune, among them the initial ratio of red to green tokens and how hard the sampler tries to pick a green token. These would balance fidelity to the original distribution against watermark detectability (minimum required content length for statistical power).
by Majromax
8/17/2026 at 4:09:58 PM
Anthropic actually tells you the approach they use, and it's not that. From their Claude Text Watermark page[0]: "Claude’s text watermark is a version of the SynthID-Text approach published by Google DeepMind in a Nature paper in 2024."The Nature paper is "Scalable watermarking for identifying large language model outputs"[1]. This method does not separate out tokens into separate classes, but merely uses a seed for the PRNG that selects which among the most likely tokens generated by the LLM will actually be output. This has the advantage that there's no green and red token sets, so no token is systematically favored or disfavored. If a particular token is overwhelmingly predicted to be the most likely candidate, it will almost certainly be selected, so the watermark doesn't affect that. Even if there are several choices of output token at a point that have similar probability of selection, the watermark doesn't systematically bias in favor of one token or the other.
This is actually a quite elegant method of watermarking that, contrary to people's fears, won't adversely affect the model output. The main concern I have with it is that it appears that you can't actually test the watermark locally, without uploading it to Anthropic. I'm not sure why that's the case, since there's no particular reason the watermarking key has to be private, except if you want to prevent others from generating text with their own LLMs that is watermarked to look like it's generated by Anthropic - but everybody wants their text to not have the watermark.
[0]: https://www.anthropic.com/news/claude-text-watermark#:~:text...
by tmp10423288442
8/17/2026 at 1:04:39 PM
Simple version: In instances wherein the otherwise statistically chosen next word is a "toss-up", watermarking removes the randomness by imposing specific choices, determined by a key. This then becomes a detectable pattern when scanned with the key (stastically—detection itself is probabilistic).>use it to store arbitrary information
No additional data is embedded. The range of available data is constrained by the text being generated (i.e. the sets of "next words" per text).
by unclebucknasty
8/17/2026 at 1:56:06 PM
From what I’ve read, they won’t be imposing specific choices, but using a different (biased) RNG for those “toss-up” choices. With enough sampling, you could detect if the RNG was biased or not.by mbreese
8/18/2026 at 12:21:39 AM
This is what I meant by "imposing specific choices, determined by a key". Maybe "impose" or "specific" were too strong in my attempt to simplify?I attempted to clarify that the impositions themselves are not deterministic, by indicating that the entire process is still probabilistic.
Maybe Anthropic's explanation is simple enough [0]:
>When watermarking is used, choices are still made at random, but the source of the randomness is different. Instead of using an arbitrary random number generator to pick the next word, watermarking uses the key and a few words that come before to settle what word the model should pick. That is, the words that Claude picks are still random, but now, one can check the sequence of words and see if it’s consistent with the choices Claude would make if it was using the key. If it is, one can assign a probability that the text was generated by Claude.
by unclebucknasty
8/19/2026 at 11:32:49 AM
If it stores unique information, by definition it can store arbitrary information because it can point to arbitrary information. So they can have it relate to anything they want. Even a full breakdown of the original text if they choose.by mickdarling
8/17/2026 at 5:53:11 PM
Removing the watermark usefulness depends on your use case.If you care to avoid detection, yes, it is useful. If you care about the best possible sequence of words, then the damage is already done once watermarked.
by cocoa19
8/18/2026 at 5:55:11 PM
[dead]by jeansilga
8/17/2026 at 12:20:35 PM
I'm sure they could use it to fingerprint people, at the very least.by nullbio
8/17/2026 at 6:49:40 PM
It’s possible to ask each model and build it.by j45
8/17/2026 at 12:16:44 PM
Doesn’t this mean Anthropic can accuse anyone of using their AI to write for them?by andy_ppp
8/17/2026 at 12:40:59 PM
Your favorite anti-AI political candidate turns out to have not written their thesis, with a 73% confidence level.by mapt
8/17/2026 at 1:59:25 PM
Is watermarking really watermarking if it can’t be independently verified?I mean, all of these text content watermarking schemes require the company to assess if the text was AI generated or not. They aren’t going to tell us where the toss-up tokens are or what is in the red vs green pools of words.
by mbreese
8/17/2026 at 3:54:36 PM
Seems like exactly the sort of problem the threat of defamation lawsuits are meant to solve.by jjk166
8/17/2026 at 4:02:36 PM
Evidence is a massive problem here. As well as the extremely high threshold for US defamation; political candidates routinely tell the most absurd lies about each other.by pjc50
8/17/2026 at 4:21:51 PM
The high threshold is only for public figures, which must show actual malice. For everyone else (e.g. your typical student falsely accused of plagiarism), they only need to show the statement was false, published, targeted the plaintiff, and caused damage - it's still defamation even if they were simply negligent in checking the facts. The company might be able to cover themselves if they are very clear they are just giving a statistical probability and not saying their model was definitely used, but if they imply they have hidden information that points to their conclusion (which is kind of the whole point of their algorithm) then that's not a valid defense. I can't say they would lose in practice, but in theory they ought to lose.by jjk166
8/18/2026 at 7:36:02 AM
In places like Germany it's a crime to say something that makes a politician look bad, even if it's true.by inigyou
8/18/2026 at 1:54:51 PM
That is simply incorrect. In Germany, truth is a complete defense as far as libel and defamation cases are concerned.by sReinwald
8/18/2026 at 11:03:09 PM
But not insult, and even for defamation, you will be prosecuted and have to prove the statement is true, even if the politician and everyone else knows it's true.by inigyou
8/17/2026 at 2:50:59 PM
yes, why would someone will use this tool for writing then?by SatishPophale
8/17/2026 at 2:21:07 PM
[dead]by marianobayu
8/17/2026 at 1:44:03 PM
[dead]by MagicMoonlight
8/17/2026 at 9:47:19 AM
I think this is a very real concern. But I’m not sure of any way around it. Any stenographic system that you have the code for can be trivially defeated.I wonder if this would be a good use for homeomorphic encryption. There might be a way to let anthropic check some text without actually giving them access to the source text. Any experts around? We could use your skills!
by josephg
8/17/2026 at 10:15:36 AM
> Any stenographic system that you have the code for can be trivially defeated.They're giving you an oracle regardless, which is almost as good. Take LLM output, make some modification, ask the detector if it's LLM output, repeat until you learn what kind of changes you have to make to defeat it.
Or don't even bother learning what to do, just make arbitrary changes until it says it's not, so when the person they're submitting to does the same check it says the same thing.
by zrm
8/17/2026 at 11:43:12 AM
I assume this oracle will be behind 20 layers of anti-bot protection, CAPTCHAs and hardware attestation challenged. It will be incredibly painful to use. It won't stop the motivated attackers, but will make it too annoying for the average person.by kevincox
8/17/2026 at 6:34:43 PM
When all else fails, you can hire a lot of folks cheaply to effectively Mechanical Turk it with their home internet connections.by teeray
8/17/2026 at 11:19:20 AM
> it says the same thingReference needed? I think it remains to be proven whether those detectors can be considered deterministic.
by sebastiennight
8/19/2026 at 3:17:06 AM
Good lord, just write the thing.by nativeit
8/17/2026 at 11:48:31 AM
They could (..and probably will..) store that version and then refuse the check if this attack is detected, i.e. the version is too close to a known LLM output.Alternatively they could also just keep saying "yes" if it's close enough to a version that was close enough.. Although that would enable the attack to allow arbitrary text to be "proven" AI, by slowly morphing close-enough generated material to the desired text. But perhaps this is not a problem they are not concerned with.
To satisfy the letter of the law I expect it's enough to just provide the oracle, without any mitigations.
by _flux
8/17/2026 at 9:49:03 AM
Won't we just be able to fine tune OSS models to detect these patterns across providers? It will be cat-and-mouse but my bet is it converges to a central detector that isn't affiliated with any model provider.by piker
8/17/2026 at 12:05:35 PM
> Won't we just be able to fine tune OSS models to detect these patterns across providers?A good fingerprint should make use of cryptographic signatures. Without knowing the keys, the fingerprint should be indistinguishable from noise (or just random token selection)
by josephg
8/19/2026 at 4:16:42 PM
Wouldn’t those hashes be trivially defeated by tweaking the language?by piker
8/17/2026 at 4:16:55 PM
>Any university using AI detection in their submission pipeline, or lawyers, editorialists, proofreaders that check for AI marks will be sending significant amounts of text like unpublished research, books, potentially internal documents and more, most of which is high quality human written, to dozens of AI companies, blindly trusting they won't train on any of that.Uh, yeah, why do you think it's setup this way? The frontier companies desperately desire more high quality human text and this is how they are planning to get it for free.
by atroon
8/17/2026 at 5:23:07 PM
And I'm sure students will use tools to have every other paragraph written in the style of a different AI, in an attempt to defeat this fingerprinting.by apparent
8/18/2026 at 3:18:30 PM
I saw that someone already created a Github repo for a Python script that strips the watermark out of Claude generated text. It was released, I think, within 24 hours of the announcement. I cannot attest to how well it works, but I found it humorous nevertheless.by hirvi74
8/17/2026 at 5:36:33 PM
Yeah, wait for LLM "scrambles" that put every paragraph and then the whole text through multiple re-write/edit style cycles.by bluecalm
8/17/2026 at 7:13:27 PM
How would that change anything? The proposed watermark is applied while the output tokens are being chosen, taking that text and running it through an LLM again would just repeat the process.by solid_fuel
8/17/2026 at 8:24:35 PM
It says it only applies to passages over 200 words, so you could use a different LLM to rephrase every other paragraph and undermine the watermarking.by apparent
8/18/2026 at 7:07:23 AM
You take the output and run it through another LLM with "please re-write this in xxx style". Then you repeat that a few times on different part of text and glue it all together at the end.by bluecalm
8/17/2026 at 12:00:14 PM
That's why detectors like Pangram exist too I think.by ShinyLeftPad
8/17/2026 at 2:27:31 PM
Pangram doesn't work.by saltwatercowboy
8/17/2026 at 5:04:50 PM
Like their fp rate is a lie? It works well in my limited testing.by adamgordonbell
8/17/2026 at 10:20:59 PM
It's idiosyncratic to the point of uselessness in mine.by saltwatercowboy
8/18/2026 at 2:58:23 AM
what do you mean by "work"? I think it works perfectly as UGC honeypot.by ShinyLeftPad
8/17/2026 at 6:06:13 PM
[dead]by redsocksfan45
8/17/2026 at 4:34:07 PM
Sounds like a business opportunity.by stillpointlab
8/17/2026 at 7:26:14 PM
You've discovered the perpetual mutually assured destruction money generator — guess what the best defense against LLM spam also is?LLMs cause a wide number of problems, the good news for our investors is that they're all solvable with LLMs.
by micromacrofoot
8/17/2026 at 7:17:48 PM
Exactly, so even if you are avoiding AI, any interaction with society is now being structured so you have to submit to the digital surveillance equivalent of a cavity-search machine. What a dystopia awaits the budding generations.by rolandog
8/17/2026 at 1:33:35 PM
Anthropic had absolutely nothing to do with this. The Chinese models will soon be adopting such devices as well. It is an overwhelming force coming inter alia from educators worldwide.by applicative
8/17/2026 at 1:24:05 PM
I expect them to work as reliably as AI text generators do now...by PunchyHamster
8/17/2026 at 3:34:00 PM
> checking any text for watermarks requires sending the entire text to AnthropicCouldn’t it be checked in the TEE using confidential computing to keep Anthropic’s algorithm secret?
by Alex3917
8/17/2026 at 5:20:18 PM
I can't speak for Anthropic, but with Google's SynthID, the algorithm is actually public. However, checking (or creating) the watermark requires a symmetric key, and the providers likely wouldn't share that key.by ghrl
8/17/2026 at 6:49:06 PM
While Anthropic is sharing this publicly, there’s isn’t much reason that other models could quietly be doing this or start.Local models could probably catch some patterns.
by j45
8/17/2026 at 2:30:57 PM
That's just storing what they output in a database and then checking, not a watermark.by LtWorf
8/17/2026 at 2:45:27 PM
> And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, MistraLI would think that these things would eventually converge and we’d get one watermarking algorithm as an industry standard. That way, all major provider would follow it and we’d get independent software for checking. This would partly limit the efficacy of the watermarks, but on the other hand if it’s done correctly, removing the mark could still be enough of a pain that casual users would not bother. That would obviously depend on a lot of factors. It would at least add significant friction in the production of daily slop.
Of course it wouldn’t do much for thing like foreign propaganda but that’s a whole other discussion we need to be having.
> Any university using AI detection in their submission pipeline, or lawyers, editorialists, proofreaders that check for AI marks will be sending significant amounts of text like unpublished research, books, potentially internal documents and more, most of which is high quality human written, to dozens of AI companies, blindly trusting they won't train on any of that.
Isn’t it already what they are doing right now with some of the plagiarism detection tools? Not every university is going to have a representative corpus, and yet they are all using the software. So I guess the provider is doing the work of feeding all that data to their algorithm.
by kergonath
8/17/2026 at 7:44:23 PM
You can’t check it with only the algorithm, you need the secret seed key. Which will be different for each provider (and they’ll probably have and use multiple). And you need the llm itself, to generate the potential tokens at each step.by ComputerGuru
8/17/2026 at 1:23:11 PM
I’m worried that until such tech is perfected, pervasive and uniform across all models, education will be dead, as it certainly is at the moment. Flat out dead.I read stacks of term papers all year and it is a reality that, apart from such schemes, we are in an extinction event for civilization.
by applicative
8/17/2026 at 4:15:45 PM
Education is in the best place ever for people who actually want to learn, they can have 24/7 access to a tutor with a wide breadth of knowledge and infinite patience for stupid questions. Education is in a terrible place for people who just want to get a degree and don't care about actual learning, but such people generally don't contribute much anyway, and are the easiest to replace with AI, so no big loss.by logicchains
8/17/2026 at 11:24:00 AM
its lovely training data. no detection? add to training set -_-.its also kind of laughable that somehow people are trying to prevent the outputs not to be altered. Asif you cannot manually paraphrase anything you can read. So the only solution would be, to make it utterly unreadable (which is not possible, it obviously defeats the purpose of the thing).
Not to mention local models ofcourse :-)
by saidnooneever
8/17/2026 at 12:37:33 PM
I think what we’re testing for here is LPM output that hasn’t even been skimmed by a human, let alone paraphrased.by taneq
8/17/2026 at 10:40:24 AM
Why not switch it around? Solves problems of privacy (no data upload), is far more doable and it's far more important to be able to verify content has not been tampered with after originating from a human or capture device (like a camera). We also have decades of cryptographic experience in reliably signing text, images, video, etc. and it doesn't break apart because a text is too short.Having proof that content (especially images and video evidence) is unmodified (whether via Photoshop, Paint or a model) is far more valuable then having evidence that an image was manipulated or generated fully by a model (which still leaves other forms of manipulation), I feel the same goes for human authored vs generated text. Free to admit that using models to generate any kind of media whole-cloth is still unappealing to me and I still pay for commissioned artwork or make it with my limited abilities for what that's worth. Do like to (poorly) write my musings too and see UX as something were thoughtful contributors (like the opinionated, sometimes controversial, but certainly talented GNOME Gitlab contributors) can make a major impact.
Code can be beautiful, interesting and serve purpose beyond execution, of course, but for most people, in most cases, it does not in the same way as audiovisual content (not limited to art). Having code just to execute and resolve a problem can have value all in itself, the code being a means to an end whose quality, let us be honest, was barely a concern in most corporations long before LLMs.
Also have rarely (honestly never) before LLMs fully owned all parts of any code base, always relied in part on someone's prior effort in (Flutter/Dart mostly) packages, whereas when writing, drawing, etc. I have far more situations where I make something from scratch and everything there is only there because of my conscious decision. Even simple marketing mockups that, quality wise, any modern model would beat feel different when I was fully in control, where to place what, etc. Objectively worse (at my skill level), probably, but still never the same.
Knowing something was made from scratch by a human has value to me, beyond misinformation prevention. Knowing for a fact that LLMs were used instead of importing a library, using a template, or something similar that leads to expending similar amounts of effort, I don't see that being nearly as valuable. Heck, with all the importing and my experience back then vs now, I am spending more effort actually fully reading any LLM output in my code then I spent back then auditing Flutter/Dart packages. Then again, LLM output fails far more unpredictable then those messy packages that simply got Gradle to take down my system...
Happy to admit, I have been skeptical of watermarking LLM output being feasible for quite some time and having looked into SynthID Text and proposals being researched, I am convinced that it is challenging to impossible beyond the lowest common denominator and less important then proofing human authorship.
It will catch people just copying LLM output into their replies without thought, which is not a negative in my book, especially if it is not discernibly affecting output quality in regular use cases. Anyone who wouldn't copy Wikipedia into their dissertation will, in my opinion, be able to bypass text watermarking as proposed however, I feel we need to be honest there.
Thing is, if that's the case and text watermarking will only ever catch LLM created slop, is that a bigger problem then the misinformation, harm to creators due to authorship questions and accusations, making it harder to use evidence in proceedings, teachers not trusting students even when they did the work themselves, etc.? Signatures for all such cases will be difficult to implement, yes, but I feel are going to be of greater value in the not to distant future and I equally feel are not impossible, not least because idiots will always want to hide their LLM usage, whereas human authorship is something they take pride in and want to proof.
by Topfi
8/17/2026 at 11:52:03 AM
> Why not switch it around?Because a malicious human will gladly copy/paste LLM text and sign it with his "I, a human, definitely wrote this academic paper" key?
by piaste
8/17/2026 at 12:14:17 PM
Fair point, for text it is far harder to prevent signatures being applied to generated text vs images at the moment of capture and most approaches I can come up with to remedy this can either be bypassed (edit histories can be output by models similar to humans) or will be controversial. Taking a page out of the anti-cheat textbook, mainly written for gaming, there are methods which might hold in the medium term. Less a fan of kernel level myself, though it might be worth exploring as there has been massive investment by the games industry into making it somewhat robust, but the approach Valve has taken with VACnet could be an inspiration worth exploring that is less invasive into peoples systems. Keystroke analysis, etc. could be relied upon as a basis for signatures, harder to spoof for current day LLMs over generating edit histories.I will fully admit that at a point in the future, maybe not too soon, models may be trained to bypass that too, at which point we are back where we started. As a skeptic of the extend that capabilities are emergent in LLMs vs specific to training data, I am somewhat hopeful that unless models are specifically trained for evading such human detection solutions, they'd struggle to do so, but it could still end up as a byproduct of improved, lower latency computer use focused training. Not emergent as the term is used in regard to models because that is still output performance improvements clearly traceable to very specific training data, but incidental as the goal of said training data was not to bypass.
For what it's worth, I find human authorship being verifiable to simply be the more crucial problem over watermarking model output, so if research is to focus on one, I'd rather it the former. Maybe both signing human authored content and watermarking LLM output are both only possible in the near term, I hope not but fear it that might be the case. If so, we as a society will have some major challenges ahead (beyond all the ones we'd have anyways).
Alternatively, we could also just start scanning everyones eyeballs...
by Topfi
8/17/2026 at 12:37:56 PM
I've been thinking for a while that all of this is just trying to grasp tighter the last bits of sand escaping between our fingers. The end game, perhaps, is trust. Do you trust or know the source? If you don't, assume it was AI generated. If you do, accept it as authentic based on whatever they disclose, but know that it's possible they aren't being totally honest or were themselves fooled in some way, depending on the context.Then build our assumptions and how we operate around those trust levels in the digital realm.
by mrngld
8/17/2026 at 6:21:03 PM
Well, the end game is that AI is better at thinking us, and having a human brain involved is a net negative. The goal is to let the AI do the toil if thinking for us, and we can get the rewards.by a2ff6eeb0
8/17/2026 at 7:57:41 PM
[flagged]by chancedurham
8/17/2026 at 12:00:13 PM
Thank you for stating clearly situation. I fully agree with your assessment. For almost 10 years now I have been saying that we need to virtually watermark reality. By "virtual" I mean store the metadata about the digital capture on a public blockchain. Then my devices could have a built-in "fake vs real" detector.Artists, photographers, journalist, etc. are going to want and need this.
by intrasight
8/17/2026 at 3:53:43 PM
You'll be happy to learn about C2PA https://c2pa.org/by warkdarrior
8/17/2026 at 12:30:52 PM
Some sort of signatures to verify legitimate unmodified photographs (taken with a relatively secure device, e.g an iPhone) seems like a reasonable ideaBut even then, people will be able to point that camera at a manipulated/generated image (either printed or on a screen). Maybe that one could be solved if the photo included some depth information?
by bluescrn
8/18/2026 at 11:43:31 AM
I doubt that NYT, WaPo, or any other mainstream journalist would do that.by intrasight
8/17/2026 at 12:32:25 PM
a lot of cameras do have an in camera hardware based cryptographic signing (i think it runs off its own chip on them?) but any modifications to the image immediately break that. so for stuff like journalism etc where youd hopefully have a lot of settings dialed in on camera for quick turn around would keep that fingerprint intact.by exhumet
8/17/2026 at 1:36:40 PM
Editing is fine as long as you keep the signed original for validation purposes.by davidgay
8/17/2026 at 12:22:10 PM
[dead]by SadErn
8/17/2026 at 9:54:18 AM
> blindly trusting they won't train on any of thatbeing allowed to train on any data that you can legally obtain ought to be a right for anyone.
After all, i am allowed to learn off anything i can legally read (and perhaps even illegally read). The only thing not allowed (rightly so) is to produce a copy with enough similarities that it can be replacing the original.
by chii
8/17/2026 at 10:03:58 AM
It's a bit different when "training on any data" means basically storing a lossily-compressed copy of that data, that could be spit out years later if the model decides to do so.by bonzini
8/17/2026 at 10:35:10 AM
It's exactly the same problem as with humans, though.It's part of why we sign NDAs, and why their duration is measured in years (and that's not even targeting the human retention - just duration after which information ages enough that its disclosure is not likely to negatively impact anyone who cares).
by TeMPOraL
8/17/2026 at 10:38:19 AM
It's not exactly the same problem, in that you can parallelize usage of an LLM and copy it over to another computer, but cannot do the same things with a brain. Put it another way, humans do not have the processing power needed to answer hundreds of millions of queries per day, while LLMs do.by bonzini
8/17/2026 at 10:43:55 AM
It’s not because humans are rate limited which is the entire point.by gmerc
8/18/2026 at 3:39:11 AM
Weirdly I think humans should have more rights than machinesby afiori
8/18/2026 at 9:05:15 AM
Yes, but in a big way, denying capabilities to LLMs is denying the capability for people to exercise the rights they have. LLMs are user agent tools.by TeMPOraL
8/19/2026 at 12:37:24 PM
Yes, but i cannot create a robot that can get married, people do not have the right to give personhood to toolsby afiori
8/17/2026 at 10:35:11 AM
> being allowed to train on any data that you can legally obtain ought to be a right for anyone.I have the opposit viewpoint to the extreme. They shouldn't be allowed to even read that data until they are very clear about what they will or not do with it.
Can they publish it? Can they store it? Can they use the information in it on prediction markets? Etc.
Humans reading texts historically come with little negative consequences, but machines reading and processing texts en masse is more dangerous and should be regulated.
by bmacho
8/17/2026 at 12:31:37 PM
> Humans reading texts historically come with little negative consequencesIDK, we do have laws against opening other people's mail. Those have been on the books for hundreds of years. Seems like someone figured out a while ago that certain unauthorized humans reading certain restricted text wouldn't be good.
by moron4hire
8/17/2026 at 11:11:10 AM
> Humans reading texts historically come with little negative consequences, but machines reading and processing texts en masse is more dangerous and should be regulated.Citation needed. This is sounding tautological.
by UqWBcuFx6NV4r
8/17/2026 at 11:42:08 AM
Not really hard to find example: License plate ALPR like Flock ?by Timshel
8/17/2026 at 3:30:48 PM
And not really hard to find examples of humans reading text being way more consequential than that. See: the Bible, nazi propaganda, etc.by ragequittah
8/18/2026 at 3:37:42 AM
A legislative solution to Flock seems a lot more feasible than one for literacyby afiori
8/17/2026 at 11:50:38 AM
> After all, i am allowed to learn off anything i can legally read (and perhaps even illegally read).Are you a tool?
Because humans gets rights, tools don't.
Arguing that untrained or partially trained models should have have rights is a different argument to arguing that a trained model should get the same rights as a human.
by lelanthran
8/17/2026 at 4:40:50 PM
What if I'm reading it for work? What am I but a tool for the corporatioN?by slackfan
8/17/2026 at 4:47:28 PM
Are you claiming that you have no human rights while you are at work?by lelanthran
8/17/2026 at 10:25:07 AM
At the risk of stating the obvious, there are a lot of legal rights that are human-specific (voting, holding office, filling lawsuits, etc.). It's not at all obvious why you think that you as a human being legally allowed to learn from something implies that it should be legal to train an LLM on.by saghm
8/17/2026 at 10:12:52 AM
> The only thing not allowed (rightly so) is to produce a copy with enough similarities that it can be replacing the original.But LLMs are replacing the original, just in different words.
And what does 'legally obtain' mean in this context? Copyrighted content is usually licensed for specific purposes. So if a license is given from training your LLM, then by all means do! But what if the license is 'for personal use'... ?
by vanviegen
8/17/2026 at 10:40:37 AM
Oh so if I use mickey mouse in a completely original production that doesn't replace the existing work by Walt Disney, you reckon they'll be fine with that?by exe34
8/17/2026 at 10:44:46 AM
You are one person. The corporation is not. Scale mattersby passwordoops
8/17/2026 at 10:54:44 AM
> You are one person. The corporation is not. Scale mattersCorrect, if you violate it too often to count, you have to pay around less than ~2.5ct per violation.
So the lesson here is: Create a company to do torrenting professionally, and resell its values for higher prices. Then get sued and pay a dime on the dollar you made.
edit: Actually it's 2.5ct per violation.
by cookiengineer
8/17/2026 at 1:39:36 PM
Anthropic settled for $3,000 per book(1) in the settlement, so it provably depends of whether the government likes you or not.(1) https://apnews.com/article/ai-anthropic-copyright-settlement...
by subscribed
8/17/2026 at 9:59:38 AM
Why would that be a legal right?Why should we hand over even MORE power to the owner class?
In a fantasy world this could be possible yes.
by Gud
8/17/2026 at 10:05:49 AM
Copyright (or any other such restriction on free use of information) creates power for owners by the simple fact that it turns information into something that can be owned.by rcxdude
8/17/2026 at 10:12:08 AM
Make it a right, then companies/universities will think twice before using said APIs. Instead of this grey area where we will never know.by juggle-anyhow
8/17/2026 at 10:07:40 AM
We don't hand over more power to the owner class by making fewer things ownable.by m12k