Watermarking generated content

A green-list watermark (Kirchenbauer et al., 2023) adds δ to the logits of a key-seeded fraction γ of the vocabulary; a detector holding only the key counts green tokens and flags z ≥ 4, a 3 × 10⁻⁵ false-positive rate.

9 min read781 wordsadvanced0 viewsupdated 2026-10-21

Definition

Watermarking generated content means embedding a signal into a model's output, at generation time, that people cannot see but a detector holding a secret key can test for statistically. For text, the reference scheme is "A Watermark for Large Language Models" (Kirchenbauer, Geiping, Wen, Katz, Miers and Goldstein, ICML 2023). Before each token is sampled it pushes the model towards a pseudorandom "green" part of the vocabulary. Google DeepMind's SynthID-Text (Dathathri et al., Nature, 2024) is a deployed variant.

The problem is provenance: was this essay, review or news article produced by a particular model? Post-hoc detectors that classify text as "AI-like" are unreliable and biased against some human writers. A watermark makes the question a hypothesis test with a known false-positive rate. The test needs the provider's cooperation, because only the key holder can detect, and it only covers models that embed the mark.

A watermark is one of three provenance tools. Metadata (signed C2PA manifests) is exact but stripped by a screenshot or re-encode. Detection classifies content without any cooperation from the generator (see Deepfake detection). Watermarks live in the content itself, so they survive copying, but they can be weakened by editing. This article covers the watermark.

The green-list scheme

At step tt, hash the previous token xt−1x_{t-1} with a secret key to seed a random number generator. Use it to split the vocabulary into a green list GtG_t of size γ∣V∣\gamma|V| and a red list of the rest. Add δ\delta to every green logit, then sample as usual (any temperature or top-p setting):

p^t(v)=exp⁡ ⁣(ℓt(v)+δ 1[v∈Gt])∑u∈Vexp⁡ ⁣(ℓt(u)+δ 1[u∈Gt]),Gt=split⁡γ ⁣(hash⁡(k,xt−1))\hat p_t(v)=\frac{\exp\!\big(\ell_t(v)+\delta\,\mathbb 1[v\in G_t]\big)}{\sum_{u\in V}\exp\!\big(\ell_t(u)+\delta\,\mathbb 1[u\in G_t]\big)},\qquad G_t=\operatorname{split}_\gamma\!\big(\operatorname{hash}(k,x_{t-1})\big)
ℓt(v)\ell_t(v)
the model's logit for token vv at step tt
γ\gamma
green fraction of the vocabulary; 0.25 is a common choice
δ\delta
logit bias added to green tokens (watermark strength); 2 is a common choice
kk
secret key; without it the lists look random
embed: at every step (γ = 0.25, δ = 2)previous token"the"hash(key, prev)seeds a randomvocab splitgreen: γ|V|red: the restgreen logit +δtoken logitp before → aftertext2.30.356 → 0.103model2.00.264 → 0.566word1.50.160 → 0.046key1.20.118 → 0.254list0.50.059 → 0.017one0.20.044 → 0.013green mass 0.382 → 0.820; sample as usualdetect: key only, no modelre-derive each token's listfrom its predecessor:themodelsayseachtokenisnotredsowe… over T = 200 tokens:|s|_G = 105 green (E = 50)z = (105 − 50)/√37.5 = 8.98 > 4 → flaghuman text: z ≈ N(0, 1)P(z ≥ 4) = 3.2 × 10⁻⁵what weakens the signallow entropyforced tokens (code, facts) cannot move to greeneditingone changed token can flip two green tests (its own and the next)paraphrasinga second model rewrites the text; most token pairs changeemoji attackask for an emoji between words, then delete them allshort textz grows like √T; tens of tokens are needed even with δ = 2image and audio schemes embed in pixels, noise or waveforms instead; same detect-by-key idea
fig.Embedding: after "the", the key and previous token seed a split with γ = 0.25. Adding δ = 2 to green logits moves the green tokens' share of the six candidates shown from 0.382 to 0.820: "model" rises from 0.264 to 0.566 while the top red token "text" falls from 0.356 to 0.103. Detection re-derives each list from the key alone. 105 green tokens out of 200 gives z = 8.98.

A strongly preferred token barely changes: if "Paris" has probability 0.99 it stays the likely choice even when red. The bias acts where the model has real choices, so quality loss is small when entropy is high. This is the difference from the paper's "hard" variant, which bans red tokens and breaks forced phrases.

Detection is a z-test

The detector tokenises the text, recomputes each green list from the key and the preceding token, and counts green tokens ∣s∣G|s|_G. Human text knows nothing of the key, so each token is green with probability γ\gamma and the count is binomial:

z=∣s∣G−γTT γ(1−γ),FPR=P(Z≥z∗)=1−Φ(z∗),z∗=4⇒FPR=3.2×10−5z=\frac{|s|_G-\gamma T}{\sqrt{T\,\gamma(1-\gamma)}},\qquad \text{FPR}=P(Z\ge z^*)=1-\Phi(z^*),\quad z^*=4\Rightarrow\text{FPR}=3.2\times10^{-5}
TT
number of scored tokens
∣s∣G|s|_G
number of green tokens in the text
z∗z^*
detection threshold; the paper uses 4 (one-sided)

The detector needs no model weights and no API call, only the key, the tokenizer and the hash rule. The strength of the evidence grows like T\sqrt T. In the best case, a flat next-token distribution, the green probability per token is pG=γeδ/(γeδ+1−γ)p_G=\gamma e^\delta/(\gamma e^\delta+1-\gamma). With γ=0.25\gamma=0.25, δ=2\delta=2 that is 0.7110.711, so zz grows by (0.711−0.25)/0.1875=1.065(0.711-0.25)/\sqrt{0.1875}=1.065 per token\sqrt{\text{token}} and passes 4 after about 14 tokens. Real text has many near-deterministic tokens, so in practice it takes more.

E[z]≈(pˉG−γ)Tγ(1−γ),Tneeded≈(z∗γ(1−γ)pˉG−γ)2\mathbb E[z]\approx\frac{(\bar p_G-\gamma)\sqrt T}{\sqrt{\gamma(1-\gamma)}},\qquad T_{\text{needed}}\approx\left(\frac{z^*\sqrt{\gamma(1-\gamma)}}{\bar p_G-\gamma}\right)^{2}
pˉG\bar p_G
average green probability per token; close to γ\gamma when the model is nearly deterministic
TextGreen tokenszOne-sided pFlag at z ≥ 4?
Watermarked (viz default)105(105 − 50)/6.12 = 8.981.3 × 10⁻¹⁹yes
Watermarked, lightly edited906.533.2 × 10⁻¹¹yes
Heavily paraphrased621.960.025no
Human-written520.330.37no
Worked example: T = 200, γ = 0.25, so E[|s|_G] = 50 and the standard deviation is √37.5 = 6.12

A one-in-31,600 error rate sounds negligible, but scale matters. Scanning a million human-written documents at z∗=4z^*=4 still flags about 32 of them. Raise the threshold, require a minimum length, and never treat a flag as proof on its own, for example in academic-misconduct cases.

Strength, entropy and edits

watermarked: first 48 of 200 tokens (green list seeded by previous token)inwetobywithcanlistbyweainonetoonethatmoremodelwordintokenonenextandnotnotandinwordseenotainweabyredredwithmodelnotaintextnextandnotainz = 4: false-positive rate 3.2 × 10⁻⁵-4048050100150200tokens scored Tz = (|s|_G − γT) / √(T γ(1 − γ))sequencegreenz at 200first z ≥ 4perplexity*watermarked105/200 (52.5%)8.98T = 359.04unwatermarked47/200 (23.5%)-0.49never6.43watermarked + edits80/200 (40.0%)4.90T = 14454.02* under the unwatermarked model · mean entropy of its next-token distribution 1.96 natsedits replace a token with a random one; each edit also changes the next token's green list
show tokens ofseed
fig.A toy bigram LM over 32 words (logits Gaussian × scale), 200 tokens, green list seeded by the previous token. Defaults (γ = 0.25, δ = 2, scale 2, edit rate 0.3, seed 1): watermarked text has 105/200 green, z = 8.98, crossing 4 at T = 35, at a perplexity of 9.04 against 6.43 for unwatermarked text (z = −0.49). Randomly editing 30% of tokens leaves 80 green and z = 4.90, crossing 4 only at T = 144. δ = 1 gives z = 2.61 (never flagged). Logit scale 4 (entropy 0.67 nats) gives z = 3.43, and δ = 4 gives 19.76 at a perplexity of 16.63.

Three lessons show up in the figure. Strength costs quality: a larger δ\delta raises zz and perplexity together. Low-entropy text resists the watermark: when the model is sure of the next token, the bias cannot move it, so code, lists and memorised passages carry little signal. And each edited token damages two tests, its own and the next token's (whose green list it seeds), so the green excess decays roughly like (1−ε)2(1-\varepsilon)^2. A real paraphraser keeps perplexity low while doing the same damage.

Schemes and results (check the papers for exact settings)

Kirchenbauer et al. 2023
Soft green-list watermark with a one-proportion z-test; experiments on OPT models with about 200-token continuations. Describes the emoji attack: ask the model to put an emoji after every word, then delete the emojis, which changes every token's predecessor.
Distortion-free sampling
Aaronson (2022, OpenAI talk): choose the token maximising rv1/pvr_v^{1/p_v} for keyed pseudorandom rvr_v. Each step is still an exact sample from the model. Kuditipudi et al. (2023) and Christ, Gunn and Zamir (2023) give robust and cryptographically undetectable variants.
SynthID-Text
Dathathri et al., Nature 2024: tournament sampling with keyed scoring functions, deployed in Google's Gemini; the authors report no measurable drop in user-rated quality in a large live comparison.
Images and audio
HiDDeN (Zhu et al., ECCV 2018) learns an encoder-decoder pair that hides bits in pixels. Tree-Ring (Wen et al., NeurIPS 2023) writes a key pattern into the Fourier transform of a diffusion model's initial noise and detects it by DDIM inversion. Stable Signature (Fernandez et al., ICCV 2023) fine-tunes the latent decoder so every image carries a fixed message. AudioSeal (San Roman et al., 2024) localises a watermark within audio.
Regulation
The EU AI Act (2024, Article 50) requires providers of systems that generate synthetic audio, image, video or text to mark outputs in a machine-readable, detectable way. This is a summary, not legal advice.
python
1import hashlib, math, random
2
3def green_list(prev_id, vocab_size, gamma, key=b"secret-key"):
4 seed = int.from_bytes(hashlib.sha256(key + prev_id.to_bytes(4, "big")).digest()[:8], "big")
5 ids = list(range(vocab_size))
6 random.Random(seed).shuffle(ids)
7 return set(ids[: int(gamma * vocab_size)])
8
9def detect(token_ids, vocab_size, gamma=0.25, z_threshold=4.0):
10 T = len(token_ids) - 1 # the first token has no predecessor
11 greens = sum(tok in green_list(prev, vocab_size, gamma)
12 for prev, tok in zip(token_ids, token_ids[1:]))
13 z = (greens - gamma * T) / math.sqrt(T * gamma * (1 - gamma))
14 p = 0.5 * math.erfc(z / math.sqrt(2)) # one-sided p-value
15 return {"T": T, "green": greens, "z": round(z, 2), "p": p, "flag": z >= z_threshold}
16
17# toy check: a "generator" that picks a green token 70% of the time vs uniform text
18V, rng = 1000, random.Random(0)
19wm, plain = [1], [1]
20for _ in range(200):
21 g = sorted(green_list(wm[-1], V, 0.25))
22 wm.append(rng.choice(g) if rng.random() < 0.7 else rng.randrange(V))
23 plain.append(rng.randrange(V))
24print(detect(wm, V)) # T=200, green=156, z=17.31, p≈2e-67, flag=True
25print(detect(plain, V)) # T=200, green=50, z=0.0, p=0.5, flag=False
Green-list detection over token ids with a keyed SHA-256 seed (standard library only; runs as is). A real detector must use the generator's exact tokenizer and hashing rule.

Use when

  • +Detection is a hypothesis test with a stated false-positive rate, not a black-box score.
  • +Needs only the key and tokenizer; no model weights or API calls at detection time.
  • +Cheap at generation: one hash and one vector addition per token, compatible with any sampler.

Avoid when

  • −Only works for cooperating providers; open-weight models can be run with the watermark removed.
  • −Weak on short or low-entropy text (code, facts, formulaic replies).
  • −Paraphrasing, translation and the emoji attack erode the signal; stronger δ costs fluency.

Check yourself

With γ = 0.25, a 400-token human-written text has 115 green tokens. What does the detector conclude at z* = 4?

Check yourself

Why does a single substituted token typically remove two green tests, not one?

Remember

  • ✓Embed: hash(key, previous token) splits the vocabulary; add δ\delta to the γ∣V∣\gamma|V| green logits, then sample normally.
  • ✓Detect: count green tokens, z=(∣s∣G−γT)/Tγ(1−γ)z=(|s|_G-\gamma T)/\sqrt{T\gamma(1-\gamma)}; z≥4z\ge4 means a false-positive rate of 3.2×10−53.2\times10^{-5} per text.
  • ✓Evidence grows like T\sqrt T and with entropy; short or low-entropy text carries little signal.
  • ✓Stronger δ means faster detection and higher perplexity; distortion-free schemes (Aaronson's, SynthID-Text's non-distortionary mode) avoid shifting the distribution.
  • ✓Edits, paraphrases and the emoji attack weaken it; absence of a mark proves nothing.
  • ✓Images and audio use learned or noise-space watermarks (HiDDeN, Tree-Ring, Stable Signature, AudioSeal); metadata (C2PA) complements but does not replace them.
← back to Generative AI roadmap