# Watermarking generated content

> A green-list watermark (Kirchenbauer et al., 2023) adds δ to the logits of a key-seeded fraction γ of the vocabulary; a detector holding only the key counts green tokens and flags z ≥ 4, a 3 × 10⁻⁵ false-positive rate.

Source: https://www.mlfoundry.in/learn/watermarking-generated-content  
Section: Generative AI › Safety, Alignment and Responsible AI  
Level: advanced · 9 min read · updated 2026-10-21

**Watermarking generated content:** means embedding a signal into a model's output, at generation time, that people cannot see but a detector holding a secret key can test for statistically. For text, the reference scheme is "A Watermark for Large Language Models" (Kirchenbauer, Geiping, Wen, Katz, Miers and Goldstein, ICML 2023). Before each token is sampled it pushes the model towards a pseudorandom "green" part of the vocabulary. Google DeepMind's SynthID-Text (Dathathri et al., *Nature*, 2024) is a deployed variant.

The problem is provenance: was this essay, review or news article produced by a particular model? Post-hoc detectors that classify text as "AI-like" are unreliable and biased against some human writers. A watermark makes the question a hypothesis test with a known false-positive rate. The test needs the provider's cooperation, because only the key holder can detect, and it only covers models that embed the mark.

A watermark is one of three provenance tools. **Metadata** (signed C2PA manifests) is exact but stripped by a screenshot or re-encode. **Detection** classifies content without any cooperation from the generator (see **Deepfake detection**). **Watermarks** live in the content itself, so they survive copying, but they can be weakened by editing. This article covers the watermark.

## The green-list scheme

At step $t$, hash the previous token $x_{t-1}$ with a secret key to seed a random number generator. Use it to split the vocabulary into a green list $G_t$ of size $\gamma|V|$ and a red list of the rest. Add $\delta$ to every green logit, then sample as usual (any temperature or top-p setting):

$$
\hat p_t(v)=\frac{\exp\!\big(\ell_t(v)+\delta\,\mathbb 1[v\in G_t]\big)}{\sum_{u\in V}\exp\!\big(\ell_t(u)+\delta\,\mathbb 1[u\in G_t]\big)},\qquad G_t=\operatorname{split}_\gamma\!\big(\operatorname{hash}(k,x_{t-1})\big)
$$

- $\ell_t(v)$: the model's logit for token $v$ at step $t$
- $\gamma$: green fraction of the vocabulary; 0.25 is a common choice
- $\delta$: logit bias added to green tokens (watermark strength); 2 is a common choice
- $k$: secret key; without it the lists look random

> Figure: Embedding: after "the", the key and previous token seed a split with γ = 0.25. Adding δ = 2 to green logits moves the green tokens' share of the six candidates shown from 0.382 to 0.820: "model" rises from 0.264 to 0.566 while the top red token "text" falls from 0.356 to 0.103. Detection re-derives each list from the key alone. 105 green tokens out of 200 gives z = 8.98.

A strongly preferred token barely changes: if "Paris" has probability 0.99 it stays the likely choice even when red. The bias acts where the model has real choices, so quality loss is small when entropy is high. This is the difference from the paper's "hard" variant, which bans red tokens and breaks forced phrases.

## Detection is a z-test

The detector tokenises the text, recomputes each green list from the key and the preceding token, and counts green tokens $|s|_G$. Human text knows nothing of the key, so each token is green with probability $\gamma$ and the count is binomial:

$$
z=\frac{|s|_G-\gamma T}{\sqrt{T\,\gamma(1-\gamma)}},\qquad \text{FPR}=P(Z\ge z^*)=1-\Phi(z^*),\quad z^*=4\Rightarrow\text{FPR}=3.2\times10^{-5}
$$

- $T$: number of scored tokens
- $|s|_G$: number of green tokens in the text
- $z^*$: detection threshold; the paper uses 4 (one-sided)

The detector needs no model weights and no API call, only the key, the tokenizer and the hash rule. The strength of the evidence grows like $\sqrt T$. In the best case, a flat next-token distribution, the green probability per token is $p_G=\gamma e^\delta/(\gamma e^\delta+1-\gamma)$. With $\gamma=0.25$, $\delta=2$ that is $0.711$, so $z$ grows by $(0.711-0.25)/\sqrt{0.1875}=1.065$ per $\sqrt{\text{token}}$ and passes 4 after about 14 tokens. Real text has many near-deterministic tokens, so in practice it takes more.

$$
\mathbb E[z]\approx\frac{(\bar p_G-\gamma)\sqrt T}{\sqrt{\gamma(1-\gamma)}},\qquad T_{\text{needed}}\approx\left(\frac{z^*\sqrt{\gamma(1-\gamma)}}{\bar p_G-\gamma}\right)^{2}
$$

- $\bar p_G$: average green probability per token; close to $\gamma$ when the model is nearly deterministic

Worked example: T = 200, γ = 0.25, so E[|s|_G] = 50 and the standard deviation is √37.5 = 6.12

| Text | Green tokens | z | One-sided p | Flag at z ≥ 4? |
| --- | --- | --- | --- | --- |
| Watermarked (viz default) | 105 | (105 − 50)/6.12 = 8.98 | 1.3 × 10⁻¹⁹ | yes |
| Watermarked, lightly edited | 90 | 6.53 | 3.2 × 10⁻¹¹ | yes |
| Heavily paraphrased | 62 | 1.96 | 0.025 | no |
| Human-written | 52 | 0.33 | 0.37 | no |

A one-in-31,600 error rate sounds negligible, but scale matters. Scanning a million human-written documents at $z^*=4$ still flags about 32 of them. Raise the threshold, require a minimum length, and never treat a flag as proof on its own, for example in academic-misconduct cases.

## Strength, entropy and edits

> Figure: A toy bigram LM over 32 words (logits Gaussian × scale), 200 tokens, green list seeded by the previous token. Defaults (γ = 0.25, δ = 2, scale 2, edit rate 0.3, seed 1): watermarked text has 105/200 green, z = 8.98, crossing 4 at T = 35, at a perplexity of 9.04 against 6.43 for unwatermarked text (z = −0.49). Randomly editing 30% of tokens leaves 80 green and z = 4.90, crossing 4 only at T = 144. δ = 1 gives z = 2.61 (never flagged). Logit scale 4 (entropy 0.67 nats) gives z = 3.43, and δ = 4 gives 19.76 at a perplexity of 16.63.

Three lessons show up in the figure. Strength costs quality: a larger $\delta$ raises $z$ and perplexity together. Low-entropy text resists the watermark: when the model is sure of the next token, the bias cannot move it, so code, lists and memorised passages carry little signal. And each edited token damages two tests, its own and the next token's (whose green list it seeds), so the green excess decays roughly like $(1-\varepsilon)^2$. A real paraphraser keeps perplexity low while doing the same damage.

**Schemes and results (check the papers for exact settings)**

- **Kirchenbauer et al. 2023:** Soft green-list watermark with a one-proportion z-test; experiments on OPT models with about 200-token continuations. Describes the emoji attack: ask the model to put an emoji after every word, then delete the emojis, which changes every token's predecessor.
- **Distortion-free sampling:** Aaronson (2022, OpenAI talk): choose the token maximising $r_v^{1/p_v}$ for keyed pseudorandom $r_v$. Each step is still an exact sample from the model. Kuditipudi et al. (2023) and Christ, Gunn and Zamir (2023) give robust and cryptographically undetectable variants.
- **SynthID-Text:** Dathathri et al., *Nature* 2024: tournament sampling with keyed scoring functions, deployed in Google's Gemini; the authors report no measurable drop in user-rated quality in a large live comparison.
- **Images and audio:** HiDDeN (Zhu et al., ECCV 2018) learns an encoder-decoder pair that hides bits in pixels. Tree-Ring (Wen et al., NeurIPS 2023) writes a key pattern into the Fourier transform of a diffusion model's initial noise and detects it by DDIM inversion. Stable Signature (Fernandez et al., ICCV 2023) fine-tunes the latent decoder so every image carries a fixed message. AudioSeal (San Roman et al., 2024) localises a watermark within audio.
- **Regulation:** The EU AI Act (2024, Article 50) requires providers of systems that generate synthetic audio, image, video or text to mark outputs in a machine-readable, detectable way. This is a summary, not legal advice.

```python
import hashlib, math, random

def green_list(prev_id, vocab_size, gamma, key=b"secret-key"):
    seed = int.from_bytes(hashlib.sha256(key + prev_id.to_bytes(4, "big")).digest()[:8], "big")
    ids = list(range(vocab_size))
    random.Random(seed).shuffle(ids)
    return set(ids[: int(gamma * vocab_size)])

def detect(token_ids, vocab_size, gamma=0.25, z_threshold=4.0):
    T = len(token_ids) - 1                      # the first token has no predecessor
    greens = sum(tok in green_list(prev, vocab_size, gamma)
                 for prev, tok in zip(token_ids, token_ids[1:]))
    z = (greens - gamma * T) / math.sqrt(T * gamma * (1 - gamma))
    p = 0.5 * math.erfc(z / math.sqrt(2))      # one-sided p-value
    return {"T": T, "green": greens, "z": round(z, 2), "p": p, "flag": z >= z_threshold}

# toy check: a "generator" that picks a green token 70% of the time vs uniform text
V, rng = 1000, random.Random(0)
wm, plain = [1], [1]
for _ in range(200):
    g = sorted(green_list(wm[-1], V, 0.25))
    wm.append(rng.choice(g) if rng.random() < 0.7 else rng.randrange(V))
    plain.append(rng.randrange(V))
print(detect(wm, V))     # T=200, green=156, z=17.31, p≈2e-67, flag=True
print(detect(plain, V))  # T=200, green=50, z=0.0, p=0.5, flag=False
```

Green-list detection over token ids with a keyed SHA-256 seed (standard library only; runs as is). A real detector must use the generator's exact tokenizer and hashing rule.

> **No watermark is not evidence of a human:** A negative test only says that this key's mark is absent. The text may come from another provider, an open-weight model run without the watermark, or a paraphrase. Watermarks give strong one-sided evidence when present and nothing when absent, the same logic as missing C2PA metadata.

> **Design choices are security choices:** Seeding on one previous token ($h=1$) is robust to edits, because each change breaks at most two tests. But an attacker who queries the model often can estimate which tokens are green after each word and learn to avoid them. Seeding on several previous tokens, or on a keyed hash of a longer window, resists that, but each edit then breaks more tests. Keeping the key private also means only the provider can detect, so public verification needs a detection API or a public-key scheme.

**Strengths**

- Detection is a hypothesis test with a stated false-positive rate, not a black-box score.
- Needs only the key and tokenizer; no model weights or API calls at detection time.
- Cheap at generation: one hash and one vector addition per token, compatible with any sampler.

**Limitations**

- Only works for cooperating providers; open-weight models can be run with the watermark removed.
- Weak on short or low-entropy text (code, facts, formulaic replies).
- Paraphrasing, translation and the emoji attack erode the signal; stronger δ costs fluency.

**Question:** With γ = 0.25, a 400-token human-written text has 115 green tokens. What does the detector conclude at z* = 4?

- A. Flagged: 115 is more than a quarter of 400
- B. Not flagged: z = (115 − 100)/√75 ≈ 1.73
- C. Flagged: z ≈ 4.3
- D. The test cannot be applied to texts longer than 200 tokens

**Answer:** B. Not flagged: z = (115 − 100)/√75 ≈ 1.73 — Expected green count is $0.25\times400=100$ with variance $400\times0.25\times0.75=75$, so $z=15/8.66\approx1.73$. Chance excesses like this are common, which is why the threshold is set in standard deviations, not raw counts.

**Question:** Why does a single substituted token typically remove two green tests, not one?

- A. The detector double-counts edited tokens
- B. The new token may be red, and it also seeds the green list for the following token, re-randomising that test
- C. The z-score uses T² in the denominator
- D. Substitutions always change the tokenizer's segmentation

**Answer:** B. The new token may be red, and it also seeds the green list for the following token, re-randomising that test — With context width 1, token $x_t$ is tested against the list seeded by $x_{t-1}$. Changing $x_t$ re-randomises its own test and the test of $x_{t+1}$, so both become green only with probability $\gamma$. Wider contexts make each edit damage more tests.

**Key points**

- Embed: hash(key, previous token) splits the vocabulary; add $\delta$ to the $\gamma|V|$ green logits, then sample normally.
- Detect: count green tokens, $z=(|s|_G-\gamma T)/\sqrt{T\gamma(1-\gamma)}$; $z\ge4$ means a false-positive rate of $3.2\times10^{-5}$ per text.
- Evidence grows like $\sqrt T$ and with entropy; short or low-entropy text carries little signal.
- Stronger δ means faster detection and higher perplexity; distortion-free schemes (Aaronson's, SynthID-Text's non-distortionary mode) avoid shifting the distribution.
- Edits, paraphrases and the emoji attack weaken it; absence of a mark proves nothing.
- Images and audio use learned or noise-space watermarks (HiDDeN, Tree-Ring, Stable Signature, AudioSeal); metadata (C2PA) complements but does not replace them.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
