Definition
Repetition penalty is a decode-time change to the next-token logits that makes tokens already present in the context less likely. The multiplicative form comes from CTRL (Keskar, McCann, Varshney, Xiong and Socher, "CTRL: A Conditional Transformer Language Model for Controllable Generation", 2019). The additive frequency and presence penalties are the form exposed by the OpenAI API and copied by many OpenAI-compatible servers.
Greedy and low-temperature decoding fall into loops. Once a phrase has been generated, the model's own output makes the same continuation more likely, and the text repeats a sentence until the token limit. Holtzman et al. ("The Curious Case of Neural Text Degeneration", ICLR 2020) documented this for greedy and beam search. Small models, long outputs and near-greedy settings make it worse.
Penalties are the cheapest fix: they need no retraining and sit in the logits-processing stage, next to Temperature sampling and Top-k and top-p sampling. They work on token identities, not meaning. A penalty cannot tell a stuck loop from a name, a keyword or a closing bracket that has to appear again.
The two families
- the tokens generated so far (Hugging Face counts the whole input, prompt included)
- repetition penalty; CTRL found a good balance, disables it
- temperature; CTRL paired the penalty with near-greedy sampling
Dividing a positive logit by lowers it, but dividing a negative one raises it. Hugging Face's repetition_penalty therefore divides positive logits and multiplies negative ones, so a seen token always loses. Either way the penalty is the same whether the token appeared once or fifty times.
- how many times token has been sampled so far
- frequency penalty: grows linearly with each repeat
- presence penalty: one-off, applied once a token has appeared at all
The additive form has a clean reading: each repeat multiplies the token's odds against an unseen token by . With that is a factor 0.61 per occurrence, and 0.082 after five occurrences. Negative values are allowed and encourage repetition.
| Where | Parameter | Form | Neutral value |
|---|---|---|---|
| Hugging Face generate | repetition_penalty | CTRL θ, sign-aware | 1.0 |
| Hugging Face generate | no_repeat_ngram_size | hard ban on any repeated n-gram | 0 (off) |
| OpenAI API (chat/completions) | frequency_penalty, presence_penalty | additive, each in [−2, 2] | 0 |
| vLLM SamplingParams | repetition_penalty, frequency_penalty, presence_penalty | both families | 1.0, 0, 0 |
Documented guidance
- CTRL (2019)
- θ ≈ 1.2 with near-greedy sampling gave a good balance between truthful generation and lack of repetition
- OpenAI docs (2023–24)
- values around 0.1–1 reduce repetition somewhat; up to 2 suppresses it strongly but can noticeably degrade quality
- n-gram blocking
- trigram blocking at test time was used in abstractive summarisation (Paulus et al., 2018); it is a hard rule, not a soft penalty
- Training-side fix
- unlikelihood training (Welleck et al., ICLR 2020) penalises repeats during training instead of at decode time
The viz shows the central difference. A flat penalty (CTRL or presence) only reorders seen against unseen tokens. When every plausible continuation has been used once, the original ranking returns and so does the loop. A frequency penalty keeps rising, so the loop's tokens fall further behind every time it comes round. Push it too far and fluency pays: the mean log-probability under the unpenalised model drops and implausible tokens start to win.
Worked example
The text so far is "the cat sat on the", so cat has count 1 and the has count 2. Five candidates follow "the"; probabilities are a softmax over these five.
| Token | c | z (p) | CTRL θ = 1.2 | z − 0.5c − 0.5·[c > 0] |
|---|---|---|---|---|
| cat | 1 | 4.00 (0.454) | 4.00 / 1.2 = 3.33 (0.299) | 4.00 − 0.5 − 0.5 = 3.00 (0.235) |
| dog | 0 | 3.50 (0.275) | 3.50 (0.353) | 3.50 (0.387) |
| mat | 0 | 3.00 (0.167) | 3.00 (0.214) | 3.00 (0.235) |
| rug | 0 | 2.50 (0.101) | 2.50 (0.130) | 2.50 (0.142) |
| the | 2 | −1.00 (0.003) | −1.00 × 1.2 = −1.20 (0.003) | −1.00 − 1.0 − 0.5 = −2.50 (0.001) |
A naive CTRL implementation would compute for "the", which raises a token it is meant to suppress. The sign-aware rule avoids that.
1import numpy as np23def apply_penalties(logits, history, repetition=1.0, frequency=0.0, presence=0.0):4 """CTRL/HF repetition penalty, then OpenAI-style frequency/presence penalties."""5 z = logits.astype(float).copy()6 counts = np.bincount(history, minlength=len(z))7 seen = counts > 08 z[seen] = np.where(z[seen] > 0, z[seen] / repetition, z[seen] * repetition)9 return z - frequency * counts - presence * seen1011np.set_printoptions(precision=3)12vocab = ["cat", "dog", "mat", "rug", "the"]13logits = np.array([4.0, 3.5, 3.0, 2.5, -1.0])14history = [4, 0, 4] # ids in "the cat ... the": cat 1, the 215print(apply_penalties(logits, history, repetition=1.2)) # [ 3.333 3.5 3. 2.5 -1.2 ]16print(apply_penalties(logits, history, frequency=0.5, presence=0.5)) # [ 3. 3.5 3. 2.5 -2.5]Where it's used
Small or quantised models
near-greedy decoding loops quickly; θ ≈ 1.1–1.2 or a light frequency penalty is a common default in local runners
Long-form generation
stories and reports drift into repeated phrases; a frequency penalty of 0.1–0.5 discourages it
Brainstorming lists
a presence penalty pushes the model towards new tokens, which tends to raise topic variety
Check yourself
A token has appeared 4 times. With frequency_penalty = 0.3 and presence_penalty = 0.6, how much is subtracted from its logit?
Check yourself
Why can a CTRL-style penalty fail to stop a long loop where a frequency penalty succeeds?
Remember
- ✓CTRL (2019): seen tokens get (sign-aware in HF), ; flat in the count.
- ✓OpenAI-style: , each in ; each repeat multiplies the odds by .
- ✓Flat penalties break short loops; only the frequency term keeps pushing a long loop away.
- ✓Too much penalty costs fluency and breaks code, JSON and names that must repeat.
- ✓Penalties act on token IDs: check whether your engine counts the prompt.