# Logit bias

> Logit bias adds a fixed value b to chosen token IDs' logits before softmax, shifting each token's log-odds by exactly b; −100 bans a token, +100 on a set restricts output to it, but it works on token IDs, not words.

Source: https://www.mlfoundry.in/learn/logit-bias  
Section: Generative AI › Decoding and Generation Control  
Level: core · 7 min read · updated 2026-10-21

**Logit bias:** is a per-token offset added to the model's next-token logits before sampling: a map from token ID to a number $b$. Positive values make a token more likely, negative values less likely, and very large magnitudes ban it or force a choice among the biased tokens. The OpenAI API exposes it as `logit_bias`, with values from −100 to 100; Hugging Face transformers, vLLM and llama.cpp offer equivalents.

Prompts ask; logit bias enforces, at the level of single tokens. Typical jobs are banning a word or a formatting token, nudging a label up or down, and turning a chat model into a classifier that can only emit one of $k$ label tokens. It costs nothing at inference time and needs no access to the weights. That is why it was one of the first steering knobs offered by hosted APIs.

It sits in the same logits-processing stage as **Temperature sampling**, **Top-k and top-p sampling** and **Repetition and frequency penalties**. **Constrained decoding** is the general version: it recomputes a mask at every step from a grammar or schema. Logit bias is a static map applied identically at every step.

## The mechanism

$$
z'_i=z_i+b_i\qquad\Longrightarrow\qquad p'_i=\frac{p_i\,e^{b_i}}{\sum_j p_j\,e^{b_j}}
$$

- $z_i$: the model's logit for token ID $i$
- $b_i$: the bias for that ID; 0 for every token not in the map
- $p,\ p'$: next-token probabilities before and after the bias

For a single biased token the effect is exact in log-odds: the bias adds $b$ to $\log\frac{p}{1-p}$. In probability it depends on where you start, which is why the same $+2$ helps a rare token far more in relative terms than a common one.

$$
\operatorname{logit}(p')=\operatorname{logit}(p)+b\quad\iff\quad p'=\frac{p\,e^{b}}{1-p+p\,e^{b}}
$$

- $e^{b}$: the odds multiplier: $e^{1}=2.72$, $e^{2}=7.39$, $e^{-100}\approx3.7\times10^{-44}$

So −100 is a ban in practice and +100 makes a token beat any unbiased competitor. If several tokens all get +100, they compete on their original logits: the output is restricted to that set at the model's own relative odds.

> Figure: A one-token classifier. The request adds +100 to the token IDs of ␣positive, ␣negative and ␣neutral (toy IDs). Before the bias the model splits its mass across ten tokens, with ␣mixed (0.190) second and a newline third. After it, only the labels survive, at their original odds e³ : e^1.8 : e^1.2 = 0.682 : 0.205 : 0.113. Below: the bias acts on token IDs, so one word can need several entries, a word piece hits every word that shares it, and a bias map belongs to one tokenizer.

**Interfaces (as documented in 2024)**

- **OpenAI API:** `logit_bias`: JSON object mapping token ID (as a string key) to a bias in [−100, 100]; the docs say ±1 shifts likelihood slightly and ±100 should ban or force a token
- **Token IDs:** come from the model's tokenizer, e.g. `tiktoken` encodings `cl100k_base` (GPT-3.5/GPT-4) and `o200k_base` (GPT-4o)
- **Hugging Face:** `sequence_bias` maps token-ID tuples to biases; `bad_words_ids` bans sequences; `suppress_tokens` sets logits to −∞
- **llama.cpp:** `--logit-bias TOKEN_ID+BIAS` (or `-inf` to ban) on the command line
- **Scope:** the map is applied at every generated position, not just the first

> Figure: Ten next-token candidates after a sentiment prompt; click a row to set its bias. No bias: ␣positive is the argmax at 0.283 and the three labels hold 41.5% of the mass. Ban ␣positive: the argmax becomes ␣mixed (0.265), yet 17.8% still goes to ␣Positive and ␣pos. Labels only (+100 ×3): ␣positive 0.682, ␣negative 0.205, ␣neutral 0.113. Nudge ␣neutral +2: 0.047 → 0.266, and it becomes the argmax.

## Worked example

Start from the viz's distribution, $p(\text{␣neutral})=0.0468$ and $p(\text{␣positive})=0.2833$.

Every number from p′ = p·e^b / Σ p·e^b; a ban rescales the rest by 1/(1 − 0.2833) = 1.395

| Change | Token | p | Computation | p′ |
| --- | --- | --- | --- | --- |
| b = +2 on ␣neutral | ␣neutral | 0.0468 | 0.0468 × 7.389 / (1 − 0.0468 + 0.0468 × 7.389) | 0.266 |
| b = +2 on ␣neutral | ␣positive | 0.2833 | 0.2833 / 1.299 | 0.218 |
| b = −100 on ␣positive | ␣mixed | 0.1899 | 0.1899 × 1.395 | 0.265 |
| b = −100 on ␣positive | ␣Positive | 0.1042 | 0.1042 × 1.395 | 0.145 |
| +100 on 3 labels | ␣positive | 0.2833 | e³ / (e³ + e^1.8 + e^1.2) | 0.682 |

```python
import tiktoken
from openai import OpenAI

MODEL = "gpt-4o-mini"
enc = tiktoken.encoding_for_model(MODEL)          # o200k_base

labels = [" positive", " negative", " neutral"]   # leading space: as it follows "Sentiment:"
ids = [enc.encode(l) for l in labels]
assert all(len(t) == 1 for t in ids), "each label must be a single token"

resp = OpenAI().chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content":
               "Review: Great food, slow service.\nSentiment:"}],
    max_tokens=1,                                # bias applies at every step
    logit_bias={t[0]: 100 for t in ids},
    logprobs=True, top_logprobs=3,               # see the odds, not just the label
)
print(resp.choices[0].message.content)
```

Restricting a chat model to three label tokens with tiktoken and the OpenAI Python SDK (not run here: neither package was available; the model name is a 2024 example)

> **Tokens are not words:** " positive", "positive" and " Positive" are usually different token IDs, and a long or rare word may be split into several pieces (see **Byte-pair encoding (BPE)**). Banning one ID leaves the other spellings open, as the viz shows. Biasing a piece such as " un" affects every word that starts with it. Encode every variant you care about with the model's own tokenizer, and re-check the map when you change models.

> **+100 is sticky:** The bias applies at every step. Force +100 on " positive" with `max_tokens` 50 and the model has to emit " positive" again and again. Use forcing with a one-token output, or use **Constrained decoding** or **Structured output (JSON mode)**, which change the allowed set as generation proceeds.

- **Closed-set classification:** +100 on single-token labels and max_tokens 1; read the label probabilities from logprobs
- **Banning:** −100 on a competitor's name, a profanity list or an unwanted formatting token (every spelling)
- **Soft steering:** ±1 to ±5 to make a style word or label more or less likely without forbidding it

**Question:** A token has p = 0.10. What is its probability after a bias of b = ln 3 ≈ 1.10 on that token alone?

- A. 0.30
- B. 0.25
- C. 0.33
- D. 0.13

**Answer:** B. 0.25 — The odds go from $0.1/0.9=1/9$ to $3/9=1/3$, so $p'=\frac{1/3}{1+1/3}=0.25$. Tripling the odds does not triple the probability.

**Question:** You set +100 on the IDs of ' yes' and ' no'. The model's logits for them are 2.0 and 1.0. What are their probabilities?

- A. 0.5 and 0.5: the bias swamps the logits
- B. About 0.731 and 0.269
- C. 1.0 and 0.0
- D. Unchanged from before

**Answer:** B. About 0.731 and 0.269 — Both get the same +100, so the bias cancels between them: $p(\text{yes})=e^2/(e^2+e^1)=0.731$. Every other token is pushed to essentially zero.

**Key points**

- $z'_i=z_i+b_i$ before softmax; a single token's log-odds shift by exactly $b$.
- −100 bans ($e^{-100}\approx3.7\times10^{-44}$); +100 on a set restricts output to that set at its original odds.
- It is keyed by token ID: encode every spelling with the model's own tokenizer.
- The map applies at every step; pair forcing with max_tokens 1.
- For formats that change token by token, use constrained decoding instead.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
