# Min-p sampling

> Min-p sampling (Nguyen et al., 2024) keeps only tokens with p ≥ p_base·p_max, so the cut-off scales with the model's confidence: tight when one token dominates, wide when many are plausible, and stable at high temperature.

Source: https://www.mlfoundry.in/learn/min-p-sampling  
Section: Generative AI › Decoding and Generation Control  
Level: advanced · 7 min read · updated 2026-10-21

**Min-p sampling:** is a truncation rule for stochastic decoding. At each step it finds the largest token probability $p_{\max}$, discards every token whose probability is below $p_\text{base}\cdot p_{\max}$, renormalises the survivors and samples. It was described by Nguyen, Baker, Neo, Roush, Kirsch and Shwartz-Ziv in "Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs" (arXiv, 2024), after first spreading through open-source inference engines such as llama.cpp in 2023.

Sampling from the full softmax lets the long tail of a 30,000–200,000-token vocabulary leak in. Each tail token is unlikely, but together they can hold a lot of mass, and one bad token can derail the rest of the output. **Top-k and top-p sampling** cut the tail with a fixed count or a fixed mass. Neither looks at how peaked the distribution is, so the same setting is too loose on a factual step and too tight on an open one.

Min-p sits in the same place in the decoder as top-p: after the logits and temperature, before the random draw. It is a drop-in replacement or a companion filter. Raising temperature is how you buy diversity (see **Temperature sampling**); min-p is meant to keep that diversity among plausible tokens.

## The rule

$$
p=\operatorname{softmax}(z/T),\qquad \mathcal V_\text{min}=\{\,i: p_i\ge p_\text{base}\cdot\max_j p_j\,\},\qquad q_i=\frac{p_i\,\mathbb 1[i\in\mathcal V_\text{min}]}{\sum_{j\in\mathcal V_\text{min}}p_j}
$$

- $z$: next-token logits over the vocabulary
- $T$: temperature, applied before the filter in most engines
- $p_\text{base}$: the user's min-p setting, typically 0.05–0.1; 0 disables the filter
- $q$: the distribution actually sampled from

The top token is always kept, so $p_\text{base}=1$ (with no ties) is greedy decoding. The size of the kept set is not fixed: it is every token within a constant probability ratio of the leader. That ratio is easier to read in logit space.

$$
\frac{p_i}{p_{\max}}=e^{(z_i-z_{\max})/T}\ge p_\text{base}\iff z_i\ge z_{\max}+T\ln p_\text{base}
$$

- $T\ln p_\text{base}$: the logit margin below the leader; $\ln 0.1=-2.30$, so at $T=1$ a token survives if it is within 2.30 logits of the top

When temperature comes first, the margin grows linearly with $T$: at $T=3$ and $p_\text{base}=0.1$ it is 6.91 logits. Min-p is far more robust to temperature than top-p, but not immune to it. Some engines let you apply temperature after the filter instead. Then the kept set is fixed by the $T=1$ distribution and temperature only reshapes the survivors.

> Figure: The same p_base = 0.1 on two toy next-token distributions at T = 1. Confident step: p_max = 0.883, cut-off 0.088, only Paris survives. Open step: p_max = 0.168, cut-off 0.017, all ten colours survive. In both panels the cut-off sits at 10% of the top bar, and every one of the 400 tail tokens falls below it. Top-p 0.9 would keep 2 and 70 tokens, 60 of the 70 from the tail.

Min-p compared with the other truncation rules (k, p and p_base are the user settings)

| Rule | Keeps | Adapts to confidence? | Typical failure |
| --- | --- | --- | --- |
| Top-k | the k most likely tokens | no | k = 50 keeps 49 junk tokens on a factual step |
| Top-p (nucleus) | smallest set with mass ≥ p | partly | flat or hot distributions pull in hundreds of tail tokens |
| Min-p | tokens with p ≥ p_base · p_max | yes, linearly in p_max | very high T or tiny p_base still lets the tail in |

**Where you will find it**

- **Hugging Face transformers:** `generate(..., do_sample=True, min_p=0.1)`; unset by default
- **vLLM:** `SamplingParams(min_p=0.1)`; 0.0 disables it
- **llama.cpp:** `--min-p`; the sampler order is configurable with `--samplers`
- **Paper setting:** benchmarks included GPQA, GSM8K with chain of thought, and AlpacaEval creative writing, swept across temperatures; the advantage over top-p was largest at high T

> Figure: Ten named tokens plus a 400-token tail with logits 0.5 to −3.5. Defaults (open colour prompt, T = 1.5, p_base 0.1, top-p 0.9): untruncated, the tail holds 54.0% of the mass; min-p keeps the 10 colours (tail 0%, P(blue) = 0.169), top-p keeps 223 tokens and 48.9% tail mass. Raise T to 3: min-p keeps 251 tokens with 83.1% tail mass (p_base 0.2 cuts that to 43 tokens, 48.1%). Switch the order to mask-then-temperature and it keeps 10 again. On the confident prompt at T = 1.5, min-p keeps 1 token and top-p 193.

## Worked example

Seven candidate tokens with $p=(0.50,0.20,0.10,0.08,0.05,0.04,0.03)$ and $p_\text{base}=0.15$. The cut-off is $0.15\times0.50=0.075$.

Min-p 0.15 keeps four tokens (mass 0.88) and renormalises; top-p 0.9 would keep five, because the cumulative mass first reaches 0.9 at the fifth token (0.93)

| Token | p | Cumulative p | p ≥ 0.075? | q after min-p |
| --- | --- | --- | --- | --- |
| A | 0.50 | 0.50 | yes | 0.50 / 0.88 = 0.568 |
| B | 0.20 | 0.70 | yes | 0.227 |
| C | 0.10 | 0.80 | yes | 0.114 |
| D | 0.08 | 0.88 | yes | 0.091 |
| E | 0.05 | 0.93 | no | 0 |
| F | 0.04 | 0.97 | no | 0 |
| G | 0.03 | 1.00 | no | 0 |

If the leader had $p_{\max}=0.12$ instead, the cut-off would fall to $0.018$ and many more tokens would pass. Top-p sees only the running sum; min-p sees the shape.

```python
import numpy as np

def min_p_filter(logits, min_p=0.1, temperature=1.0):
    """Temperature, then min-p: keep tokens with p >= min_p * p_max, renormalise."""
    z = logits / temperature
    p = np.exp(z - z.max())
    p /= p.sum()
    keep = p >= min_p * p.max()
    q = np.where(keep, p, 0.0)
    return q / q.sum()

p = np.array([0.50, 0.20, 0.10, 0.08, 0.05, 0.04, 0.03])
q = min_p_filter(np.log(p), min_p=0.15)
print(np.round(q, 3))            # [0.568 0.227 0.114 0.091 0.    0.    0.   ]

rng = np.random.default_rng(0)
token = rng.choice(len(q), p=q)  # sample from the truncated distribution

# transformers: model.generate(**inputs, do_sample=True, temperature=1.5, min_p=0.1)
```

Min-p in numpy (runs as is). In Hugging Face transformers the same filter is the min_p argument of generate (not run here).

> **Stacking filters and order:** Engines often apply top-k, top-p and min-p together, each on the output of the last, and with a fixed temperature position. A leftover top-p 0.9 or top-k 40 can silently undo the wider set that min-p would allow; temperature before versus after the filter changes the kept set at high $T$. When you switch to min-p, set top-p to 1.0 and top-k to 0 (disabled) unless you mean to combine them, and check your engine's sampler order.

> **Why it suits high temperature:** High $T$ flattens everything, including the tail. Top-p must then add tail tokens until it reaches its mass target. Min-p asks only "is this token within a factor $1/p_\text{base}$ of the best one?", so on a confident step the leader still dominates the kept set.

**Strengths**

- One intuitive knob whose effect scales with the model's own confidence.
- Keeps outputs coherent at temperatures where top-p degrades.
- Cheap: a max, a comparison and a renormalisation; no sort needed.

**Limitations**

- Still a heuristic: the best p_base depends on model, task and temperature, so tune it on your own evaluations.
- With temperature applied first, very high T widens the margin T·ln p_base and the tail returns.
- Not offered by every hosted API; availability varies by provider.

**Question:** With p_base = 0.1, the top token has p = 0.6. Which tokens are kept?

- A. The tokens that make up 90% of the mass
- B. Every token with p ≥ 0.06
- C. Every token with p ≥ 0.1
- D. The 10 most likely tokens

**Answer:** B. Every token with p ≥ 0.06 — The threshold is relative: $0.1\times0.6=0.06$. Option A is top-p, option D is top-k, and option C treats $p_\text{base}$ as an absolute cut-off.

**Question:** Temperature is applied before min-p with p_base = 0.1. How far below the top logit can a token be and still survive at T = 2?

- A. 2.30 logits
- B. 4.61 logits
- C. 0.2 logits
- D. It does not depend on T

**Answer:** B. 4.61 logits — The condition is $z_i\ge z_{\max}+T\ln p_\text{base}$, and $2\times\ln0.1=-4.61$. If the mask were computed before temperature, the margin would stay at 2.30.

**Key points**

- Keep $p_i\ge p_\text{base}\cdot p_{\max}$, renormalise, sample; typical $p_\text{base}$ is 0.05–0.1.
- The cut-off scales with confidence: one survivor on a peaked step, many on a flat step.
- In logits: keep $z_i\ge z_{\max}+T\ln p_\text{base}$; temperature-first widens the margin with T.
- Its main use is high-temperature creative sampling, where top-p admits hundreds of tail tokens.
- Disable leftover top-k/top-p and check the sampler order when you adopt it.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
