Definition
Min-p sampling is a truncation rule for stochastic decoding. At each step it finds the largest token probability , discards every token whose probability is below , renormalises the survivors and samples. It was described by Nguyen, Baker, Neo, Roush, Kirsch and Shwartz-Ziv in "Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs" (arXiv, 2024), after first spreading through open-source inference engines such as llama.cpp in 2023.
Sampling from the full softmax lets the long tail of a 30,000–200,000-token vocabulary leak in. Each tail token is unlikely, but together they can hold a lot of mass, and one bad token can derail the rest of the output. Top-k and top-p sampling cut the tail with a fixed count or a fixed mass. Neither looks at how peaked the distribution is, so the same setting is too loose on a factual step and too tight on an open one.
Min-p sits in the same place in the decoder as top-p: after the logits and temperature, before the random draw. It is a drop-in replacement or a companion filter. Raising temperature is how you buy diversity (see Temperature sampling); min-p is meant to keep that diversity among plausible tokens.
The rule
- next-token logits over the vocabulary
- temperature, applied before the filter in most engines
- the user's min-p setting, typically 0.05–0.1; 0 disables the filter
- the distribution actually sampled from
The top token is always kept, so (with no ties) is greedy decoding. The size of the kept set is not fixed: it is every token within a constant probability ratio of the leader. That ratio is easier to read in logit space.
- the logit margin below the leader; , so at a token survives if it is within 2.30 logits of the top
When temperature comes first, the margin grows linearly with : at and it is 6.91 logits. Min-p is far more robust to temperature than top-p, but not immune to it. Some engines let you apply temperature after the filter instead. Then the kept set is fixed by the distribution and temperature only reshapes the survivors.
| Rule | Keeps | Adapts to confidence? | Typical failure |
|---|---|---|---|
| Top-k | the k most likely tokens | no | k = 50 keeps 49 junk tokens on a factual step |
| Top-p (nucleus) | smallest set with mass ≥ p | partly | flat or hot distributions pull in hundreds of tail tokens |
| Min-p | tokens with p ≥ p_base · p_max | yes, linearly in p_max | very high T or tiny p_base still lets the tail in |
Where you will find it
- Hugging Face transformers
generate(..., do_sample=True, min_p=0.1); unset by default- vLLM
SamplingParams(min_p=0.1); 0.0 disables it- llama.cpp
--min-p; the sampler order is configurable with--samplers- Paper setting
- benchmarks included GPQA, GSM8K with chain of thought, and AlpacaEval creative writing, swept across temperatures; the advantage over top-p was largest at high T
Worked example
Seven candidate tokens with and . The cut-off is .
| Token | p | Cumulative p | p ≥ 0.075? | q after min-p |
|---|---|---|---|---|
| A | 0.50 | 0.50 | yes | 0.50 / 0.88 = 0.568 |
| B | 0.20 | 0.70 | yes | 0.227 |
| C | 0.10 | 0.80 | yes | 0.114 |
| D | 0.08 | 0.88 | yes | 0.091 |
| E | 0.05 | 0.93 | no | 0 |
| F | 0.04 | 0.97 | no | 0 |
| G | 0.03 | 1.00 | no | 0 |
If the leader had instead, the cut-off would fall to and many more tokens would pass. Top-p sees only the running sum; min-p sees the shape.
1import numpy as np23def min_p_filter(logits, min_p=0.1, temperature=1.0):4 """Temperature, then min-p: keep tokens with p >= min_p * p_max, renormalise."""5 z = logits / temperature6 p = np.exp(z - z.max())7 p /= p.sum()8 keep = p >= min_p * p.max()9 q = np.where(keep, p, 0.0)10 return q / q.sum()1112p = np.array([0.50, 0.20, 0.10, 0.08, 0.05, 0.04, 0.03])13q = min_p_filter(np.log(p), min_p=0.15)14print(np.round(q, 3)) # [0.568 0.227 0.114 0.091 0. 0. 0. ]1516rng = np.random.default_rng(0)17token = rng.choice(len(q), p=q) # sample from the truncated distribution1819# transformers: model.generate(**inputs, do_sample=True, temperature=1.5, min_p=0.1)Use when
- +One intuitive knob whose effect scales with the model's own confidence.
- +Keeps outputs coherent at temperatures where top-p degrades.
- +Cheap: a max, a comparison and a renormalisation; no sort needed.
Avoid when
- −Still a heuristic: the best p_base depends on model, task and temperature, so tune it on your own evaluations.
- −With temperature applied first, very high T widens the margin T·ln p_base and the tail returns.
- −Not offered by every hosted API; availability varies by provider.
Check yourself
With p_base = 0.1, the top token has p = 0.6. Which tokens are kept?
Check yourself
Temperature is applied before min-p with p_base = 0.1. How far below the top logit can a token be and still survive at T = 2?
Remember
- ✓Keep , renormalise, sample; typical is 0.05–0.1.
- ✓The cut-off scales with confidence: one survivor on a peaked step, many on a flat step.
- ✓In logits: keep ; temperature-first widens the margin with T.
- ✓Its main use is high-temperature creative sampling, where top-p admits hundreds of tail tokens.
- ✓Disable leftover top-k/top-p and check the sampler order when you adopt it.