Min-p sampling

Min-p sampling (Nguyen et al., 2024) keeps only tokens with p ≥ p_base·p_max, so the cut-off scales with the model's confidence: tight when one token dominates, wide when many are plausible, and stable at high temperature.

7 min read526 wordsadvanced0 viewsupdated 2026-10-21

Definition

Min-p sampling is a truncation rule for stochastic decoding. At each step it finds the largest token probability pmax⁡p_{\max}, discards every token whose probability is below pbase⋅pmax⁡p_\text{base}\cdot p_{\max}, renormalises the survivors and samples. It was described by Nguyen, Baker, Neo, Roush, Kirsch and Shwartz-Ziv in "Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs" (arXiv, 2024), after first spreading through open-source inference engines such as llama.cpp in 2023.

Sampling from the full softmax lets the long tail of a 30,000–200,000-token vocabulary leak in. Each tail token is unlikely, but together they can hold a lot of mass, and one bad token can derail the rest of the output. Top-k and top-p sampling cut the tail with a fixed count or a fixed mass. Neither looks at how peaked the distribution is, so the same setting is too loose on a factual step and too tight on an open one.

Min-p sits in the same place in the decoder as top-p: after the logits and temperature, before the random draw. It is a drop-in replacement or a companion filter. Raising temperature is how you buy diversity (see Temperature sampling); min-p is meant to keep that diversity among plausible tokens.

The rule

p=softmax⁡(z/T),Vmin={ i:pi≥pbase⋅max⁡jpj },qi=pi 1[i∈Vmin]∑j∈Vminpjp=\operatorname{softmax}(z/T),\qquad \mathcal V_\text{min}=\{\,i: p_i\ge p_\text{base}\cdot\max_j p_j\,\},\qquad q_i=\frac{p_i\,\mathbb 1[i\in\mathcal V_\text{min}]}{\sum_{j\in\mathcal V_\text{min}}p_j}
zz
next-token logits over the vocabulary
TT
temperature, applied before the filter in most engines
pbasep_\text{base}
the user's min-p setting, typically 0.05–0.1; 0 disables the filter
qq
the distribution actually sampled from

The top token is always kept, so pbase=1p_\text{base}=1 (with no ties) is greedy decoding. The size of the kept set is not fixed: it is every token within a constant probability ratio of the leader. That ratio is easier to read in logit space.

pipmax⁡=e(zi−zmax⁡)/T≥pbase  ⟺  zi≥zmax⁡+Tln⁡pbase\frac{p_i}{p_{\max}}=e^{(z_i-z_{\max})/T}\ge p_\text{base}\iff z_i\ge z_{\max}+T\ln p_\text{base}
Tln⁡pbaseT\ln p_\text{base}
the logit margin below the leader; ln⁡0.1=−2.30\ln 0.1=-2.30, so at T=1T=1 a token survives if it is within 2.30 logits of the top

When temperature comes first, the margin grows linearly with TT: at T=3T=3 and pbase=0.1p_\text{base}=0.1 it is 6.91 logits. Min-p is far more robust to temperature than top-p, but not immune to it. Some engines let you apply temperature after the filter instead. Then the kept set is fixed by the T=1T=1 distribution and temperature only reshapes the survivors.

logits zfrom the modelp = softmax(z/T)temperaturep_max = max pmodel confidencep ≥ p_base·p_maxkeep; p_base = 0.1renormalisethen sampleconfident: “The capital of France is”p_max = 0.8833 · cut-off 0.0883 → keeps 1 tokenParis0.8833the0.0267a0.0162located0.0108Lyon0.0036not0.0033known0.0027one0.0022also0.0018home0.0013tail Σ0.0481400 tail tokens, each ≤ 0.0005: all droppedopen: “Name a colour:”p_max = 0.1681 · cut-off 0.0168 → keeps 10 tokensblue0.1681red0.1521green0.1245purple0.0923yellow0.0835orange0.0618teal0.0415black0.0339white0.0307pink0.0278tail Σ0.1838400 tail tokens, each ≤ 0.0019: all droppeddashed line = p_base·p_max, always 10% of the top bar: the cut-off scales with confidencesame p_base: min-p keeps 1 and 10 tokens; top-p 0.9 keeps 2 and 70 (60 from the tail)
fig.The same p_base = 0.1 on two toy next-token distributions at T = 1. Confident step: p_max = 0.883, cut-off 0.088, only Paris survives. Open step: p_max = 0.168, cut-off 0.017, all ten colours survive. In both panels the cut-off sits at 10% of the top bar, and every one of the 400 tail tokens falls below it. Top-p 0.9 would keep 2 and 70 tokens, 60 of the 70 from the tail.
RuleKeepsAdapts to confidence?Typical failure
Top-kthe k most likely tokensnok = 50 keeps 49 junk tokens on a factual step
Top-p (nucleus)smallest set with mass ≥ ppartlyflat or hot distributions pull in hundreds of tail tokens
Min-ptokens with p ≥ p_base · p_maxyes, linearly in p_maxvery high T or tiny p_base still lets the tail in
Min-p compared with the other truncation rules (k, p and p_base are the user settings)

Where you will find it

Hugging Face transformers
generate(..., do_sample=True, min_p=0.1); unset by default
vLLM
SamplingParams(min_p=0.1); 0.0 disables it
llama.cpp
--min-p; the sampler order is configurable with --samplers
Paper setting
benchmarks included GPQA, GSM8K with chain of thought, and AlpacaEval creative writing, swept across temperatures; the advantage over top-p was largest at high T
“Name a colour:” · T = 1.5 · vocabulary: 10 named + 400 tail tokenssoftmax(z/T), untruncatedmin-p 0.10top-p 0.90both renormalised0.000.300.60blueredgreenpurpleyelloworangetealblackwhitepinktail×400p_base·p_max = 0.0078min-p keeps 10 tokens · tail mass 0.0% · P(top) 0.169top-p keeps 223 tokens · tail mass 48.9% · P(top) 0.086untruncated: p_max 0.0776 · tail mass 54.0%min-p after T keeps z ≥ z_max + T·ln p_base = 5.0 − 3.45 = 1.55 (tail logits ≤ 0.5)top-p must add tail tokens to reach its mass target; min-p does not
context
order
fig.Ten named tokens plus a 400-token tail with logits 0.5 to −3.5. Defaults (open colour prompt, T = 1.5, p_base 0.1, top-p 0.9): untruncated, the tail holds 54.0% of the mass; min-p keeps the 10 colours (tail 0%, P(blue) = 0.169), top-p keeps 223 tokens and 48.9% tail mass. Raise T to 3: min-p keeps 251 tokens with 83.1% tail mass (p_base 0.2 cuts that to 43 tokens, 48.1%). Switch the order to mask-then-temperature and it keeps 10 again. On the confident prompt at T = 1.5, min-p keeps 1 token and top-p 193.

Worked example

Seven candidate tokens with p=(0.50,0.20,0.10,0.08,0.05,0.04,0.03)p=(0.50,0.20,0.10,0.08,0.05,0.04,0.03) and pbase=0.15p_\text{base}=0.15. The cut-off is 0.15×0.50=0.0750.15\times0.50=0.075.

TokenpCumulative pp ≥ 0.075?q after min-p
A0.500.50yes0.50 / 0.88 = 0.568
B0.200.70yes0.227
C0.100.80yes0.114
D0.080.88yes0.091
E0.050.93no0
F0.040.97no0
G0.031.00no0
Min-p 0.15 keeps four tokens (mass 0.88) and renormalises; top-p 0.9 would keep five, because the cumulative mass first reaches 0.9 at the fifth token (0.93)

If the leader had pmax⁡=0.12p_{\max}=0.12 instead, the cut-off would fall to 0.0180.018 and many more tokens would pass. Top-p sees only the running sum; min-p sees the shape.

python
1import numpy as np
2
3def min_p_filter(logits, min_p=0.1, temperature=1.0):
4 """Temperature, then min-p: keep tokens with p >= min_p * p_max, renormalise."""
5 z = logits / temperature
6 p = np.exp(z - z.max())
7 p /= p.sum()
8 keep = p >= min_p * p.max()
9 q = np.where(keep, p, 0.0)
10 return q / q.sum()
11
12p = np.array([0.50, 0.20, 0.10, 0.08, 0.05, 0.04, 0.03])
13q = min_p_filter(np.log(p), min_p=0.15)
14print(np.round(q, 3)) # [0.568 0.227 0.114 0.091 0. 0. 0. ]
15
16rng = np.random.default_rng(0)
17token = rng.choice(len(q), p=q) # sample from the truncated distribution
18
19# transformers: model.generate(**inputs, do_sample=True, temperature=1.5, min_p=0.1)
Min-p in numpy (runs as is). In Hugging Face transformers the same filter is the min_p argument of generate (not run here).

Use when

  • +One intuitive knob whose effect scales with the model's own confidence.
  • +Keeps outputs coherent at temperatures where top-p degrades.
  • +Cheap: a max, a comparison and a renormalisation; no sort needed.

Avoid when

  • −Still a heuristic: the best p_base depends on model, task and temperature, so tune it on your own evaluations.
  • −With temperature applied first, very high T widens the margin T·ln p_base and the tail returns.
  • −Not offered by every hosted API; availability varies by provider.

Check yourself

With p_base = 0.1, the top token has p = 0.6. Which tokens are kept?

Check yourself

Temperature is applied before min-p with p_base = 0.1. How far below the top logit can a token be and still survive at T = 2?

Remember

  • ✓Keep pi≥pbase⋅pmax⁡p_i\ge p_\text{base}\cdot p_{\max}, renormalise, sample; typical pbasep_\text{base} is 0.05–0.1.
  • ✓The cut-off scales with confidence: one survivor on a peaked step, many on a flat step.
  • ✓In logits: keep zi≥zmax⁡+Tln⁡pbasez_i\ge z_{\max}+T\ln p_\text{base}; temperature-first widens the margin with T.
  • ✓Its main use is high-temperature creative sampling, where top-p admits hundreds of tail tokens.
  • ✓Disable leftover top-k/top-p and check the sampler order when you adopt it.
← back to Generative AI roadmap