Zero-shot prompting

Zero-shot prompting states the task in words with no examples; read the answer by scoring the label words, ideally minus their content-free prior (PMI), not by parsing free text. GPT-3 175B scored 76.2% zero-shot on LAMBADA.

7 min read990 wordscore0 viewsupdated 2026-10-21

Definition

Zero-shot prompting is asking a pretrained language model to do a task from a natural-language description alone, with no worked examples in the prompt and no change to the weights. Radford et al. ("Language Models are Unsupervised Multitask Learners", 2019, GPT-2) showed zero-shot task transfer from pretraining alone. Brown et al. ("Language Models are Few-Shot Learners", 2020, GPT-3) fixed the terms: zero-shot gives only an instruction, one-shot adds one demonstration, few-shot adds several.

It is the default way people use a chat assistant, and the first baseline for any new task. It is also the cheapest: the prompt is short, there is nothing to collect or label, and changing the task means editing a sentence. The cost is control. The model must infer the task, the label set and the output format from words alone. When any of these is ambiguous, accuracy drops, and the output may not parse.

A zero-shot prompt has three parts. The instruction states the task and the allowed outputs. The input is the item to process. The output cue is the text right before the answer, such as Sentiment:, A: or TL;DR:, and it tells the model what kind of token comes next. GPT-2 summarised articles zero-shot just by appending TL;DR:. In a chat model the instruction usually goes in the system message (see System prompts and role prompting), and the cue becomes an explicit format rule.

zero-shot prompt: the task is described, k = 0 demonstrationsinstructionClassify the sentiment of the review.Answer with one word: positive or negative.inputReview: "Watchable, but forgettable."output cueSentiment:LMfrozenp(next token)positive0.42negative0.18The0.16Mixed0.10(other)0.14three ways to turn the distribution into a label1 free generationdecode, then parse the textgreedy → "positive" (0.42)may also emit "Mixed" ora sentence: parser must cope2 label scoringkeep only the label words0.42 : 0.18→ 0.700 / 0.300picks positive3 PMI / calibrated÷ prior p(y | cue, no review)0.700/0.80 = 0.8750.300/0.20 = 1.500→ 0.368 / 0.632: negativethe cue picks the task: "TL;DR:" → summary (GPT-2) · "French:" → translation"A:" → answer · "Sentiment:" → label · prior 0.80/0.20 = same prompt, review removed
fig.A zero-shot prompt: instruction, input and output cue, with k = 0 demonstrations, fed to a frozen model. The next-token distribution (illustrative) can be read three ways. Free generation decodes and parses text. Label scoring keeps only the label words: 0.42 : 0.18 renormalises to 0.700 / 0.300, so positive. PMI scoring divides by the label prior from the same prompt with the review removed (0.80 / 0.20): 0.875 against 1.500, so negative.

Reading the answer

For open-ended tasks (summaries, rewrites, answers) you decode text and use it. For classification there is a more reliable option: never let the model choose freely. Score each allowed label instead. A verbalizer vv maps each class to a label string, such as positive → " positive". The prediction is the class whose string is most likely after the prompt:

y^=arg⁡max⁡y∈Y  ∑t=1∣v(y)∣log⁡Pθ(v(y)t∣T(x), v(y)<t)\hat y=\arg\max_{y\in\mathcal Y}\;\sum_{t=1}^{|v(y)|}\log P_\theta\big(v(y)_t\mid T(x),\,v(y)_{<t}\big)
T(x)T(x)
the prompt template filled with input x (instruction, input, cue)
v(y)v(y)
the token sequence of the label for class y; one token in the simple case
Y\mathcal Y
the closed label set; nothing outside it can be predicted

This removes format failures, because the answer is always a valid label. It does not remove bias. Some label words are simply more frequent, so the model prefers them whatever the input says. Labels of several tokens also compete unequally: a summed log-probability penalises longer strings, which is why some evaluations divide by the token count.

Holtzman et al. ("Surface Form Competition: Why the Highest Probability Answer Isn't Always Right", EMNLP 2021) proposed scoring by how much the input raises each label's probability. Their domain-conditional PMI divides by the label's probability under a prompt that keeps the task wording but drops the input:

y^=arg⁡max⁡y  [log⁡Pθ(v(y)∣T(x))−log⁡Pθ(v(y)∣T(∅))]\hat y=\arg\max_{y}\;\Big[\log P_\theta\big(v(y)\mid T(x)\big)-\log P_\theta\big(v(y)\mid T(\varnothing)\big)\Big]
T(∅)T(\varnothing)
the same template with the input removed or replaced by a domain premise such as the bare cue
log⁡P(v(y)∣T(∅))\log P(v(y)\mid T(\varnothing))
the label's prior: how much the model likes that word in this context regardless of input

The same idea, applied to the demonstrations in a few-shot prompt, is contextual calibration (see Few-shot prompting). Both cost one extra forward pass per label, and the prior can be cached because it does not depend on the input.

Why zero-shot works at all

A base model is trained only to continue text (see Language modeling), so a zero-shot prompt works only if the pretraining data contained text shaped like "task description, then answer". For base models that is weak. GPT-3's zero-shot scores trailed its few-shot scores on most tasks, and the authors noted that zero-shot is sometimes hard even for humans to interpret without examples. Instruction tuning fixed most of this by training on many tasks phrased as instructions. RLHF then tuned models to follow instructions the way people want. Today's chat models are zero-shot learners by construction.

Reference results

GPT-3 175B, LAMBADA
76.2% zero-shot, 86.4% few-shot (Brown et al., 2020)
GPT-3 175B, TriviaQA
64.3% zero-shot, 68.0% one-shot, 71.2% few-shot, closed book
FLAN 137B (Wei et al., 2022)
instruction-tuned; zero-shot FLAN beat zero-shot GPT-3 175B on 20 of 25 datasets
InstructGPT (Ouyang et al., 2022)
labellers preferred outputs of the 1.3B instruction-following model over those of 175B GPT-3
Prompt format sensitivity
Sclar et al. (2024) found accuracy spreads of up to 76 points across meaning-preserving format changes (spacing, separators, casing) for LLaMA-2-13B

Try it: template, label words and readout

prompt for review #15 (true label: terrible), no examples"Watchable, but forgettable."Is this review great or terrible? ▍next-token probabilities (full vocabulary)great0.271terrible0.113The0.345I0.125This0.077It0.068readout for #15free: greedy token "The"not a label: parse failslabel: p(great) 0.707 · p(terrible) 0.293→ great; no-review prior 0.818/0.182PMI: great −0.145 · terrible +0.474log p(y|x) − log p(y|∅) picks terribleall 40 reviews (even # positive, odd # negative)free generationargmax over all tokens22/40label scoringargmax over 2 label words32/40PMI scoringminus content-free prior39/40■ correct · ■ wrong label · ■ not a label word (click a cell to select)valid format 23/40 · "great" predicted for 28/40 by label scoring, 21/40 by PMI (truth 20/40)toy model: label logit = ±c·s + b(word) + noise; the template sets c and the filler-token level
template
label words
fig.A toy model classifies 40 reviews (20 positive) zero-shot. Label logits are ±c·s plus a per-word bias; the template sets c and how likely filler tokens are. Default (question template, great / terrible): free generation gets 22/40 and produces a label at all for only 23; label scoring gets 32/40 but says "great" for 28; PMI scoring gets 39/40. Review #15 ("Watchable, but forgettable.", negative): the greedy token is "The" (0.345); the label pair is 0.707 / 0.293 for great, while PMI gives great −0.145 and terrible +0.474. The instruction template with positive / negative gives 39, 39, 40 and 40/40 valid. The bare template with positive / negative produces no valid label at all (0/40), yet label scoring still gets 34/40 and PMI 37/40.

The three readouts fail in different ways. Free generation fails on format: the model may say something true in a form your parser rejects. Label scoring fixes the format but inherits the prior of the chosen words: "great" is a common, positive-leaning word, so it wins borderline cases. PMI subtracts that prior. A clear instruction with an explicit cue fixes most of the format problem by itself, which is why production prompts always say exactly what to output.

Worked example: PMI flips a borderline call

SteppositivenegativeDecision
P(label | full prompt), all tokens0.420.18—
Renormalise over the two labels0.42/0.60 = 0.7000.18/0.60 = 0.300positive
Prior P(label | no review)0.800.20—
PMI = ln(0.700/0.80), ln(0.300/0.20)ln 0.875 = −0.134ln 1.5 = +0.405negative
Normalised ratio0.875/2.375 = 0.3681.5/2.375 = 0.632negative
The diagram's numbers. Natural logs; the prior comes from the same prompt with the review removed.

The review raised "negative" from 0.20 to 0.30 and lowered "positive" from 0.80 to 0.70. Plain scoring sees only that 0.70 > 0.30. PMI sees which label the input pushed up.

Writing zero-shot prompts

  1. 1State the task and the complete label set or output format in the instruction: "Answer with one word: positive or negative."
  2. 2End with an output cue (Sentiment:, Answer:) or, in chat, a format rule; ask for JSON when a program consumes the result (see Structured output (JSON mode)).
  3. 3Separate the input from the instructions with delimiters (quotes, XML-style tags, triple backticks) so the model cannot confuse the two, and so injected text is easier to spot (see Prompt injection).
  4. 4For classification with log-probability access, score the label words instead of parsing; add the content-free prior subtraction if one label dominates.
  5. 5Use greedy decoding (temperature 0) for labels and extraction so runs are repeatable.
  6. 6Test several paraphrases of the instruction on a labelled sample of 50–200 items before trusting one; if accuracy is still short, add examples (Few-shot prompting) or reasoning (Chain-of-thought prompting).
python
1import torch
2from transformers import AutoTokenizer, AutoModelForCausalLM
3
4tok = AutoTokenizer.from_pretrained("gpt2")
5lm = AutoModelForCausalLM.from_pretrained("gpt2").eval()
6
7@torch.no_grad()
8def logprob(prompt: str, continuation: str) -> float:
9 """Sum of log p(continuation tokens | prompt)."""
10 p, c = tok(prompt).input_ids, tok(continuation).input_ids
11 logp = lm(torch.tensor([p + c])).logits.log_softmax(-1)[0, len(p) - 1 : -1]
12 return logp.gather(1, torch.tensor(c)[:, None]).sum().item()
13
14TEMPLATE = ("Classify the sentiment of the review. Answer with one word: "
15 "positive or negative.\nReview: {x}\nSentiment:")
16LABELS = [" positive", " negative"] # leading space: GPT-2 word tokens
17
18def classify(x: str, pmi: bool = True) -> str:
19 s = [logprob(TEMPLATE.format(x=x), y) for y in LABELS]
20 if pmi: # subtract the content-free prior
21 s = [a - logprob(TEMPLATE.format(x=""), y) for a, y in zip(s, LABELS)]
22 return LABELS[max(range(len(s)), key=s.__getitem__)].strip()
23
24print(classify("Watchable, but forgettable."))
Zero-shot classification by label scoring with optional PMI, using Hugging Face transformers and GPT-2 (not run here: transformers and torch were not available in this environment)

Use when

  • +No labelled data and the shortest prompt: lowest cost and latency of any prompting method.
  • +Changing the task is an edit to one sentence; no retraining.
  • +Label scoring guarantees a valid label and gives a probability you can threshold.

Avoid when

  • −The model must infer the label boundaries and format; ambiguous instructions fail silently.
  • −Sensitive to wording and label-word priors; results can swing widely between paraphrases.
  • −Weak on unusual formats and specialised tasks the model rarely saw described in words.

Check yourself

Label scoring returns "great" for 28 of 40 reviews when only 20 are positive. What does PMI scoring change?

Check yourself

A base model's zero-shot output often doesn't parse: it starts "The movie…" instead of giving a label. Which fix does not need log-probabilities?

Remember

  • ✓Zero-shot: instruction + input + output cue, no demonstrations, frozen weights (terminology from Brown et al., 2020).
  • ✓Classify by scoring the label words, arg⁡max⁡ylog⁡P(v(y)∣T(x))\arg\max_y \log P(v(y)\mid T(x)): always a valid label, but biased towards frequent words.
  • ✓PMI scoring subtracts log⁡P(v(y)∣T(∅))\log P(v(y)\mid T(\varnothing)), the label's prior with the input removed (Holtzman et al., 2021).
  • ✓Instruction tuning and RLHF are what made zero-shot reliable; base models do much better with examples.
  • ✓Prompt wording alone can move accuracy by tens of points: test paraphrases on a labelled sample and version the winning template.
← back to Generative AI roadmap