Definition
Prompt tuning is the parameter-efficient adaptation method of Brian Lester, Rami Al-Rfou and Noah Constant, "The Power of Scale for Parameter-Efficient Prompt Tuning" (EMNLP 2021). Every weight of a pretrained language model stays frozen. For each task you learn a soft prompt: a matrix of free vectors that live in the model's -dimensional embedding space but are not tied to any vocabulary token. is concatenated in front of the embedded input, , and trained by backpropagation through the frozen network to maximise . A task is then just a file of numbers.
GPT-3 (Brown et al., 2020) showed that a big enough model can do a task from a text prompt alone: an instruction plus a few examples, and no weight updates. This prompt design has two problems. Results depend heavily on the exact wording. And on benchmarks like SuperGLUE, few-shot GPT-3 stayed far below fine-tuned models that were much smaller. Model tuning (full fine-tuning) gives the best quality, but every task then needs its own copy of every weight, which is 11B parameters per task for T5-XXL.
Searching for better discrete prompts is the obvious next step. AutoPrompt (Shin et al., 2020) used gradients to pick trigger tokens. But a discrete prompt can only pick points that are rows of the embedding matrix, and the search over token sequences is combinatorial. Prompt tuning relaxes the constraint: let the prompt be any point in embedding space and optimise it with gradient descent, exactly like a weight. Two concurrent works took similar routes. Prefix tuning (Li & Liang, 2021) learns vectors at every layer. P-Tuning (Liu et al., "GPT Understands, Too", 2021) learns input-layer prompts produced by a small prompt encoder. Lester et al. stripped the idea down to its simplest form, prompts at the input only with no reparameterisation, and studied how it behaves with scale.
The mechanism
- the frozen embedding rows of the input tokens
- the soft prompt, "virtual tokens"; the only trainable tensor
- concatenation along the sequence axis: the encoder sees positions
- the target as text, e.g. the word "positive"; T5 generates it with its decoder, so there is no new classification head
- all model weights, including the embedding table; gradients flow through them into but they are never updated
From the model's point of view, nothing changes. Attention, feed-forward layers and positional handling treat the prompt positions exactly like token positions. They just carry vectors that no token maps to. Every input position can attend to the prompt at every layer, so the prompt can steer the whole computation even though it is injected only once, at the bottom. The gradient is the ordinary backpropagated gradient with respect to the input embeddings at those positions:
- the layer-0 input at prompt position
- activations are still stored for the backward pass, but optimiser state exists only for numbers
What the paper found
Scale closes the gap. Lester et al. prompt-tuned T5 v1.1 at five sizes, Small to XXL (about 11B parameters), on SuperGLUE. At small sizes, prompt tuning trails model tuning by a wide margin. The gap shrinks as the model grows, and at XXL prompt tuning matches the strong multi-task model-tuning baseline. At that size it tunes less than 0.01% of the parameters. Few-shot prompt design with GPT-3 stayed far below both curves.
The checkpoint matters. Off-the-shelf T5 is pretrained with span corruption. It has only ever produced outputs like "<X> missing span <Y>", with sentinel tokens, and it has never written natural text. A frozen model like that is hard to steer into emitting a fluent label, and prompt tuning on it was poor and unstable. The fix was LM adaptation: continue training T5 for up to 100K steps with a plain prefix language-modelling objective, then freeze it. The authors released these "LM-adapted" T5 checkpoints, and all main results use them.
Initialisation and length matter less with scale. The paper compares three initialisations: random uniform in , embeddings of tokens sampled from the 5,000 most common vocabulary entries, and class-label initialisation. The class-label method starts each prompt vector at the embedding of a label word ("positive", "entailment", …) and fills any remaining slots with sampled vocabulary. Class labels worked best, and random worst, but the differences mostly vanished at XXL. Prompt lengths of 1, 5, 20, 100 and 150 were tested. Going beyond about 20 tokens gave small gains, and the largest model did well even with very short prompts.
Robustness and ensembling. Because the model cannot change, prompt tuning overfits less to the training domain. Prompts trained on SQuAD transferred to out-of-domain question-answering sets better than model tuning on several of them, with a similar effect for paraphrase detection moved between datasets. Several prompts trained for the same task can be run in one batch on one frozen model. This gives a cheap prompt ensemble that beat the average single prompt.
Interpretability is limited. Learned prompt vectors are not near any single token. Their nearest vocabulary neighbours often form tight semantic clusters, and with class-label initialisation the label words tend to remain nearest neighbours of some prompt tokens. Still, the prompt cannot be read back as text.
Setup in Lester et al. (2021)
- Backbone
- T5 v1.1, LM-adapted (extra 100K steps of prefix LM), frozen. Sizes Small, Base, Large, XL, XXL; . XXL: 24 encoder + 24 decoder layers, about 11B parameters
- Prompt
- default length , class-label initialisation; lengths 1, 5, 20, 100 and 150 in the ablation
- Trainable
- : 409,600 numbers at on XXL, which is less than 0.01% of the model
- Optimisation
- Adafactor, learning rate 0.3, batch size 32, 30,000 steps, checkpoint chosen by early stopping on the dev set
- Output
- labels written as text tokens and generated by the decoder (T5's text-to-text format)
- Benchmarks
- SuperGLUE (main results); domain transfer from SQuAD to MRQA out-of-domain sets, and between paraphrase datasets
Prompt tuning, prefix tuning, P-Tuning and discrete prompts
| method | what is learned | where it enters | per-task size |
|---|---|---|---|
| discrete prompt (GPT-3 style, AutoPrompt) | a sequence of real tokens | input, as vocabulary rows | k token ids |
| Prompt tuning (Lester et al. 2021) | k free vectors | input embeddings only | |
| P-Tuning (Liu et al. 2021) | input vectors produced by a BiLSTM+MLP prompt encoder; can be mixed with anchor tokens | input embeddings only | at inference |
| Prefix tuning (Li & Liang 2021) | k key/value vectors per layer, trained through an MLP reparameterisation | every attention layer | about |
| P-Tuning v2 (Liu et al. 2022) | deep prompts at every layer, no reparameterisation needed | every layer | about |
The key difference from Prefix tuning is depth. A prefix supplies keys and values directly at every layer, so it can shape every attention map. A soft prompt is injected once, and its influence on deeper layers has to pass through the frozen network. That makes prompt tuning about times smaller, but also weaker on small models. P-Tuning v2 (Liu et al., ACL 2022) showed that adding prompts at every layer recovers fine-tuning quality on models of a few hundred million parameters and on hard sequence-labelling tasks. In other words, the deep form is what you need when you cannot rely on scale.
Compared with discrete prompting, the soft prompt is a strict relaxation. Any discrete prompt is a point in the soft search space, since each token is a row of . Gradient descent can therefore only do at least as well at the global optimum, though it can stop in a worse local one. You give up readability, and you give up the ability to paste the prompt into another model: the vectors only mean something to the exact checkpoint they were trained on.
Worked example: sizes, and one forward pass
| model | d_model | k = 1 | k = 20 | k = 100 | k = 150 |
|---|---|---|---|---|---|
| Small | 512 | 512 | 10,240 | 51,200 | 76,800 |
| Base | 768 | 768 | 15,360 | 76,800 | 115,200 |
| Large | 1024 | 1,024 | 20,480 | 102,400 | 153,600 |
| XL | 2048 | 2,048 | 40,960 | 204,800 | 307,200 |
| XXL | 4096 | 4,096 | 81,920 | 409,600 | 614,400 |
At on XXL the prompt is 409,600 numbers, or 1.64 MB in fp32. One bf16 copy of an 11B model is about 22 GB. That is a ratio of roughly 13,000 to 1 per task, and a fraction of the parameters.
The toy model of the figure on input x = (0.086, −0.871), class-label init P₁ = e(positive) = (0.6, 0.2), P₂ = e(negative) = (−0.2, −0.25)
- 1Attention logits : . Softmax gives , so the input reads .
- 2Hidden layer: and , so .
- 3Verbalizer: , logit , .
- 4The teacher prompt gives on this input. Cross-entropy , against the teacher's entropy floor of : excess on this example.
- 5Backward: . This pushes up, the error flows through and into , and passes it to with weight . also gets a share through the attention logits. No other number in the model changes.
1import torch, torch.nn as nn2from transformers import AutoTokenizer, T5ForConditionalGeneration3from transformers.optimization import Adafactor45name = "google/t5-base-lm-adapt" # LM-adapted T5 v1.1, as in the paper6tok = AutoTokenizer.from_pretrained(name)7model = T5ForConditionalGeneration.from_pretrained(name)8model.requires_grad_(False) # freeze every weight, including embeddings9emb = model.get_input_embeddings() # nn.Embedding(32128, d_model)1011class SoftPrompt(nn.Module):12 def __init__(self, k, emb, init_words, tok):13 super().__init__()14 ids = [i for w in init_words for i in tok(w, add_special_tokens=False).input_ids]15 common = torch.randint(0, 5000, (k,)) # stand-in for "sampled from the 5,000 most common tokens"16 common[: min(k, len(ids))] = torch.tensor(ids[:k]) # class-label init first17 self.P = nn.Parameter(emb.weight[common].detach().clone()) # (k, d)18 def forward(self, input_ids, attention_mask):19 x = emb(input_ids) # (B, n, d), frozen20 P = self.P.unsqueeze(0).expand(x.size(0), -1, -1) # (B, k, d), trainable21 mask = torch.cat([attention_mask.new_ones(x.size(0), P.size(1)), attention_mask], 1)22 return torch.cat([P, x], 1), mask2324prompt = SoftPrompt(k=100, emb=emb, init_words=["positive", "negative"], tok=tok)25opt = Adafactor(prompt.parameters(), lr=0.3, scale_parameter=False, relative_step=False, warmup_init=False)2627batch = tok(["sst2 sentence: the film was great", "sst2 sentence: a dull, lifeless plot"],28 return_tensors="pt", padding=True)29labels = tok(["positive", "negative"], return_tensors="pt", padding=True).input_ids30labels[labels == tok.pad_token_id] = -1003132embeds, mask = prompt(batch.input_ids, batch.attention_mask)33loss = model(inputs_embeds=embeds, attention_mask=mask, labels=labels).loss34loss.backward() # gradients reach prompt.P only35opt.step(); opt.zero_grad()3637print(sum(p.numel() for p in prompt.parameters())) # 100 * 768 = 76,800 trainable for T5-Base38# Same thing with the PEFT library:39# from peft import PromptTuningConfig, PromptTuningInit, TaskType, get_peft_model40# cfg = PromptTuningConfig(task_type=TaskType.SEQ_2_SEQ_LM, num_virtual_tokens=100,41# prompt_tuning_init=PromptTuningInit.TEXT,42# prompt_tuning_init_text="positive negative", tokenizer_name_or_path=name)43# model = get_peft_model(model, cfg)Use when
- +Tiny per-task state: numbers, 409,600 at on T5-XXL
- +One frozen model serves many tasks, even mixed in one batch
- +Matches full fine-tuning on SuperGLUE once the model reaches about 11B parameters
- +Less overfitting to the training domain; cheap prompt ensembles
Avoid when
- −Clearly worse than fine-tuning on small and mid-size models
- −Sensitive to the checkpoint (needs LM adaptation for T5) and to initialisation at small scale
- −Adds positions to every sequence, which costs context length and compute
- −Prompts are not human-readable and do not transfer between models
Check yourself
Which statement best distinguishes prompt tuning from prefix tuning?
Check yourself
In Lester et al., what happened to the gap between prompt tuning and full model tuning on SuperGLUE as T5 grew from Small to XXL?
Remember
- ✓Prompt tuning learns , prepended to the frozen input embeddings: with frozen.
- ✓Lester et al. (EMNLP 2021) used LM-adapted T5 v1.1 with , class-label init, Adafactor at lr 0.3. At XXL (11B, ) it matched model tuning while training 409,600 numbers.
- ✓Small models lag clearly. Initialisation (class labels > sampled vocab > random) and prompt length matter less as the model grows.
- ✓Prefix tuning and P-Tuning v2 put vectors at every layer and work at smaller scale. P-Tuning uses an LSTM prompt encoder at the input. Discrete prompts are the special case where every vector is a vocabulary row.
- ✓Soft prompts are a relaxation of discrete prompts. They are not readable, they are tied to one checkpoint, and they cost k extra positions per sequence.