# Instruction tuning

> Instruction tuning fine-tunes an LM on many tasks phrased as natural-language instructions, so it follows instructions for unseen tasks zero-shot. FLAN (137B, 62 datasets, 12 clusters) beat zero-shot GPT-3 on 20 of 25 tasks, and Flan-PaLM scaled this to 1,836 tasks.

Source: https://www.mlfoundry.in/learn/instruction-tuning  
Section: Deep Learning › Large Language Models (LLMs)  
Level: core · 15 min read · updated 2026-10-13

**Instruction tuning:** is multi-task fine-tuning in which every training example is written as a natural-language instruction plus input, and the target is the answer. The aim is generalisation to **tasks never seen in training**, specified only by their instruction. It was introduced by Wei et al. ("Finetuned Language Models Are Zero-Shot Learners", ICLR 2022), whose 137B model **FLAN** was tuned on 62 datasets rendered with templates. Sanh et al. (T0, "Multitask Prompted Training Enables Zero-Shot Task Generalization", ICLR 2022) did the same with an encoder–decoder. Chung et al. ("Scaling Instruction-Finetuned Language Models", 2022) scaled it to 1,836 tasks in Flan-T5 and Flan-PaLM.

GPT-3 showed that a large LM can do a task from a few examples in its prompt. Zero-shot, with only a description, it was much weaker. Wei et al. argued that one reason is that a pretraining corpus rarely contains text shaped like "here is an instruction, now do it". The fix is to teach that format directly. Take existing supervised NLP datasets, such as NLI, sentiment, QA, translation and summarisation, and rewrite each example as an instruction. Fine-tune on the mixture, then ask the model to do something it was never trained on.

Several earlier lines of work point the same way. T5 (Raffel et al., 2020) cast every task as text-to-text with a task prefix, but evaluated on the tasks it trained on. Natural Instructions (Mishra et al., 2022) collected 61 tasks with full human-written instructions, and Super-NaturalInstructions (Wang et al., 2022) grew it to 1,616 tasks of 76 types. Instruction tuning is the bridge between this multi-task NLP tradition and the chat-oriented **Supervised fine-tuning (SFT)** that followed. The objective is the same, a masked next-token loss on the target. What differs is the data: templated academic tasks, rather than open-ended demonstrations.

## Templates and held-out clusters

FLAN wrote **ten templates per dataset**. Up to three of them "turn the task around", for example asking the model to write a hypothesis for a given label rather than classify it. Classification templates list the answer options, so the model sees which strings are valid outputs. Each example is rendered with a randomly chosen template, so the model meets many phrasings of the same task.

The evaluation protocol is what makes "zero-shot" meaningful. The 62 datasets are grouped into **12 task clusters**. To score the model on a cluster, say NLI, a separate model is tuned on all the other clusters. No NLI dataset in any phrasing is seen in training. So the score measures transfer to a new *kind* of task, not to a new dataset of a familiar kind.

> Figure: FLAN's protocol. The 62 datasets fall into 12 task clusters. One cluster is held out (click to change it), and the model is tuned on the rest: each example is rendered with one of 10 templates and drawn from a capped mixture. It is then evaluated zero-shot on the held-out cluster. Below: instruction tuning against the two earlier paradigms. Pretrain-then-fine-tune needs one model per task. Prompting needs no training but relies on few-shot exemplars. Instruction tuning trains on many tasks and infers on an unseen one.

## Objective and mixture

$$
\mathcal L(\theta)=-\sum_{m}r_m\;\mathbb E_{(x,y)\sim D_m}\;\mathbb E_{\tau\sim\mathcal T_m}\;\log p_\theta\big(y\mid\tau(x)\big)
$$

- $D_m$: dataset m in the mixture, from one of the training clusters
- $\mathcal T_m$: templates for dataset $m$; $\tau(x)$ is the instruction-formatted input
- $y$: target text (a label word, an answer or a summary); loss only on $y$, as in **Supervised fine-tuning (SFT)**
- $r_m$: mixing rate: the probability that a training example comes from dataset m

$$
r_m=\frac{\min(e_m,K)^{1/T}}{\sum_n\min(e_n,K)^{1/T}}\qquad\text{epochs over }D_m=\frac{r_m N}{e_m}
$$

- $e_m$: number of training examples in dataset m
- $K$: cap: datasets larger than $K$ count as size $K$ (Raffel et al.'s "examples-proportional mixing")
- $T$: temperature: $T=1$ is proportional, and larger $T$ flattens toward uniform
- $N$: total training examples drawn (steps × batch)

The mixture matters because dataset sizes span three orders of magnitude or more. SNLI has 550,152 training pairs and CB has 250. Mixing by raw size gives SNLI 37% of the training signal in the figure below and CB 0.02%. The cap evens this out. The price is that small datasets are repeated more often, and repetition beyond a few epochs means memorisation. FLAN capped each dataset at 30k examples and used examples-proportional mixing.

**The main instruction-tuned models**

- **FLAN (2022):** LaMDA-PT 137B decoder; 62 datasets, 12 clusters, 10 templates each; 30k steps, batch 8,192 tokens, Adafactor, lr $3\cdot10^{-5}$; input 1,024 and target 256 tokens. Zero-shot FLAN beat zero-shot GPT-3 175B on 20 of 25 datasets, and beat few-shot GPT-3 by a large margin on ANLI, RTE, BoolQ, AI2-ARC, OpenbookQA and StoryCloze
- **T0 (2022):** LM-adapted T5 11B; prompts from the PromptSource collection (P3); held-out task types NLI, coreference, sentence completion and word-sense disambiguation; often beat models up to 16 times its size zero-shot
- **Super-NaturalInstructions (2022):** 1,616 tasks of 76 types with expert-written instructions; Tk-Instruct (T5) beat InstructGPT on the held-out tasks despite being much smaller
- **Flan-T5 / Flan-PaLM (2022):** 1,836 tasks from four mixtures (Muffin, T0-SF, NIV2 and chain-of-thought); T5 80M to 11B and PaLM 8B to 540B. Flan-PaLM 540B scored 75.2% on five-shot MMLU, using about 0.2% of the pretraining compute
- **Self-Instruct (2023):** 175 human-written seed tasks; GPT-3 generated 52K instructions with 82K instances, filtered by ROUGE-L < 0.7 against the existing pool; +33% absolute over vanilla GPT-3 on Super-NaturalInstructions, close to InstructGPT-001
- **Alpaca (2023):** LLaMA 7B on 52K instruction–response pairs generated by text-davinci-003 with a Self-Instruct pipeline; under $600 in total, 3 hours on eight 80 GB A100s

> Figure: Nine real datasets with their official training-set sizes (1,474,215 examples in total), mixed by r_m ∝ min(e_m, K)^(1/T). The right column is how many epochs of each dataset a budget of N training examples covers. At the default (K = 3,000, T = 1, N = 100,000), the six datasets above the cap each get 14.19%, RTE 11.78%, COPA 1.89% and CB 1.18%. SNLI is then seen for only 0.026 of an epoch, while RTE, COPA and CB are all repeated 4.73 times. NLI takes 41.3% of the mixture. With no cap, every example is equally likely: each dataset gets the same 0.068 epochs, SNLI alone takes 37.32% and NLI takes 64.1%, but CB is 0.02% and commonsense 0.0%. A cap of 250 makes the mixture uniform (11.11% each) and repeats CB 44.4 times. Raising the temperature to 8 at K = 3,000 lifts CB to 8.64% and 34.5 epochs. With K = 3,000 and N = 1,000,000, BoolQ is repeated 15.1 times and RTE, COPA and CB 47.3 times: a recipe for memorising small datasets. Below the mixture, one RTE-style example is rendered with five FLAN-style templates: two zero-shot phrasings with an options suffix, few-shot, chain of thought, and an inverted task that generates the hypothesis from the label.

## What scales: tasks, model size, templates

**More task clusters help.** In FLAN's ablation, adding clusters to the training mix steadily raised average zero-shot accuracy on the held-out clusters, with no sign of saturation at the largest mixture tried. Chung et al. later scaled the number of tasks from 9 to 89, 282, 682 and 1,836. The held-out gains kept coming but flattened: most of the improvement arrived by 282 tasks. Super-NaturalInstructions found that performance grows roughly log-linearly with the number of training tasks, while more instances per task saturate quickly. Diversity of tasks matters more than examples per task.

**Scale is a precondition.** FLAN tuned models of 422M, 2B, 8B, 68B and 137B parameters. At 68B and 137B, instruction tuning improved held-out tasks substantially. At 8B and below, it **hurt** held-out performance. The authors' reading is that a small model spends its capacity fitting the training tasks, while a large one can also learn the general skill of following instructions. Later work with better mixtures, such as Flan-T5, brought gains down to small models, so this threshold depends on the recipe.

**Instructions matter.** FLAN compared training with natural-language templates against training with only the dataset name, or with no instruction. Both alternatives did clearly worse on held-out clusters, so the gain is not just multi-task learning. T0 found that more templates per dataset improved median held-out accuracy and reduced the variance across prompts. The Flan Collection (Longpre et al., ICML 2023) showed that **mixing zero-shot, few-shot and chain-of-thought templates** helps in every evaluation setting. Training on a mix of zero-shot and few-shot prompts improved performance in both settings. Chung et al. found that leaving out the nine chain-of-thought datasets degraded reasoning ability.

## Synthetic instructions: Self-Instruct and Alpaca

Templated academic datasets cover classification and short answers well, but not the open-ended requests users make. Writing thousands of new tasks by hand is expensive. **Self-Instruct** (Wang et al., ACL 2023) bootstraps them. It starts from 175 seed tasks and prompts GPT-3 with a few of them to propose new instructions. GPT-3 then generates inputs and outputs, and new tasks are kept only if they differ from every existing one (ROUGE-L below 0.7). Tuning GPT-3 on the resulting 52K instructions improved it by 33 points on Super-NaturalInstructions.

**Alpaca** (Taori et al., Stanford, 2023) applied the same pipeline with text-davinci-003 as the generator and LLaMA 7B as the student. For under $600 it produced a model that, in the authors' small study, behaved qualitatively like text-davinci-003 on many prompts. Alpaca started the wave of distilled open chat models. It also showed the limits: the student imitates the teacher's style much faster than its knowledge, and the data inherits the teacher's errors. In practice, this is **Knowledge distillation** through text.

## Worked example: mixing four datasets

Four NLI-and-QA datasets with real training sizes, a budget of $N=20{,}000$ examples, and three mixing rules.

Mixing rates and epochs covered for N = 20,000 (computed in Python)

| dataset | e_m | proportional | cap K = 3,000 | epochs (K = 3,000) | T = 2, no cap |
| --- | --- | --- | --- | --- | --- |
| SNLI | 550,152 | 97.84% | 34.32% | 0.0125 | 82.00% |
| BoolQ | 9,427 | 1.68% | 34.32% | 0.728 | 10.73% |
| RTE | 2,490 | 0.44% | 28.49% | 2.29 | 5.52% |
| CB | 250 | 0.04% | 2.86% | 2.29 | 1.75% |

**Computing the rates**

1. **Proportional**: $r_m=e_m/\sum e_n$, with $\sum e_n=562{,}319$. SNLI gets $550{,}152/562{,}319=97.84\%$, and every dataset is covered for the same $20{,}000/562{,}319=0.0356$ epochs. CB would contribute about 9 examples in total.
2. **Cap** $K=3{,}000$: the capped sizes are $3000,3000,2490,250$, with sum $8{,}740$. SNLI and BoolQ get $3000/8740=34.32\%$ each, RTE $28.49\%$ and CB $2.86\%$.
3. **Epochs** under the cap: SNLI $0.3432\cdot20000/550152=0.0125$, BoolQ $0.728$, and RTE and CB both $2.29$. Every dataset below the cap is seen for the same number of epochs, $N/\sum\min(e_n,K)$.
4. **Temperature** $T=2$ without a cap uses weights $\sqrt{e_m}=741.7,97.1,49.9,15.8$, giving $82.0\%,10.7\%,5.5\%,1.75\%$. That is gentler than a cap: SNLI still dominates, and CB rises from 9 to 350 examples.

```python
import random
import numpy as np

# A template turns one example into (instruction-formatted input, target).
def nli_templates():
    opts = "OPTIONS:\n- yes\n- it is not possible to tell\n- no"
    return [
        lambda e: (f"Premise: {e['p']}\nHypothesis: {e['h']}\nDoes the premise entail the hypothesis?\n{opts}", e["y"]),
        lambda e: (f"{e['p']}\nBased on the sentence above, can we conclude that \"{e['h']}\"?\n{opts}", e["y"]),
        # "turn the task around": given the label, write the hypothesis
        lambda e: (f"Premise: {e['p']}\nWrite a hypothesis that the premise {'supports' if e['y'] == 'yes' else 'contradicts' if e['y'] == 'no' else 'neither supports nor rules out'}.", e["h"]),
    ]

def sentiment_templates():
    return [
        lambda e: (f"Review: {e['x']}\nIs this review positive or negative?", e["y"]),
        lambda e: (f"\"{e['x']}\"\nHow would the author rate the movie: positive or negative?", e["y"]),
    ]

rte = [{"p": "The cat sat on the warm mat all afternoon.", "h": "The cat was outside.", "y": "it is not possible to tell"}]
sst = [{"x": "A tender, funny and beautifully acted film.", "y": "positive"}]
datasets = [  # (name, cluster, train size, examples we hold in memory here, templates)
    ("SNLI", "NLI", 550_152, rte, nli_templates()),
    ("RTE", "NLI", 2_490, rte, nli_templates()),
    ("SST-2", "sentiment", 67_349, sst, sentiment_templates()),
]

def mixing_rates(sizes, cap=3_000, temperature=1.0):
    """Examples-proportional mixing with a per-dataset cap (Raffel et al.), optionally temperature-scaled."""
    w = np.minimum(np.asarray(sizes, float), cap) ** (1.0 / temperature)
    return w / w.sum()

def held_out(datasets, cluster):
    """FLAN-style evaluation split: train on every cluster except the one evaluated zero-shot."""
    return [d for d in datasets if d[1] != cluster], [d for d in datasets if d[1] == cluster]

train, test = held_out(datasets, "NLI")
print([d[0] for d in train], [d[0] for d in test])                 # ['SST-2'] ['SNLI', 'RTE']

rates_all = mixing_rates([d[2] for d in datasets])
print({d[0]: round(float(r), 4) for d, r in zip(datasets, rates_all)})  # {'SNLI': 0.3534, 'RTE': 0.2933, 'SST-2': 0.3534}

rng = random.Random(0)
def sample(datasets, rates, n):
    for _ in range(n):
        name, _, _, examples, templates = datasets[rng.choices(range(len(datasets)), weights=rates)[0]]
        template = rng.choice(templates)              # a fresh phrasing every time the example is drawn
        yield name, template(rng.choice(examples))

for name, (x, y) in sample(datasets, rates_all, 2):
    print(f"[{name}]\n{x}\n-> {y}\n")
# [SST-2]
# "A tender, funny and beautifully acted film."
# How would the author rate the movie: positive or negative?
# -> positive
#
# [SNLI]
# Premise: The cat sat on the warm mat all afternoon.
# Write a hypothesis that the premise neither supports nor rules out.
# -> The cat was outside.
```

Instruction-tuning data: templates, a FLAN-style held-out split, capped mixing, and sampling a fresh template per draw (runs with numpy; outputs shown)

> **Zero-shot here means zero tasks of that kind:** FLAN's headline numbers come from leaving out the **whole cluster**: no NLI dataset of any kind appears in training when NLI is scored. Holding out only RTE while training on MNLI and SNLI would measure dataset transfer, which is much easier. When you read an instruction-tuning result, check which of the two protocols was used.

> **Contamination and template overfitting:** Academic test sets leak easily into instruction mixtures. The same dataset can come in through another collection, or its examples can be paraphrased by a generator model. Use held-out *clusters* and deduplicate against evaluation sets. Small models also overfit the template surface: they learn "OPTIONS: yes / no" and degrade on free-form requests. Mixing in open-ended and few-shot formats counters both problems.

> **Instruction tuning, SFT and RLHF:** The three terms overlap. Instruction tuning usually means multi-task tuning on templated NLP tasks for zero-shot generalisation (FLAN, T0, Flan-T5). **Supervised fine-tuning (SFT)** usually means tuning on chat demonstrations (InstructGPT, LIMA). **RLHF** and **Direct preference optimization (DPO)** follow either one with preference data. Modern post-training mixtures combine all of them.

**Strengths**

- Cheap: Flan-PaLM's tuning used about 0.2% of its pretraining compute
- Large zero-shot gains on unseen task types at sufficient scale (20 of 25 datasets over GPT-3 for FLAN)
- Builds on existing labelled datasets; templates are cheap to write
- Gives a better starting point for later task-specific fine-tuning (Flan-T5 converged faster than T5 in Longpre et al.)

**Limitations**

- Hurt held-out performance for FLAN models of 8B and below
- Academic tasks have short targets; tuned models can become terse and template-bound on open-ended requests
- Mixture design (caps, temperature, formats) changes results and is easy to get wrong
- Synthetic instruction data (Self-Instruct, Alpaca) inherits the generator's errors and style

**Question:** In FLAN's evaluation, what does it mean for NLI to be held out?

- A. The NLI test split is not used for early stopping
- B. No NLI dataset, in any template, is in the training mixture of the model that is scored on NLI
- C. NLI examples are seen only with zero-shot templates
- D. Only RTE is excluded from training

**Answer:** B. No NLI dataset, in any template, is in the training mixture of the model that is scored on NLI — FLAN groups datasets into task clusters and trains a separate model for each held-out cluster. This tests generalisation to a new task type, not to a new dataset of a type already seen. Holding out a single dataset would let MNLI and SNLI teach the task.

**Question:** With examples-proportional mixing capped at K = 3,000 and N = 100,000 training examples, why is CB (250 examples) repeated 4.73 times, the same as RTE (2,490)?

- A. Because the cap raises CB to 3,000 examples
- B. Every dataset below the cap gets a rate proportional to its own size, so all of them are covered for the same N / Σ min(e_n, K) epochs
- C. Because temperature is 1
- D. CB and RTE are in the same cluster, so they share a rate

**Answer:** B. Every dataset below the cap gets a rate proportional to its own size, so all of them are covered for the same N / Σ min(e_n, K) epochs — Below the cap, $r_m\propto e_m$, so epochs $=r_mN/e_m=N/\sum_n\min(e_n,K)=100{,}000/21{,}140=4.73$ for every uncapped dataset. The cap only shrinks the large datasets. Raising $T$ is what boosts the smallest ones further.

**Key points**

- Instruction tuning fine-tunes on many tasks phrased as natural-language instructions, so the model can do unseen tasks from an instruction alone.
- FLAN: 137B, 62 datasets, 12 clusters, 10 templates each, evaluated on whole held-out clusters. It beat zero-shot GPT-3 on 20 of 25 datasets.
- Mixing: $r_m\propto\min(e_m,K)^{1/T}$. Caps stop huge datasets from dominating, but repeat small ones for $N/\sum\min(e_n,K)$ epochs.
- Gains grow with the number and diversity of tasks (Flan: 9 → 1,836 tasks, flattening after about 282) and with model size. FLAN at 8B and below got worse on held-out tasks.
- Varied templates help: zero-shot, few-shot, chain of thought and inverted tasks. Self-Instruct and Alpaca replaced human-written tasks with model-generated ones.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
