Definition
Synthetic data generation is producing training examples with a model instead of collecting them from people: instructions and answers, dialogues, preference pairs, code with tests, or labelled examples for a classifier. The standard recipe for instruction data is Self-Instruct (Wang et al., "Self-Instruct: Aligning Language Models with Self-Generated Instructions", 2022, ACL 2023). Its main long-run risk, model collapse, was characterised by Shumailov et al. ("AI models collapse when trained on recursively generated data", Nature, 2024).
Human demonstrations are slow and expensive. InstructGPT used about 13K prompts with demonstrations written by contractors. A generator model can write tens of thousands of examples in hours for a few hundred dollars. That is why most open instruction-tuned models since 2023 were trained largely on synthetic data (see Instruction tuning and Knowledge distillation).
In a real pipeline, synthetic data sits between a task spec and Supervised fine-tuning (SFT). Seed examples and a generator produce candidates. Filters drop the bad ones. Then deduplication, decontamination and a quality pass prepare the final mix (see Dataset curation for fine-tuning). Most of the engineering goes into the filters.
Self-Instruct and Alpaca
Self-Instruct starts from 175 human-written seed tasks, each with one instruction and one instance. Every step samples 8 tasks for the prompt, 6 from the seeds and 2 from earlier generations, and asks GPT-3 for a ninth. The model then writes input–output instances. Classification tasks are generated output-first (label, then an input for it) so that labels stay balanced; other tasks are input-first. A new instruction joins the pool only if its ROUGE-L overlap with every existing instruction is below 0.7.
- candidate instruction and an existing pool instruction, as word sequences
- length of the longest common subsequence (in order, gaps allowed)
- Self-Instruct's novelty threshold; lower values force more diverse tasks but reject more
Stanford's Alpaca (Taori et al., 2023) reused the recipe with text-davinci-003 as the generator. It wrote 20 instructions per call and skipped the classification step. The result was 52K instruction–response pairs for under $500 of API calls. Fine-tuning LLaMA 7B on them took 3 hours on eight 80 GB A100s. Alpaca's licence was non-commercial, because of the LLaMA licence and the generator's terms of use.
Making data harder and better
Self-Instruct's tasks are short and easy. Evol-Instruct (Xu et al., "WizardLM", 2023) asks a model to rewrite each instruction. In-depth evolution adds constraints, deepens the question, makes it more concrete, adds reasoning steps or complicates the input. In-breadth evolution writes a new task on a rarer topic. A filter drops failed evolutions, such as copies of the original or responses that are only an apology. Four rounds starting from Alpaca's 52K gave about 250K instructions for WizardLM.
Where answers can be checked, use the check as the filter. Run generated code against generated unit tests. Keep chain-of-thought solutions only when the final answer matches a known one, as in STaR (Zelikman et al., 2022). Pick the best of several samples with a reward model or LLM-as-a-judge. Phi-1 (Gunasekar et al., "Textbooks Are All You Need", 2023) shows how far careful curation goes: a 1.3B code model trained on 6B tokens of filtered web code plus under 1B tokens of GPT-3.5-written textbooks, then fine-tuned on about 180M tokens of synthetic exercises, reached 50.6% pass@1 on HumanEval.
Reference numbers
- Self-Instruct
- 175 seeds → 52K instructions, 82K instances from GPT-3. Tuned GPT-3 gained 33 points absolute on Super-NaturalInstructions, on par with InstructGPT-001.
- Alpaca
- 52K pairs from text-davinci-003, under $500 to generate; LLaMA 7B fine-tuned in 3 h on 8 × A100 80 GB.
- Evol-Instruct
- 4 evolution rounds from Alpaca's 52K → about 250K instructions; WizardLM is LLaMA fine-tuned on them.
- Model collapse
- Shumailov et al. fine-tuned OPT-125m on WikiText-2 over many generations. Perplexity on real data rose each generation; keeping 10% of the original data in every generation made the degradation much smaller.
- Accumulation
- Gerstgrasser et al. (2024): if each generation adds synthetic data to all previous data instead of replacing it, test error stays bounded rather than growing.
Model collapse
Suppose each model generation is trained only on samples from the previous one. Every finite sample under-represents rare events, and a model fitted to it gives them less probability, so the next sample has fewer still. Shumailov et al. found that the tails of the distribution go first (early collapse). Later the model converges to a narrow distribution unlike the original (late collapse). The simplest case shows the mechanism. Fit a Gaussian by maximum likelihood to samples of the previous fit:
- variance of the generation- fit; each MLE step shrinks it by in expectation
- probability the current model gives to a rare type (a phrasing, a topic, a dialect)
- training samples per generation; a type with zero count gets probability 0 and never returns
Worked example
| Quantity | Computation | Result |
|---|---|---|
| Candidate "Write a short poem about the ocean" vs "Write a poem about the sea" | LCS = write a poem about the = 5; P = 5/7, R = 5/6; 2PR/(P+R) | 0.769 → rejected |
| "List three uses of baking soda" vs "Give three tips for staying healthy" | LCS = three = 1; P = R = 1/6 | 0.167 → kept |
| Type rank 60 lost in one generation | q = 0.0028, n = 500: (1 − 0.0028)^500 | 0.246 |
| Type rank 30 lost in one generation | q = 0.0060: (1 − 0.0060)^500 | 0.049 |
| Gaussian variance after 10 generations | n = 10: (9/10)^10 | 0.349 σ² |
Rare types are lost far more often than common ones. Once lost, nothing in a closed loop can bring them back. That is why the tail goes first.
1import re23def lcs(a, b):4 """Length of the longest common subsequence of two token lists."""5 prev = [0] * (len(b) + 1)6 for x in a:7 cur = [0]8 for j, y in enumerate(b):9 cur.append(prev[j] + 1 if x == y else max(prev[j + 1], cur[j]))10 prev = cur11 return prev[-1]1213def rouge_l(cand, ref):14 c, r = cand.lower().split(), ref.lower().split()15 m = lcs(c, r)16 if m == 0:17 return 0.018 p, rec = m / len(c), m / len(r)19 return 2 * p * rec / (p + rec)2021BLOCK = re.compile(r"\b(image|images|picture|graph|graphs)\b", re.I)2223def keep(new, pool, threshold=0.7):24 if BLOCK.search(new) or not 3 <= len(new.split()) <= 150:25 return False26 return max((rouge_l(new, old) for old in pool), default=0.0) < threshold2728pool = ["Write a poem about the sea", "Give three tips for staying healthy"]29for cand in ["Write a short poem about the ocean",30 "List three uses of baking soda",31 "Describe the picture in one sentence"]:32 print(f"{cand!r}: max ROUGE-L {max(rouge_l(cand, p) for p in pool):.3f}, keep={keep(cand, pool)}")33# 'Write a short poem about the ocean': max ROUGE-L 0.769, keep=False34# 'List three uses of baking soda': max ROUGE-L 0.167, keep=True35# 'Describe the picture in one sentence': max ROUGE-L 0.167, keep=FalseUse when
- +Orders of magnitude cheaper and faster than human annotation.
- +Targeted: generate exactly the tasks, formats and edge cases you lack.
- +Verifiable domains (code, maths, SQL) allow automatic correctness filters.
Avoid when
- −Bounded by the generator: errors, biases and blind spots carry over.
- −Low diversity without explicit novelty filters; easy to overfit one style.
- −Recursive training on model output loses the tails (model collapse); needs real data in the mix.
Check yourself
A team retrains its model every month only on text sampled from last month's model. What happens first?
Check yourself
Why does Self-Instruct generate classification instances output-first?
Remember
- ✓Self-Instruct: 175 seeds, 8 in-context tasks (6 seed + 2 generated), novelty filter ROUGE-L < 0.7 → 52K instructions; Alpaca reused it with text-davinci-003 for under $500.
- ✓Evol-Instruct rewrites instructions deeper (constraints, reasoning steps) or broader (new rare tasks), about 250K from Alpaca's 52K.
- ✓Filter on correctness (tests, answer checks, judges) and diversity, not fluency; read samples.
- ✓Model collapse: and a rare type is lost with chance per generation, so tails vanish first.
- ✓Keep real data in every generation, or accumulate rather than replace; both stop the collapse.