# Synthetic data generation

> Strong models write training data (Self-Instruct, Alpaca's 52K pairs, Evol-Instruct) that must be filtered hard; training generation after generation on model output alone erases rare cases (model collapse) unless real data stays in.

Source: https://www.mlfoundry.in/learn/synthetic-data-generation  
Section: Generative AI › Customization  
Level: core · 8 min read · updated 2026-10-21

**Synthetic data generation:** is producing training examples with a model instead of collecting them from people: instructions and answers, dialogues, preference pairs, code with tests, or labelled examples for a classifier. The standard recipe for instruction data is **Self-Instruct** (Wang et al., "Self-Instruct: Aligning Language Models with Self-Generated Instructions", 2022, ACL 2023). Its main long-run risk, **model collapse**, was characterised by Shumailov et al. ("AI models collapse when trained on recursively generated data", Nature, 2024).

Human demonstrations are slow and expensive. InstructGPT used about 13K prompts with demonstrations written by contractors. A generator model can write tens of thousands of examples in hours for a few hundred dollars. That is why most open instruction-tuned models since 2023 were trained largely on synthetic data (see **Instruction tuning** and **Knowledge distillation**).

In a real pipeline, synthetic data sits between a task spec and **Supervised fine-tuning (SFT)**. Seed examples and a generator produce candidates. Filters drop the bad ones. Then deduplication, decontamination and a quality pass prepare the final mix (see **Dataset curation for fine-tuning**). Most of the engineering goes into the filters.

## Self-Instruct and Alpaca

Self-Instruct starts from 175 human-written seed tasks, each with one instruction and one instance. Every step samples 8 tasks for the prompt, 6 from the seeds and 2 from earlier generations, and asks GPT-3 for a ninth. The model then writes input–output instances. Classification tasks are generated *output-first* (label, then an input for it) so that labels stay balanced; other tasks are input-first. A new instruction joins the pool only if its ROUGE-L overlap with every existing instruction is below 0.7.

$$
\text{ROUGE-L}(c,r)=\frac{2PR}{P+R},\quad P=\frac{\mathrm{LCS}(c,r)}{|c|},\;R=\frac{\mathrm{LCS}(c,r)}{|r|};\qquad \text{keep } c \iff \max_{r\in\text{pool}}\text{ROUGE-L}(c,r)<0.7
$$

- $c,\ r$: candidate instruction and an existing pool instruction, as word sequences
- $\mathrm{LCS}$: length of the longest common subsequence (in order, gaps allowed)
- $0.7$: Self-Instruct's novelty threshold; lower values force more diverse tasks but reject more

> Figure: The Self-Instruct loop. A pool of 175 seed tasks supplies 8 in-context examples; the model writes a new task and its instances; filters reject near-duplicates (ROUGE-L ≥ 0.7), tasks a text model cannot do (images, pictures, graphs) and malformed instances; survivors join the pool. Evol-Instruct then rewrites instructions to be harder or broader. The output feeds SFT.

Stanford's **Alpaca** (Taori et al., 2023) reused the recipe with `text-davinci-003` as the generator. It wrote 20 instructions per call and skipped the classification step. The result was 52K instruction–response pairs for under $500 of API calls. Fine-tuning LLaMA 7B on them took 3 hours on eight 80 GB A100s. Alpaca's licence was non-commercial, because of the LLaMA licence and the generator's terms of use.

## Making data harder and better

Self-Instruct's tasks are short and easy. **Evol-Instruct** (Xu et al., "WizardLM", 2023) asks a model to rewrite each instruction. *In-depth* evolution adds constraints, deepens the question, makes it more concrete, adds reasoning steps or complicates the input. *In-breadth* evolution writes a new task on a rarer topic. A filter drops failed evolutions, such as copies of the original or responses that are only an apology. Four rounds starting from Alpaca's 52K gave about 250K instructions for WizardLM.

Where answers can be checked, use the check as the filter. Run generated code against generated unit tests. Keep chain-of-thought solutions only when the final answer matches a known one, as in STaR (Zelikman et al., 2022). Pick the best of several samples with a reward model or **LLM-as-a-judge**. Phi-1 (Gunasekar et al., "Textbooks Are All You Need", 2023) shows how far careful curation goes: a 1.3B code model trained on 6B tokens of filtered web code plus under 1B tokens of GPT-3.5-written textbooks, then fine-tuned on about 180M tokens of synthetic exercises, reached 50.6% pass@1 on HumanEval.

**Reference numbers**

- **Self-Instruct:** 175 seeds → 52K instructions, 82K instances from GPT-3. Tuned GPT-3 gained 33 points absolute on Super-NaturalInstructions, on par with InstructGPT-001.
- **Alpaca:** 52K pairs from text-davinci-003, under $500 to generate; LLaMA 7B fine-tuned in 3 h on 8 × A100 80 GB.
- **Evol-Instruct:** 4 evolution rounds from Alpaca's 52K → about 250K instructions; WizardLM is LLaMA fine-tuned on them.
- **Model collapse:** Shumailov et al. fine-tuned OPT-125m on WikiText-2 over many generations. Perplexity on real data rose each generation; keeping 10% of the original data in every generation made the degradation much smaller.
- **Accumulation:** Gerstgrasser et al. (2024): if each generation adds synthetic data to all previous data instead of replacing it, test error stays bounded rather than growing.

## Model collapse

Suppose each model generation is trained only on samples from the previous one. Every finite sample under-represents rare events, and a model fitted to it gives them less probability, so the next sample has fewer still. Shumailov et al. found that the tails of the distribution go first (early collapse). Later the model converges to a narrow distribution unlike the original (late collapse). The simplest case shows the mechanism. Fit a Gaussian by maximum likelihood to $n$ samples of the previous fit:

$$
\mathbb E\big[\hat\sigma^2_t\big]=\Big(\frac{n-1}{n}\Big)^t\sigma^2,\qquad \Pr[\text{type } i \text{ lost in one generation}]=(1-q_i)^n
$$

- $\hat\sigma^2_t$: variance of the generation-$t$ fit; each MLE step shrinks it by $(n-1)/n$ in expectation
- $q_i$: probability the current model gives to a rare type $i$ (a phrasing, a topic, a dialect)
- $n$: training samples per generation; a type with zero count gets probability 0 and never returns

> Figure: Repeated maximum-likelihood fits of a 60-type Zipf distribution (s = 1.1). Defaults: 500 samples per generation, no real data, replacing the training set each time. Generation 1 keeps 57 of 60 types. By generation 10 only 25 remain; 25 of the 30 tail types are gone, tail mass is 0.044 instead of 0.119, and TV distance is 0.217. At generation 40, 12 types remain and TV is 0.419. With 10% fresh real data, 38 types survive to generation 40 (TV 0.263); with 20%, 41 survive (TV 0.211). Accumulating all data keeps 57 types and TV stays near 0.146.

## Worked example

Self-Instruct's ROUGE-L filter (threshold 0.7) and the collapse formulas, with the numbers from the figure

| Quantity | Computation | Result |
| --- | --- | --- |
| Candidate "Write a short poem about the ocean" vs "Write a poem about the sea" | LCS = write a poem about the = 5; P = 5/7, R = 5/6; 2PR/(P+R) | 0.769 → rejected |
| "List three uses of baking soda" vs "Give three tips for staying healthy" | LCS = three = 1; P = R = 1/6 | 0.167 → kept |
| Type rank 60 lost in one generation | q = 0.0028, n = 500: (1 − 0.0028)^500 | 0.246 |
| Type rank 30 lost in one generation | q = 0.0060: (1 − 0.0060)^500 | 0.049 |
| Gaussian variance after 10 generations | n = 10: (9/10)^10 | 0.349 σ² |

Rare types are lost far more often than common ones. Once lost, nothing in a closed loop can bring them back. That is why the tail goes first.

```python
import re

def lcs(a, b):
    """Length of the longest common subsequence of two token lists."""
    prev = [0] * (len(b) + 1)
    for x in a:
        cur = [0]
        for j, y in enumerate(b):
            cur.append(prev[j] + 1 if x == y else max(prev[j + 1], cur[j]))
        prev = cur
    return prev[-1]

def rouge_l(cand, ref):
    c, r = cand.lower().split(), ref.lower().split()
    m = lcs(c, r)
    if m == 0:
        return 0.0
    p, rec = m / len(c), m / len(r)
    return 2 * p * rec / (p + rec)

BLOCK = re.compile(r"\b(image|images|picture|graph|graphs)\b", re.I)

def keep(new, pool, threshold=0.7):
    if BLOCK.search(new) or not 3 <= len(new.split()) <= 150:
        return False
    return max((rouge_l(new, old) for old in pool), default=0.0) < threshold

pool = ["Write a poem about the sea", "Give three tips for staying healthy"]
for cand in ["Write a short poem about the ocean",
             "List three uses of baking soda",
             "Describe the picture in one sentence"]:
    print(f"{cand!r}: max ROUGE-L {max(rouge_l(cand, p) for p in pool):.3f}, keep={keep(cand, pool)}")
# 'Write a short poem about the ocean': max ROUGE-L 0.769, keep=False
# 'List three uses of baking soda': max ROUGE-L 0.167, keep=True
# 'Describe the picture in one sentence': max ROUGE-L 0.167, keep=False
```

Self-Instruct-style novelty and keyword filters (pure Python, runs as is). The generator call is left out; in production, embedding similarity often replaces ROUGE-L for scale.

> **The generator's mistakes become the data:** A synthetic set inherits the generator's errors, refusals, verbosity and biases. Small students copy the *style* of a strong teacher much more easily than its correctness. They can sound confident while being wrong more often. Filter on correctness, not fluency: execution, answer checks, or a judge with a rubric. Read a random sample of 100 examples before training.

> **Terms of use:** Some API providers' terms restrict using model outputs to train competing models, and open-weight licences can carry conditions on outputs too. Check the generator's terms before publishing a dataset or a model trained on it (see **Copyright and licensing**; this is not legal advice).

**Strengths**

- Orders of magnitude cheaper and faster than human annotation.
- Targeted: generate exactly the tasks, formats and edge cases you lack.
- Verifiable domains (code, maths, SQL) allow automatic correctness filters.

**Limitations**

- Bounded by the generator: errors, biases and blind spots carry over.
- Low diversity without explicit novelty filters; easy to overfit one style.
- Recursive training on model output loses the tails (model collapse); needs real data in the mix.

**Question:** A team retrains its model every month only on text sampled from last month's model. What happens first?

- A. The most common phrasings disappear
- B. Rare cases in the tails disappear
- C. Variance grows without bound
- D. Nothing, if the model is large enough

**Answer:** B. Rare cases in the tails disappear — A type with probability $q$ gets zero samples with chance $(1-q)^n$, which is large only for small $q$. Once at zero it never returns, so tails go first (early collapse) and variance shrinks rather than grows. Mixing in real data or accumulating data prevents it.

**Question:** Why does Self-Instruct generate classification instances output-first?

- A. It is faster
- B. Input-first generation skews towards one label
- C. ROUGE-L only works on labels
- D. GPT-3 cannot read inputs

**Answer:** B. Input-first generation skews towards one label — Given an input first, the model tends to produce inputs that all fit one label. Writing the label first and then an input for it keeps the classes balanced.

**Key points**

- Self-Instruct: 175 seeds, 8 in-context tasks (6 seed + 2 generated), novelty filter ROUGE-L < 0.7 → 52K instructions; Alpaca reused it with text-davinci-003 for under $500.
- Evol-Instruct rewrites instructions deeper (constraints, reasoning steps) or broader (new rare tasks), about 250K from Alpaca's 52K.
- Filter on correctness (tests, answer checks, judges) and diversity, not fluency; read samples.
- Model collapse: $\mathbb E[\hat\sigma^2_t]=((n-1)/n)^t\sigma^2$ and a rare type is lost with chance $(1-q)^n$ per generation, so tails vanish first.
- Keep real data in every generation, or accumulate rather than replace; both stop the collapse.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
