Synthetic Data & Model Collapse, Playable

EP 01

The photocopy loop — copy a copy of a copy

The mechanism in miniature. A "model" learns a distribution of data (the histogram below), then generates a new dataset by sampling from what it learned, and the next model trains on that. Each generation loses a little: sampling under-represents rare values, and the next fit treats "rare" as "gone". Press the button repeatedly and watch a rich distribution photocopied into a spike.

✓ good case: generation 1 — the copy is faithful; you'd never notice the loss✗ bad case: generation 6 — variance down 80%, the edges extinct, everything average

—

This is a real resampling experiment: each click draws 400 samples from the current fitted distribution and refits. The variance shrinkage you see is the mathematically guaranteed part of model collapse (finite-sample estimation bias), not an illustration.

No single generation looks broken — that's the trap. Each copy is a 95%-faithful photocopy; the damage is compound interest. The original model-collapse paper (Shumailov et al.) called the early stage exactly this: the tails go first, silently.

EP 02

The dying tails — why the weird stuff matters most

Why care about the edges of a distribution? Because that's where the value lives. Below is a "knowledge population": common facts (center) and rare ones — edge cases, minority dialects, unusual bugs, niche expertise (tails). Run the recursion and watch coverage by category: the common stuff survives every generation; the rare stuff has a measured half-life. Then quiz the generation-5 model on tail questions.

✓ good case: "what's the capital of France" — survives 10 generations untouched✗ bad case: "how do you handle this rare compiler bug" — coverage 92% → 4% by generation 5; the model answers confidently from the center instead

—

Knowledge items are binned by rarity; per-generation survival probability is proportional to frequency — the compounding is computed, and the quiz samples real items from the surviving set.

Collapse isn't models getting dumb — it's models getting generic. Fluency survives; the long tail of hard-won specifics dies. And the tail is precisely what separates a useful expert from a plausible one, which is why collapse is scarier than it looks on any average-case benchmark.

EP 03

Five generations of faces — watch diversity die in pictures

The same experiment, visual. A generator learns a population of "faces" (parameterized dots: size, color, shape mix), then the next generator trains on its samples. The grid shows each generation's output. By generation 5 every face is the same face — the mode. This is the image-model version documented with real diffusion models: recursively trained generators converge to eerily similar outputs.

✓ good case: generation 0 — the population's full variety, ugly ducklings included✗ bad case: generation 5 — one beige face, printed 24 times; the ugly ducklings (and everything interesting) are extinct

generation 0 (real data) —

Each generation's population is genuinely resampled-and-refitted from the previous (same math as EP 01, applied per attribute). The diversity index is measured from the displayed population.

You've already seen mild versions of this — the "AI look" in generated images, the same five adjectives in AI text. Some homogenization comes from RLHF, but recursive training amplifies it: the internet's aesthetic averaging itself, one scrape at a time.

EP 04

The fresh-blood fix — how much real data keeps the pool alive?

Collapse isn't destiny — it's a mixing ratio. Rerun the recursion, but each generation trains on X% fresh real data + (100−X)% synthetic. Drag the dial and find the stabilization threshold: with enough real anchor, variance stops decaying and the tails persist indefinitely. The follow-up literature's core finding, on your screen.

✗ bad case: 0% fresh — guaranteed decay, just a question of when✓ good case: ~20-30% fresh real data — the distribution holds its variance across 10 generations

fresh real data per generation 0% —

Same resampling engine as EP 01 with real-data injection each round; the variance-vs-generation curves are measured per run. "Accumulate, don't replace" (keeping old real data in the mix) is why the milder scenarios in the literature don't collapse.

The web's future isn't doom, it's dilution management. As long as training pipelines keep anchoring on human-made data — and accumulate it rather than replace it — collapse stays theoretical. The real fight is knowing which is which… which is why AI-text detection and data provenance quietly became infrastructure.

EP 05

The plot twist — the best models are built ON synthetic data

Here's the confusing part: while everyone worried about collapse, frontier labs were training their best models on mountains of synthetic data — on purpose, successfully. The difference is a filter. Unfiltered self-output shrinks the distribution; synthetic data passed through a verifier (does the code run? is the proof valid? did the judge approve?) shifts it toward correctness instead. Toggle the filter and watch the same generator produce poison or fuel.

✗ bad case: raw self-output, no filter — quality drifts down, diversity dies: classic collapse✓ good case: generate 10×, keep the verified best — quality of the training pool goes UP each round (this is how math/code models got great)

—

Both runs generate from the current model; the filtered run rejects samples below a correctness bar (simulating a unit-test/proof-checker gate) before training the next generation. Quality trajectories are measured per run.

Synthetic data is a photocopier without a filter and a ratchet with one. The verifier converts generation from "sampling the middle" into "climbing toward verified-correct" — which is why self-improvement worked first in math and code (cheap perfect verifiers) and why "who has the best verifier" became a frontier-lab competition as real as who has the most GPUs.