Diffusion Language Models, Playable

EP 01

Typewriter vs darkroom — watch both write the same sentence

Same target sentence, two philosophies. Left: the autoregressive typewriter — one committed token at a time, each conditioned on everything before it. Right: the diffusion darkroom — every position starts as a masked blur, and each refinement step unmasks/sharpens tokens anywhere in the sentence, in parallel. Run them side by side and watch the different shapes of "in progress."

✓ good case: diffusion's draft is globally coherent early — the skeleton appears before the details✗ bad case: the typewriter at 50% has a perfect first half and no idea how it ends — commitment without foresight

—

The unmasking order on the right follows real text-diffusion behavior: high-confidence tokens (common words, constrained slots) resolve first, ambiguous ones last. Simulated dynamics; the paradigm contrast is the point.

Autoregression is a commitment machine; diffusion is a revision machine. The typewriter can never take back a token; the darkroom holds every position open until the end. Everything else on this page — the speed, the editing tricks, the failure modes — falls out of that one difference.

EP 02

Denoise it yourself — you are the model now

Feel the job from the inside. A sentence has been corrupted: some words masked, some swapped for wrong ones. Each round, click the position you're most confident about and pick its word — exactly the greedy-confidence policy real text diffusers use. Your accuracy is scored against the original. Then press "worst-first" to see why order matters: resolving the ambiguous slots first forces guesses without context.

✓ good case: confident-first — early picks create context that makes later picks easy; 90%+ recovery✗ bad case: worst-first — guessing "the hard slot" blind, errors cascade into the easy slots too

—

The "confidence" per slot is computed from real bigram context around it (how many candidate words fit its neighbors) — resolving a slot genuinely updates its neighbors' confidences, which is the whole engine of iterative denoising.

Denoising text = repeatedly answering "what am I most sure about?" Every resolved token is new context that de-blurs its neighbors — confidence cascades outward like a crossword filling itself in. That cascade is why a few parallel passes can do the work of many serial steps.

EP 03

The speed race — 1,000 little steps vs 8 big ones

The commercial argument. Autoregression needs one forward pass per token — 1,000 tokens, 1,000 sequential passes, each waiting for the last. Diffusion needs one pass per refinement round, updating all 1,000 positions each time — maybe 4 to 16 rounds total. Set the sequence length and the diffusion round count; the race runs with honest arithmetic. Then push rounds down to 2 and watch quality pay for the speed.

✓ good case: 1,000 tokens, 8 rounds — diffusion finishes ~100× fewer sequential passes (this is Mercury's headline)✗ bad case: 2 rounds — blazing fast, but the quality meter drops: not enough passes for the confidence cascade to settle

sequence length 1024 diffusion rounds 8 —

Sequential-pass counts are exact; the quality estimate follows the published rounds-vs-quality tradeoff shape (steep loss below ~4 rounds). Real deployments also exploit that each diffusion pass batches perfectly on GPUs.

Autoregression's cost is fundamentally serial; diffusion's is fundamentally parallel. That's why the first commercial text diffusers marketed tokens-per-second above all else — and why the AR camp answered with speculative decoding, its own borrowed parallelism. The latency war between the two shapes is just beginning.

EP 04

Editing superpowers — infill the middle, hold the ends

Ask a typewriter to fill a hole in the middle of a paragraph and it sweats — it only knows how to continue leftward context. For diffusion this isn't even a special case: freeze the tokens you like, mask the ones you don't, denoise. Try it: click words in the sentence below to mask them (any pattern — middle, scattered, alternating), then regenerate only the masked slots under the constraint of everything you kept.

✓ good case: mask the middle five words — regenerated text respects BOTH ends seamlessly✗ bad case: the AR baseline on the same infill — it satisfies the left context and plows straight through the right boundary

click words to mask them, then denoise

The demo regenerates masked slots from a small phrase bank scored against both neighbors — a toy of the real property: diffusion conditions on arbitrary token subsets natively, no special infill training needed.

Every position is a first-class citizen. Infilling, templated generation ("fill only the JSON values"), simultaneous multi-span editing — things AR models need special tricks for (FIM training, suffix prompting) come free with the diffusion shape. Code editors noticed first.

EP 05

Where it cracks — commitment has its virtues

Now the honest ledger. Diffusion's weakness is the mirror of its strength: with no arrow of time, it struggles with strict sequential dependency — arithmetic carried digit by digit, code where line 10 must reference a variable named on line 2, chains of reasoning. Run the three benchmark tasks below on both paradigms (measured over 200 instances each) and watch the winner flip with the task's structure.

✓ good case: parallel-structure tasks (summarize, translate, template-fill) — diffusion matches AR at a fraction of the latency✗ bad case: multi-step arithmetic — diffusion's parallel guesses can't carry the 1; AR wins by 20+ points

—

Task simulations encode the structural difference honestly: sequential tasks make later slots depend on earlier RESOLVED values (which parallel denoising can violate); parallel tasks don't. 200 instances per click.

The two shapes are converging, not fighting to the death: block-diffusion (diffuse a chunk, commit, diffuse the next) and AR-with-revision hybrids are already shipping. The likely future isn't typewriter or darkroom — it's a printer that drafts pages in parallel and proofreads them in order.