Noise Schedules, Playable

EP 01

One image, two clocks — scrub the dissolve yourself

Diffusion training destroys an image over 1000 timesteps. A schedule decides how much of the original survives at each step t (the number ᾱ_t). Below, the same image and the same random static are shown at the same t under the two most famous schedules. Drag the slider: linear (the original DDPM recipe) hits pure static long before step 1000; cosine keeps something recognizable far deeper.

timestep t

The toy "photo" is generated in this page and both panels share one fixed noise pattern, so the ONLY difference you see is the schedule. Real pipelines noise 512×512 latents the same way, just bigger.

Feel it: around t = 750 the linear panel is already indistinguishable from television static (✗ nothing left to learn from), while cosine still shows the sun and horizon (✓ a real denoising problem). Same network, same data — one slider setting, totally different training diet.

EP 02

The survival curve — how much image is left at step t

Plot ᾱ_t (the fraction of original signal variance surviving at step t) for both schedules and the difference is stark. Linear free-falls: it crosses 1% signal around two-thirds of the way in, then flatlines at zero for the rest. Cosine was designed by OpenAI (Nichol & Dhariwal, 2021) to descend gently and spend its steps where the image is half-noise — the informative middle. Move the scrubber and read both curves.

scrub t

The red strip marks steps where a schedule keeps under 1% of the signal — computed live from the actual β_t products, not drawn by hand.

The flatline is the bug: every training step sampled from linear's long zero-tail shows the network pure static and asks it to find an image inside. There is nothing to find — the gradient is noise about noise. That flat region is the ✗ bad case, and it's a third of the whole schedule.

EP 03

SNR — the only honest ruler for a schedule

The clean way to compare schedules is signal-to-noise ratio: SNR(t) = ᾱ_t / (1−ᾱ_t), plotted on a log scale. Learning happens in a band around SNR ≈ 1 — where image and noise fight on equal terms. Too far above: the task is trivially easy. Too far below: impossible. Widen or narrow the band and watch what fraction of each schedule's 1000 steps actually lands inside it.

informative band: |log₁₀ SNR| ≤

"Informative band" is a pedagogical simplification — real loss-weighting papers (e.g. min-SNR, v-prediction) define usefulness more carefully, but the shape of the argument is exactly this.

✓ Why cosine wins: at any reasonable band width, cosine parks a visibly larger share of its steps inside the useful zone, because its log-SNR falls almost linearly instead of cliff-diving. This is the exact argument the improved-DDPM paper made — and why cosine-like schedules became the default in Stable Diffusion-era models.

EP 04

The dead zone — a filmstrip of steps that teach nothing

Here is linear's failure made visible. The filmstrip shows 10 evenly spaced timesteps from your chosen start to t = 999. Frames where the surviving signal amplitude √ᾱ drops under 6% are flagged STATIC — a human can't tell them apart, and neither can a useful gradient. Flip the schedule and count how many frames come back to life.

strip starts at t =

Each frame gets its own fresh static pattern (as it would in training), so identical-looking frames really are carrying zero distinguishable image — that's the point.

✗ The wasted tail: under linear, hundreds of the 1000 timesteps sit below the static threshold — every training sample drawn there is a wasted forward pass, wasted GPU, wasted gradient. ✓ Under cosine the dead tail shrinks to a few dozen steps. Same compute bill, radically different amount of actual teaching.

EP 05

The training race — same budget, different diet

Final proof. Two toy models train side by side with an identical budget of 400 steps and the identical random timesteps — only the schedule differs. Each step teaches an amount proportional to how balanced signal and noise are at that t (peaking when ᾱ = ½, zero at the extremes). Watch each model's "mental image" sharpen as useful signal accumulates.

Conceptual simulation: "learning per step" is the analytic weight 4·ᾱ(1−ᾱ), not a trained network. But the ratio between the two bars is the real ratio of informative steps between the schedules.

This is the whole lesson: ✓ the cosine model finishes visibly sharper because more of its 400 draws landed in the informative middle; ✗ the linear model spent a big slice of the same budget staring at pure static and learning nothing. Swapping one cheap formula — no extra parameters, no extra compute — was worth a real FID improvement in Improved DDPM, and schedule design (cosine, v-prediction, flow-matching time warps) is still an active lever in every modern image model.