Encode to a cloud, not a point
A plain autoencoder maps each input to one point in latent space. A VAE's encoder instead outputs two vectors: a mean μ and a spread σ — a gaussian blob saying "this image lives around here." Pick an input below, then use the slider to widen or narrow its blob and watch the rings (1σ and 2σ) breathe on the latent map.
The encoder here is hand-coded (it knows each input's true coordinates); a real VAE learns μ and σ per input with a neural network. The rings are the actual 1σ / 2σ radii for the slider value.
Press sample — one input, a family of outputs
Now the generative part: draw random points from the blob and decode each one. Every press of the button rolls fresh gaussian noise. With a small σ the variants are siblings; crank σ up and the family gets weird. The variation number below is measured from the actual decoded faces, not from the slider.
Samples use real Box–Muller gaussian noise — run it twice with the same σ and you get different variants every time. That randomness is the point.
The trick — rerouting gradients around a dice roll
Training needs backprop: the loss must send gradients back to μ and σ. But "sample from N(μ,σ²)" is not differentiable — a dice roll has no slope. Watch the green gradient pulse travel back from the loss: in the naive graph it dies at the sampling node. The trick rewrites the same randomness as z = μ + σ·ε with ε rolled outside the network — now z is plain arithmetic and the gradient flows through + and × straight into μ and σ.
The numbers are computed live from a 1-D toy: μ = 0.20, σ = 0.50, loss L = (z − 1)². Resample ε and watch both the forward value and the gradients change — in the naive graph the gradient simply doesn't exist.
Why the noise heals the holes
Same 12 training images, two latent maps. The plain autoencoder pins each one to a dot — the space between dots is unclaimed no-man's-land. The VAE forces each image to own a fuzzy blob, so blobs overlap and the space gets covered. Slide the interpolation between images A and B and watch what each decoder does with the exact same in-between point.
Coverage is measured live by probing a 30×30 grid of latent points: a point counts as covered if it's close enough to a training point (dot radius for the AE, dot + blob radius for the VAE).
The σ dial of doom — blur on one end, holes on the other
So just crank σ up? No. If the blob is huge, wildly different z values must all reconstruct the same input — the decoder's best move is to output the average, and averages are blurry. That's the famous VAE blur. If σ → 0, samples never leave the mean: sharp outputs, but you're back to a plain autoencoder full of holes. Drag the dial across the whole range and watch both meters, measured live.
The "blurry" render is honest about the mechanism: it overlays the decodings of 9 fixed noise samples at the current σ — exactly the set of targets the decoder would be averaging over during training. Sharpness is computed from their measured spread; coverage reuses the EP 04 probe.