Generative Adversarial Networks, Playable

EP 01

Be the forger — sneak your fakes into the real crowd

White dots are real data (say, measurements of real faces, squashed to 2D). Gold dots are your forgeries. The detective below is the best possible straight-line classifier, retrained live every time you move a slider. Your job as forger: drag your fakes until the detective's accuracy drops to a coin flip.

Forgery position Spread

The core idea: the generator's entire training signal is "how badly did the detective bust me?". When detective accuracy hits ~50%, fakes and reals are statistically indistinguishable — that is the equilibrium every GAN is chasing. StyleGAN's photorealistic faces are exactly this game, played in a million dimensions.

EP 02

Now be the detective — early training vs late training

One mystery point at a time: is it a real sample or a forgery? The faint cloud shows where real data lives. In early training the generator is clueless, so you should crush this. Switch to late training and see what an equilibrium feels like from the detective's chair. Play at least 10 rounds of each.

Make your first call.

In late training the "generator" here literally samples from the real distribution — that is what a perfect generator means. Your long-run accuracy mathematically cannot beat 50%.

Feel the asymmetry: early on, detective work is easy — and that easy gradient is what teaches the generator. Late in training, even a perfect detective is reduced to guessing. Your own accuracy collapsing toward 50% is the success metric of the other player. In a GAN, one network's failure is the other's loss function.

EP 03

Step the duel — watch both players improve each other

Now the full loop, automated. Each step: the detective (a logistic classifier) trains a little on real-vs-fake, then the generator moves its forgery cloud along the detective's own gradient — literally using the enemy's brain as a compass. The gold shading is the region the detective currently calls "real"; the dashed line is its decision boundary.

Simulated dynamics: a real GAN moves millions of weights, not a cloud's center — but the alternating "D learns, then G climbs D's gradient" loop is exactly this.

Why it matters: nobody ever tells the generator where the real data is. It only ever sees the detective's opinion — and that is enough to walk the fakes right into the real distribution. This indirect supervision is the trick that made GANs generate faces, art and audio without any "correct answer" labels.

EP 04

Break it — dead gradients and the oscillation trap

Same duel, two classic ways to ruin it. Overtrain the detective: it becomes so confident that its gradient flattens to nothing — the forger gets no signal and freezes far from the data. Or crank the learning rate: the forger overshoots the real data, the detective flips sides, and they chase each other forever. The strip at the bottom charts the forger's distance to the real data — flat is frozen, zigzag is oscillating, decaying is healthy.

G learning rate ×1.0

Press Run. Try: balanced at ×1 (converges), balanced at ×5 (oscillates), then Overtrain the detective (gradient dies).

The real-world scar tissue: "discriminator too good → generator gradient vanishes" is why the original GAN paper already swapped in a non-saturating loss, and why WGAN redesigned the critic entirely. The oscillation you just caused is why GAN training curves look like seismographs and why practitioners babysit learning rates for both players separately.

EP 05

The moving target — why a duel is not a descent

Strip the duel to its skeleton: forger picks number g, detective picks number d, and they fight over the score g × d (forger minimizes, detective maximizes). The only equilibrium is the center. Ordinary gradient steps don't descend a valley here — each player's move rotates the other's landscape. Watch the trajectory.

Update rule Step size 0.20

This g·d game is the standard minimal model of adversarial optimization — every GAN inherits its rotational force, plus worse.

The punchline: naive simultaneous gradient steps provably spiral away from the equilibrium of even this toy game — no bug, pure math. That's why GAN training is famously unstable and why fixes (extragradient, two-timescale rules, spectral norm) are about taming the spiral, not finding a better valley. Ordinary training minimizes; adversarial training orbits.