Curriculum Learning, Playable

EP 01

Hit the wall — train on the hard task and watch nothing happen

The agent must learn how hard to jump across a gap: land on the far platform and it gets a reward; fall short or overshoot and it gets nothing — no hint, no partial credit. It learns only from successful jumps (sparse reward, like most of RL). Train it on the easy gap: successes trickle in, the success curve climbs. Now train a fresh agent directly on the hard gap: it never lands even once, so there is no signal to learn from. The curve isn't slow — it's flat, and it would stay flat for a million epochs.

Pick a task to train on.

This is a real learner (an evolution-strategies-style policy: jump power θ with exploration noise σ, updated toward whatever succeeded), not a scripted animation. Every number and curve comes from its live training run.

The core problem curricula solve: when success is the only teacher, a task you can never succeed at teaches you nothing. Gradient descent has the same disease in disguise — a plateau with no slope. This is why "just train it on the final task" fails for sparse-reward RL, long-horizon reasoning, and robots.

EP 02

The staircase — eight small steps up an unclimbable cliff

Same agent, same 200-epoch budget, one change: the gap starts at 1 and widens by one every 25 epochs, ending at the same gap 8 that was unlearnable in EP 01. Each stage is easy from where the agent already is: today's skill puts tomorrow's task within reach of its exploration noise. Run both and compare the curves — the dips at each stage boundary are the difficulty step-ups, and the recovery after each dip is the transfer.

Both runs get exactly 200 epochs × 20 jump attempts. The only difference is the order of difficulty.

✓ Good case: an impossible task becomes learnable purely through ordering — zero extra compute, zero extra data. This is AlphaStar training against a league of gradually stronger opponents, LLMs pretrained before RLHF, and every RL robot that learned to walk before it learned to run.

EP 03

You are the teacher — design the curriculum yourself

Four stages, 55 epochs each, then a final exam on gap 8. You set the difficulty of every stage. Can you build a ladder to 8? Two rules will emerge if you experiment: the steps must stay within reach of each other, and the last step must actually reach the exam. Try the presets to see a working ramp, a ramp with a fatal leap, and the brute-force syllabus.

Stage 1 2 Stage 2 4 Stage 3 6 Stage 4 8

Design a curriculum and train.

The final exam (success rate on gap 8, measured over 200 fresh attempts) is computed live from the trained agent — there is no hidden script deciding pass or fail.

What you just did is a real research job: curriculum design. Automated versions of it — teacher networks, self-play leagues, difficulty samplers that keep success around 50% — exist precisely because hand-tuning these sliders for a billion-parameter model is somebody's whole PhD.

EP 04

Mind the gap — one oversized step and progress collapses

A two-stage curriculum: warm up on gap 2, then jump straight to a gap you choose. Watch the success curve at the boundary. If stage 2 is within the agent's post-warm-up reach, there's a dip and a recovery. Push stage 2 to 7 or 8 and the curve doesn't dip — it dies: success crashes to zero at the jump and never comes back, because the agent is right back in EP 01's no-signal desert, only now with most of its budget spent. Find the exact difficulty where recovery turns into collapse.

Stage 2 difficulty gap 7

The collapse threshold you find isn't hardcoded — it falls out of the agent's exploration noise: a stage is learnable only while success is still sampleable from current skill.

✗ Bad case: a curriculum is only as strong as its widest gap. This is the failure behind "we fine-tuned on expert data and the model got worse at everything", and behind self-play agents that suddenly face a much stronger opponent and stop improving. Good curricula keep every jump inside the learner's reach — roughly where it still succeeds sometimes.

EP 05

Kindergarten forever — too easy for too long is its own failure

Gaps that are too big kill curricula — but so does the opposite sin. Total budget: 220 epochs. You choose how long the agent stays on the easy gap-2 stage; whatever remains is split across gaps 4, 6 and 8. Stay too long and two things go wrong at once: those mastered epochs teach nothing new (watch the wasted-compute counter), and the agent specializes — its exploration noise anneals away while it polishes an already-perfect skill, leaving it narrower and less able to reach the next stage. Compare a 30-epoch warm-up with a 160-epoch one.

Epochs on easy stage 160 / 220

"Overfitting to easy" here is literal and measurable: the exploration noise σ shrinks while success is cheap — the agent becomes exquisitely tuned to a task that no longer teaches it anything.

The full picture: curriculum design is a corridor, not a dial — too hard and the signal vanishes (EP 04), too easy and you burn budget sharpening a skill you already have. Real systems automate the corridor by promoting the learner the moment success gets boring (~50–80%). If your training curve has been at 99% for a while, you're not training — you're paying for electricity.