Temperature vs Top-p, Playable

EP 01

Same distribution, two different surgeries

Below is a real next-word distribution from a tiny language model living in this page (context: “… the keeper ___”). Move temperature and watch every bar change height — nothing dies, the curve just flattens or sharpens. Move top-p and watch bars get executed outright — the survivors keep their relative shape. Reshape vs amputate. That's the whole distinction.

Temperature 1.00 Top-p 1.00

Temperature is applied first (it reshapes the softmax), then top-p truncates the reshaped list — the same order real APIs use.

Good case — they compose: because they operate on different things (shape vs support), you can use both: temperature sets how adventurous the model is among survivors, top-p sets who survives. Watch the “effective candidates” readout: T changes it smoothly, p changes it by beheading.

EP 02

The 2D control pad — every API call lives somewhere on this square

Drag the dot. Horizontal = temperature (0 → 2), vertical = top-p (0.1 → 1). The right half of the canvas renders the resulting distribution live, and Generate makes the tiny model write 18 words from wherever the dot sits. The presets are real-world defaults — feel why they sit where they sit.

Drag the dot, then generate.

The model is a trigram model trained on a 150-word lighthouse story — tiny, but it samples with genuine temperature + top-p math, identical to GPT's decoding loop.

Good case: corners have meanings. Bottom-left = obedient and repetitive (good for facts, forms, code). Top-right = wild and diverse (good for ideas). The two bad corners — top-left and bottom-right — are EP 03 and EP 04.

EP 03

Word salad on tap — high temperature with no safety net

Crank temperature to 1.9 and the tail words get boosted until they're nearly as likely as the good ones. With top-p = 1.0 (no truncation) the model now regularly picks words it barely believes in — the text dissolves. Same temperature but clamp top-p down to 0.3 and the salad firms back up, because truncation deletes the boosted tail before the dice roll. (For this episode the model runs in bigram mode — shorter memory means wider distributions, closer to a real 100k-word vocabulary.) Run both and compare the live “desperation picks” counter.

Words the model picked from outside its own top-3 get marked red.

Why it matters: this is the classic “I set temperature 2 for creativity and got gibberish” failure. Temperature alone has no floor — it happily resurrects garbage. Truncation is the floor. That's why creative presets raise both knobs, never temperature alone.

EP 04

Frozen solid — at T=0, top-p is a knob connected to nothing

The opposite corner. At temperature 0 the reshaping is total: the top word gets ~100% of the mass before top-p even looks at the list — so any p keeps exactly one word. Run all three: identical output, and a deterministic path through a finite vocabulary eventually revisits a state and loops forever (underlined in red). Then nudge the temperature and watch the loop break.

Run the three frozen settings — then compare them.

The lesson: the knobs are not symmetric. Top-p can rescue a hot model (EP 03), but nothing rescues a frozen one — at T=0 every other sampling parameter is decorative. If your chatbot repeats itself with top-p carefully tuned, check the temperature first.

EP 05

The preset audition — right knobs, right job

Two jobs, two presets, four combinations. Factual job: complete “the keeper lit ___” — there is one right answer in this model's world, and we score how many of 8 runs get it. Brainstorm job: open a story with “one night ___” — we score how many distinct openings 8 runs produce. Audition each preset on each job and watch the scores flip.

Job:

Pick a job, audition a preset — 8 runs each.

The whole page in one sentence: temperature and top-p don't have good values, they have good pairings with tasks — low/low when one answer is right, high/high when many answers are welcome, and the mismatches (consistent nonsense, or eight identical “ideas”) are failures you just produced yourself.