Attention Sinks, Playable

EP 01

The heatmap nobody expected — follow the mass

Below: attention maps for six heads processing a sentence, patterned after what interpretability tools show on real models. Some heads do honest work — attending to a nearby noun, tracking syntax. But look at the left column of the map: several heads dump most of their probability on token #1, regardless of what it is. Swap the first word for something meaningless and… nothing changes.

✓ good case: the syntax heads — mass lands on genuinely related tokens✗ bad case: heads 2, 4, 5 — over half their budget on a token that means nothing (swap it and check)

—

Simulated head patterns calibrated to the published phenomenon (Xiao et al., 2023): sink mass stays high no matter which token sits at position 0 — that indifference is the fingerprint.

The first clue that something structural is going on: if heads "liked" the first token for its content, changing the content would change the attention. It doesn't. Position 0 isn't popular — it's being used for something else entirely.

EP 02

Why it forms — softmax must spend 100%

Attention weights are a probability distribution: they must sum to exactly 1. But sometimes a head has nothing useful to say for this query — ideally it would attend to nothing. There is no "nothing." So where does the forced budget go? To the one token that is visible from every position, always present, and safely bland: token 0. Drag the relevance slider down and watch the mass drain into the sink. Then flip the "allow sum < 1" switch — the black hole evaporates.

✗ bad case: zero relevant context + softmax constraint — 92% of the budget lands on position 0✓ good case: same scores with an escape valve (sum allowed < 1) — sink mass collapses to ~5%

how relevant is the context? 0.8 —

The escape valve is real research: "off-by-one softmax" / registers add a virtual token so heads can abstain. The demo computes both normalizations from the same scores — the sink is pure bookkeeping.

The sink is where "no comment" lives. Softmax forces every head to vote with its whole budget every time; position 0 became the abstention box. That's why it's learned so consistently across models — the constraint is universal, so the workaround is too.

EP 03

Break a streaming LLM — evict the black hole, watch it die

Infinite chat needs a finite KV cache, so the obvious plan is a sliding window: keep the most recent 512 tokens, evict the oldest. Run the stream below. Everything is fine… right until the moment position 0 slides out of the cache. The perplexity trace goes vertical — the model's heads just lost their abstention box, and the forced probability sprays across random recent tokens, corrupting every layer above.

✓ good case: generation while token 0 is still cached — flat, boring perplexity✗ bad case: one token later — the sink is evicted and the output degenerates within a few steps

—

Simulation of the StreamingLLM paper's headline experiment: sliding-window Llama perplexity explodes precisely at first-token eviction. The cliff's location here is computed from the window arithmetic, not scripted by hand.

This is why "just use a sliding window" failed for years — and why the failure looked so mysterious: the model wasn't missing old information, it was missing old infrastructure. The most important token in the cache was the one with no content at all.

EP 04

The four-token fix — keep the drain, slide the rest

StreamingLLM's fix is almost comically small: never evict the first k tokens (keep them pinned forever) and slide the window over everything else. Drag k: at 0 you get EP 03's crash; at 4, the stream runs forever on flat perplexity — same memory budget, same window, plus four pinned tokens. Then try the deluxe version: a dedicated learned sink token trained into the model from day one.

✗ bad case: k=0 — collapse at the eviction cliff, every time✓ good case: k=4 pinned sinks — 10,000 tokens streamed, perplexity flat as a table

pinned sink tokens k 0 —

The cache diagram shows exactly what's held: gold pins + sliding grey window. Perplexity dynamics follow the paper's findings — k=4 recovers dense-attention quality within a small margin; k=1 mostly works if the model used a fixed <bos>.

Four numbers in a cache-eviction policy — that's the whole patch that turned LLMs into infinite streamers (live captioning, day-long assistants, log monitors). And the deeper design lesson landed industry-wide: newer models train dedicated sink/register tokens from scratch, giving the black hole an official job title.

EP 05

Sinks are everywhere — quantization pain and vision "registers"

Once you know the shape, you see it everywhere. Two famous sightings: (1) sink tokens carry enormous activation values — the outliers that make int8 quantization of LLMs miserable (toggle the histogram). (2) Vision transformers grow the same organ: high-norm "ghost" patches in empty background regions of images — sinks wearing camouflage. The fix there got a name: registers, extra tokens added purely to absorb this traffic. Toggle them and watch the feature map clean up.

✗ bad case: no registers — the ViT feature map has garbage high-norm patches sitting in blank sky✓ good case: +4 registers — artifacts migrate into the registers, the map over the image goes clean

—

Patterns follow the published findings (massive activations on sink tokens; Darcet et al.'s "Vision Transformers Need Registers"). The feature maps here are schematic reconstructions of those figures, generated per toggle.

The sink is a general law of softmax architectures: give a network a forced budget and no abstention, and it will invent a landfill — in text, in images, in audio. The mature response isn't to fight it but to zone it: build the landfill on purpose, put it outside town, and keep it off the maps that matter.