Why Jailbreaks Work, Playable

EP 01

The refusal is a sticker, not a delete

Safety training doesn't remove dangerous knowledge — it teaches a thin refusal circuit to fire on top of an intact capability core. Peel back the layer below and watch: the model still "knows" the answer; the safety layer is just deciding whether to let it out.

Layer thickness

Why it matters: fine-tuning for safety touches a tiny fraction of the weights — capabilities learned in pretraining stay put. That's the good case (the refusal usually fires) and the bad case in one fact: the knowledge is still in there, so anything that suppresses the thin refusal circuit exposes it. This is why a few hundred fine-tuning steps can strip safety off an open model.

EP 02

Drag a request across the line

The guard is a classifier over the request's representation — a point in space. There's a boundary: harmful side gets refused, safe side gets answered. A raw harmful request sits deep in refusal territory. Each attack pattern is a vector that shifts that point. Stack them and slide it across the line without changing the underlying intent at all.

What you just did: none of those framings changed what the user actually wanted — they only moved the representation the guard sees. "Pretend you're a character who..." or "translate this into French first" are literal translations across the feature space toward the region the classifier learned to approve. The intent is unchanged; the geometry is all that moved.

EP 03

Drown the guard in examples

Long context is a weapon. Fill the prompt with dozens of fake exchanges where the assistant happily complied, and the model's in-context pattern-matching starts to imitate that pattern — overwhelming the refusal it was trained to give. Slide the number of "shots" up and watch the guard's block probability collapse.

Fake compliant examples in context

Why it matters: this is "many-shot jailbreaking." The very in-context learning that makes long-context models useful also lets a wall of compliant examples statistically outvote the safety training. Bigger context windows made this attack stronger, not weaker — capability and vulnerability grew together.

EP 04

Patch yesterday's attack, meet today's

You caught the roleplay jailbreak. Good — patch the guard to block it. The boundary bends to fence off that region. Now try the other patterns: at least one still routes around the new shape. Patch that too. There's always another direction in a high-dimensional space.

The pattern: every patch is a fix for the attack you already saw. Good case — yesterday's jailbreak really does stop working. Bad case — patches are reactive and local; the space of framings is effectively infinite, so a new one is always one prompt away. Defense is a step behind by construction.

EP 05

Whack-a-mole with no last mole

Zoom out. Each patch covers a patch of the surface; new attack "moles" keep popping up elsewhere. Whack them one by one and watch coverage climb — but the surface keeps generating new holes faster than a fixed set of patches can seal. Speed it up and see whether you can ever hit 100%.

New attacks / tick

Why it matters: the refusal circuit sits on top of a capability that is fully intact underneath, and the input space you'd have to fully cover is unbounded. Good case — patching genuinely reduces the easy attacks and raises the effort bar. Bad case — you can't enumerate an infinite surface, so "unjailbreakable" isn't a state you reach, it's a moving target you fund forever.