Video Temporal Consistency, Playable

EP 01

One scene, two directors — frames that talk vs frames that don't

Both panels below "generate" a 48-frame video of the same scene: a circle, a square, a triangle. Left: every frame is sampled independently — each one is a perfectly plausible still, but nothing links it to the frame before. Right: each frame is conditioned on the previous ones (temporal attention), so objects keep their position, color and identity. Press play.

Conceptual demo: "independent" samples each object anywhere plausible in the scene; "linked" continues the previous frame's motion. Real video models face exactly this choice inside their architecture.

The core problem: per-frame image quality is identical on both sides — every left-hand frame would pass as a good image. Video quality lives between frames. That is why you can't make a video model by running an image model 48 times.

EP 02

The flicker meter — dial temporal attention from zero to full

Consistency isn't on/off — it's a dial. The slider blends each frame between "sampled from scratch" and "continue the last frame". The graph below the video plots the frame-to-frame jump of every frame, measured live. Find the point where your eye stops seeing strobing — then check what the meter says.

Temporal attention strength 0.35

Why it matters: researchers score video models with exactly this kind of metric (warping error, flow consistency). Notice the trap at high strength: perfectly consistent but sluggish — copy the last frame too hard and nothing moves. Real models must be consistent and dynamic at once.

EP 03

The occlusion test — does the ball survive going behind the wall?

Babies learn object permanence at ~8 months. Video models have to learn it too. A ball rolls behind a pillar. A model with temporal memory knows a mustard ball entered, so a mustard ball must exit, on the same trajectory. A frame-blind model sees only the current frame — while the ball is hidden there is zero evidence it exists. Widen the pillar and watch the frame-blind model get worse.

Pillar width 110px

Pick a run. The verdict is computed from what actually happens to the ball.

This is a real benchmark: occlusion tests are used to probe Sora-class models and world models. Early video generators famously made people walk behind poles and come out as different people — exactly the frame-blind failure you just triggered.

EP 04

The attention window — memory costs compute, so how far back can you afford to look?

Temporal attention isn't free: every frame that attends to W past frames pays for W comparisons — so models cap the window. Below, a ball is hidden for L frames. The arcs show which past frames the first frame after the gap can actually see. If every frame it can reach is an occluded one, the ball's identity is unrecoverable. Move both sliders and find the breaking point.

Attention window W 4 frames Occlusion length L 6 frames

The real trade-off: attention cost grows with window size, and video has ~30× more tokens than an image per second of content. This is why most video models attend over a short sliding window — and why anything that leaves the screen for longer than the window comes back wrong.

EP 05

The 60-frame drift — tiny errors compound until the object morphs

Even with temporal links, each new frame copies the last one imperfectly — a 1% error per frame. Alone it's invisible. Compounded over 60 frames it's a new object. Generate the same 60-frame clip twice with identical noise: left conditions only on the previous frame (errors accumulate like a random walk), right also re-attends to frame 1 as an anchor. Scrub the timeline and compare each blob to its grey frame-1 outline.

Frame 1 / 60

Drift is measured live: how far each frame's shape and color have wandered from frame 1.

Why long AI videos are hard: this accumulation is exactly why most generated clips are a few seconds long, and why faces slowly melt in long autoregressive videos. Anchoring to keyframes, or generating all frames jointly instead of one-after-another, is how current models fight the drift.