ControlNet, Playable

EP 01

The composition lottery — generate twice, get two universes

Here's a generator running the prompt "a figure made of burning embers". Click Generate a few times with no control map: every run scatters the composition somewhere new — the prompt says what, never where. Then attach the pose skeleton and generate again: the layout locks. The number below is measured live by comparing each run's layout to the previous one.

layout overlap vs previous run: —

Overlap is the cosine similarity of a 12×6 occupancy grid between consecutive runs — 100% means the composition landed in the same place twice.

Why it matters: this lottery is why "generate a character in THIS exact pose" was nearly impossible with prompts alone. ControlNet bolts a trainable copy of the diffusion U-Net onto the model that injects the structure map at every layer — turning composition from dice-roll into contract.

EP 02

Draw the bones, pick the skin

Your turn to be the control map. Sketch anything on the canvas — a stick figure, a letter, a spiral — then choose a "prompt" and hit Generate. The particles assemble along your strokes while the prompt decides everything else: color, glow, texture. Re-generate with a different prompt and the same sketch: bones survive, skin changes.

Prompt skeleton points: 0

Conceptual simulator — no trained model here. Real ControlNet conditions a U-Net on your sketch; this toy conditions a particle renderer on it, which is the same contract: structure from you, appearance from the prompt.

The division of labor: the control map carries the low-frequency decisions (where things are), the prompt carries the high-frequency ones (what they're made of). That split is why artists love ControlNet — you keep authorship of composition and outsource the rendering.

EP 03

One pose, three prompts — structure survives the style

The killer demo of ControlNet: hold the skeleton constant, swap the prompt. Left to right, the same dancing figure rendered as embers, wireframe and fireflies — pose agreement across the panels is measured live. Then press the second button: identical three prompts, control removed. Same style, three unrelated compositions.

pose agreement across panels: —

Agreement = average pairwise similarity of the three panels' occupancy grids. High means the three "images" share one composition.

Why it matters: this is exactly how consistent characters, storyboards and pose-matched product shots get made with Stable Diffusion + ControlNet: one OpenPose skeleton, N prompts, N styles — one composition. Without the control input, each prompt re-rolls the universe.

EP 04

The strength knob — from ignored to strangled

Real ControlNet ships a conditioning scale. Drag it. Near 0 the skeleton is a suggestion the model ignores. Around 1 you get the sweet spot: pose obeyed, texture alive. Push past ~1.5 and the control signal overpowers generation — particles clamp rigidly onto the lines, the style's randomness gets crushed into banding, and duplicates pile up on identical pixels. All three numbers are measured from the particles on screen.

Control strength 1.00

pose fidelity: —style texture: —stuck duplicates: —

The trade-off is real: in Stable Diffusion, conditioning scale ≈ 1 follows the map while letting the model shade, light and vary; crank it to 2 and you get embossed line-art artifacts — the control literally overwrites what the prompt wanted to paint. Strong guidance is not free; it spends the model's freedom.

EP 05

Tug-of-war — the skeleton says standing, the prompt says sitting

What if the two masters disagree? The control map (gold dashes) is a standing figure; set the prompt to "a person sitting down" (grey dashes) and slide the control weight. At either extreme one side wins cleanly. In the middle, every joint is pulled toward two places at once — knees bend halfway, the spine splits the difference, and the tension meter (red particles) spikes. Switch the prompt to "standing" and the war ends at any weight.

Prompt Control weight 0.50

anatomy tension: —

The lesson every ControlNet user learns: control and prompt must tell the same story. When they conflict, the model doesn't pick a winner — it averages, and averaged anatomy is body horror: half-bent knees, doubled limbs, chairs fused into legs. Fix the conflict in the inputs, not the strength slider.