Meanwhile, in image generation: diffusion

While language modeling settled on the typewriter, image generation took a completely different path. Systems like Stable Diffusion and Midjourney are diffusion models. The recipe: take a real image and gradually corrupt it with random noise over many steps until nothing but static remains (the forward process). Then train a network to undo one step of that corruption (the reverse process). To generate, start from pure static and repeatedly denoise:

pure noisestatic

→

vague shapescomposition emerges

→

cat-like blobstructure

→

sharp cat photodetails

Coarse-to-fine: every pixel is updated in parallel at every step. The image gets globally better each pass, rather than being painted left to right.

Here is that forward process live, on an actual (very small) image. Drag the corruption level yourself:

Notice what diffusion gives you that the typewriter cannot: the whole output exists (in rough form) from the very first step, every step refines everything at once, and the number of steps is a dial you can turn — a handful of steps for a fast draft, more for higher quality. Early diffusion models needed ~1,000 denoising steps; the 2020 DDIM paper (the “Diffusion Models” link on Inception’s page, co-authored by Ermon) showed how to restructure sampling so that a few dozen steps suffice — using the same trained network, no retraining. That obsession with cutting sampling steps is the company’s DNA.

The obvious question: if diffusion is so good for images, why not text? The blocker is that images are continuous — you can add a little Gaussian noise to a pixel value. Text is discrete: token #4711 is “cat”, and there is no such thing as “cat plus 0.1 noise”. Finding the right notion of noise for text is where the intuition section picks up.

← Autoregressive generation: the typewriter Quiz: Background →