Raghuram Shankar
← Diffusion Language Models

Part 2 of 8

Intuition

The core idea that makes diffusion work for text is almost embarrassingly simple once you see it: for discrete tokens, the analogue of noise is masking.

  1. Noising text = hiding it
  2. The catch: parallel commits are independent commits
  3. Bidirectional sight
  4. The editor: fixing your own mistakes
  5. Head-to-head
  6. Quiz: 5 questions