dLLM Cheat Sheet

Every formula from the diffusion-language-models explainer, with its punchline

companion to the Inception Labs papers explainer · 2026-08-02

The One-Pager

The two regimes

Typewriter (AR): N tokens = N sequential passes, left→right, write-once, memory-bandwidth-bound.

Editor (dLLM): k parallel refinement passes, k ≪ N, bidirectional context, revisable. Fewer passes than tokens is the entire speed story.

Continuous forward process

q(x_t | x₀) = 𝒩( √ᾱ_t·x₀, (1−ᾱ_t)·I )

One-jump corruption to any noise level; variance-preserving. Training never simulates the chain.

Training loss (DDPM)

L_simple = 𝔼 ‖ ε − ε_θ(x_t, t) ‖²

“Guess the noise I added.” The ELBO's Gaussian KLs, collapsed and unweighted.

Score link

s_θ(x_t) = −ε_θ(x_t, t) / √(1−ᾱ_t)

Denoising = gradient ascent on log-density. Bridge to the SDE view.

DDIM update (σ = 0)

x̂₀ = x_t − √(1−ᾱ_t)·ε_θ√ᾱ_t; x_t−1 = √ᾱ_t−1·x̂₀ + √(1−ᾱ_t−1)·ε_θ

Deterministic, same marginals, same network — so you can skip steps. Step count is a dial, not architecture.

Discrete forward process

q(z_t | x) = Cat( α_t·x + (1−α_t)·m )

Noise for text = masking. Absorbing mask state; independent per position.

Reveal posterior

P(reveal) = α_s − α_t1 − α_t

Masked tokens reveal (to the true token) with this probability; visible tokens carry over with probability 1 — the frozen-token theorem ReMDM later breaks.

MDLM objective

L_∞ = ∫₀¹ −α′_t1−α_t · (BERT loss at level t) dt

Fill-in-the-blank, integrated over all masking ratios, is a bound on log-likelihood. Linear schedule ⇒ weight 1/t. Value is schedule-invariant.

Mean-field error

Tokens committed in one pass are sampled independently from per-position marginals (“hot cream”). Fewer steps ⇒ more simultaneous commits ⇒ more error. The step dial trades coordination for speed; T→∞ is exact.

The five dLLM fixes

MDLM: clean objective, likelihoods competitive.
Block Diffusion: AR across blocks — variable length + KV cache.
ReMDM: marginal-preserving remasking — revision, no retraining.
Guidance: (p_cond/p_uncond)^γ per position — steering.
d1: SFT + diffu-GRPO (one-step masked log-probs) — reasoning.

The foundations

DDIM (2020): few-step sampling. Decision Transformer (2021): RL as sequence modeling. FlashAttention (2022): exact attention, IO-aware tiling. DPO (2023): r = β·log(π/π_ref) — alignment without a reward model.

Mercury, in one breath

Inception Labs' commercial dLLM family: MDLM training + Block Diffusion caching + ReMDM revision + guidance + d1-style reasoning, on FlashAttention-class kernels — claiming ~5× AR speed. Frontier quality remains the open bet.

Papers: DDIM · Decision Transformer · FlashAttention · DPO · MDLM · Guidance · Block Diffusion · ReMDM · d1 — full story: the explainer · drills: review deck