Part 2 · Intuition

The next-token game, played by hand

Fix a vocabulary and a context. The model outputs one number per vocabulary entry, and those numbers are turned into a distribution by a softmax. Training nudges the number for the token that actually appeared upward, and the rest down. Generation samples from the result, appends, and repeats.

The sampler is not a detail. The same model is a different writer depending on how you draw from it, and the knobs below are the ones every deployed system exposes. Set the temperature near zero and it becomes deterministic and repetitive; open it up and it becomes surprising and occasionally incoherent. Watch the empirical bars converge onto the theoretical distribution as you draw.