Raghuram Shankar
← Language Modelling

Part 2 of 9

Intuition

Before any equations, three pictures. The Maths sections that follow will formalize precisely these, and nothing else.

  1. The next-token game, played by hand
  2. Attention as soft lookup
  3. What scale actually buys
  4. Quiz: 5 questions