Part 6 of 9

The Maths IV: Inside the SOTA Methods

Four ideas that separate a 2026 model from the 2020 template. Each is one equation changed — a routing rule, a cache layout, or a training objective — and each is shown here as that equation rather than as a description of what the paper claims.

1. Mixture of experts: parameters without proportional FLOPs
2. Shrinking the KV cache: MQA, GQA, MLA
3. DPO: preference learning without the RL
4. GRPO and verifiable rewards
5. Quiz: 5 questions