Part 9 · The Papers & Sources
The foundations
Vaswani et al. (2017) 1706.03762 removed recurrence entirely, which is what let sequence models saturate an accelerator; the 1/√dk derived in Part 4 is their footnote. Brown et al. (2020) 2005.14165 showed that scale alone turns a next-token predictor into a system you can instruct without gradient updates. Sennrich et al. (2015) 1508.07909 brought BPE from compression into NLP, ending the unknown-word problem, and Kudo & Richardson (2018) 1808.06226 made it language-agnostic and reversible.