Fifty Evaluations to Find the Best: Bayesian Optimization
Gaussian process surrogates, acquisition functions derived by hand, and the 2024–2026 state of the art — every paper verified on arXiv
This is a deep dive into Bayesian optimization — how to find the best value of an expensive black-box function in a few dozen evaluations. It was built from a fresh literature survey (every arXiv paper cited is API-verified), and it is meant to be worked through, not skimmed: each part ends with a hard five-question quiz, the math parts carry faded derivations to attempt on paper, and the simulations ask you to predict outcomes before running them.
Read the parts in order. If you already know why gradient descent doesn’t apply to black boxes, start at Part 2; if you want only the story without equations, Parts 1, 2 and 8 stand on their own. The interactive GP playground in Part 2 is the heart of the page — everything before it motivates it, everything after it formalizes it.