Part 1 of 8

Background

Bayesian optimization is what you reach for when every single measurement hurts: finding the best value of a function you cannot see inside, cannot differentiate, and can only afford to evaluate a few dozen times. This page builds it from the ground up — the Gaussian process machinery, the acquisition functions derived by hand, and the 2024–2026 research threads — with every paper verified against arXiv. If you already know why gradient descent doesn’t apply here, skip to the second subtopic.

1. Optimization when you can query, not differentiate
2. The budget problem
3. A brief history: 1964 → 1998 → 2012
4. Quiz: 5 questions