Part 8 · The Papers & Sources

The modern toolbox

Beyond-GP surrogates: TPE (Bergstra et al., NeurIPS 2011, ~5k) models p(x|good)/p(x|bad) densities — the engine of Optuna and Hyperopt, natural for conditional, tree-shaped search spaces; SMAC (Hutter et al., 2011) uses random forests for the same reason. Scaling (the equations are in Maths III): TuRBO (1910.01739, NeurIPS 2019) with trust regions; SAASBO (2103.00349, UAI 2021) with sparsity priors; Bounce (2307.00618, NeurIPS 2023) for combinatorial and mixed spaces; qEHVI (2006.05078, NeurIPS 2020) for parallel multi-objective; πBO (2204.11051, ICLR 2022) for expert beliefs. Numerics: LogEI (2310.20708, NeurIPS 2023) fixed EI’s silent underflow and is the modern default. Software: BoTorch (1910.06403, NeurIPS 2020) rebuilt acquisitions on Monte Carlo + autograd; Ax wraps it; Optuna (KDD 2019) ships TPE to everyone; HEBO won the NeurIPS 2020 black-box competition and was modernized in 2607.10669 (July 2026). The application: Shields et al. (Nature 2021) — BO matching-to-beating expert chemists, the template for self-driving laboratories.