Three live SOTA threads (2024 → mid-2026)

1. The high-dimensional controversy. Vanilla BO Performs Great in High Dimensions (2402.02229, ICML 2024) and Standard GP is All You Need (2402.02746, ICLR 2025) argue the specialized machinery was never necessary — the √d prior from Maths III does the work. We Still Don’t Understand High-Dimensional BO (2512.00170, Nov 2025) pushes back: wins are benchmark-dependent, explanations don’t hold. Follow-ups continue (2605.20249, May 2026). Genuinely unresolved. 2. Pretrained surrogates. OptFormer (2205.13320, NeurIPS 2022), PFNs4BO (2305.17535, ICML 2023), GIT-BO (2505.20685, 2025): amortized Bayesian inference in a transformer forward pass (the objective is derived in Maths III). 3. LLMs in the loop. LLAMBO (2402.03921, ICLR 2024), evidence-gated LLM priors (2606.01730, 2026), LLM-evolved MOBO algorithms (2607.08791, 2026) — against budget-matched skepticism (2606.21641; 2509.21403). Current verdict: strong for warm starts and priors, not yet a surrogate replacement.

The knowledge-gradient line. Frazier, Powell & Dayanik (2008) introduced the knowledge gradient for discrete alternatives, and Scott, Frazier & Powell (2011) carried it to Gaussian process beliefs over continuous spaces as the KGCP. q-KG (Wu & Frazier, arXiv:1606.04414) gives the Bayes-optimal batch, d-KG (Wu et al., arXiv:1703.04389) adds gradient observations, and the one-shot reformulation in BoTorch (arXiv:1910.06403) is what made KG cheap enough to use by default in a modern library. Frazier's tutorial (arXiv:1807.02811) is the clearest single treatment of where KG beats EI and why.

← The modern toolbox The complete reference list →