Optimization when you can query, not differentiate

Most optimization you meet in machine learning is white-box: you have a formula for the objective, you differentiate it, and gradient descent takes millions of cheap, informed steps. Now delete every one of those advantages. The function f you want to maximize is a black box: you choose an input x, something slow and expensive happens — a neural network trains for six hours, a chemist runs a reaction overnight, a wind-tunnel test burns a day of scheduling — and a single number comes back. No formula, no gradient, often some noise.

xhyperparameters · reaction conditions · design

→

black box fhours or days per evaluation

→

f(x)one number (maybe noisy)

The setting: each query is precious, and nothing else about f is visible. The question is not “how do I step downhill” but “where do I spend my next evaluation?”

Classical answers exist. Grid search tiles the space with a lattice of evaluations. Random search draws inputs uniformly. Evolutionary strategies like CMA-ES mutate populations of candidates. All of them share one blind spot: they never build a model of what they have already learned. Evaluation #37 is chosen exactly as if evaluations #1–36 had never happened.

← Background The budget problem →