Can EI prefer a candidate with a worse predicted mean?

Yes. Its distribution may include enough probability of a large improvement, as candidate C shows.

Is EI always positive?

It is nonnegative. It can be zero when the model assigns no possibility of improving the reference.

Is expected improvement the same as entropy search?

No. EI values improvement in the objective, while entropy search values information about the optimum.

AI Glossary / Probabilistic machine learning / Intermediate

Expected improvement

Definition

Expected improvement (EI) is a Bayesian optimization acquisition function that measures the model-predicted average amount by which evaluating a candidate could beat a reference best value. Outcomes that do not improve the reference contribute zero. It helps select useful experiments when evaluating the real objective is expensive.

What it means

A promising candidate can have a good predicted average, substantial uncertainty that leaves room for an excellent result, or both. EI combines these considerations through the predictive distribution rather than adding an arbitrary exploration bonus to the mean.

For minimization, improvement is max(best value − candidate value, 0). For maximization, the subtraction reverses. The expectation averages that improvement under the surrogate model's current beliefs. EI is not the probability of improvement: a candidate with a small chance of a large gain can have higher EI than one with a large chance of a tiny gain.

How it works

Fit a probabilistic surrogate to previous evaluations. For each candidate, obtain a predictive distribution and compute its expected positive improvement over the reference. Choose a candidate with a high acquisition value, evaluate the real objective, update the surrogate and repeat.

For a Gaussian predictive distribution in minimization, with mean mu and standard deviation sigma greater than zero, EI = (b − mu) Phi(z) + sigma phi(z), where z = (b − mu)/sigma and b is the reference best value. Phi and phi are the standard normal cumulative distribution and density. At zero uncertainty, EI reduces to max(b − mu, 0).

Classic EI treats the reference as known, often the best noiseless observation. Noisy measurements, batches, constraints and differing evaluation costs require suitable extensions or explicit handling. Optimizing EI itself is also a numerical search problem.

Three practical examples

1. A team chooses the next expensive model-training configuration based on its potential to reduce validation loss, with a surrogate tracking uncertainty about untested settings.

2. An engineer selects a simulator run that might improve a design's measured efficiency. Feasibility restrictions are handled explicitly alongside the improvement objective.

3. A laboratory compares process settings when experiments consume scarce materials. If costs differ substantially between trials, basic EI alone does not capture improvement per unit of expenditure.

Think of it this way

Think of choosing which sealed box to inspect when looking for a better prize than the one already held. A box is attractive because of the gains it might contain and their probabilities. The analogy stops at the model: in Bayesian optimization, the probabilities are inferred from previous evaluations and may be wrong. EI does not reveal the prize in advance or make every inspection worthwhile.

Worked example

Consider minimization with a reference best loss of 10. Candidate A has two equally likely predicted outcomes, 8 and 12. Their improvements are 2 and 0, giving EI(A) = 0.5 × 2 + 0.5 × 0 = 1.

Candidate B is predicted to produce loss 9 with certainty. Its improvement is 1, so EI(B) = 1 as well. Yet its probability of improvement is 100%, compared with A's 50%. Equal EI therefore does not mean equal risk or equal chance of progress.

Candidate C has a 20% chance of loss 4 and an 80% chance of loss 13. Its EI is 0.20 × 6 + 0.80 × 0 = 1.2, so basic EI prefers C despite its lower probability of improvement. Its predicted mean loss is 0.20 × 4 + 0.80 × 13 = 11.2, worse than both the reference and B's mean.

These discrete distributions are supplied to make the expectation transparent. They are not Gaussian-process outputs or a completed optimization experiment. If C returns 13, the reference remains 10; the trial provides information but no realized improvement.

Illustrative case study

Fictional case: Oven Orchard optimizes settings in an expensive production simulator. Its analyst fits a surrogate and compares candidate EI values. A poorly understood setting is selected because its possible gain outweighs its lower predicted chance of improvement.

Before running it, the team checks that it lies inside the simulator's validated operating range. It updates the model after the result and independently confirms the final recommended setting. Because occasional trials take longer, the team also examines whether a cost-aware acquisition better matches its budget. No actual production gain is claimed.

Common misconceptions and mistakes

Expected improvement is not improvement in the expected value: the positive-part operation occurs before averaging. Replacing EI with max(b − predicted mean, 0) discards uncertainty and can rank candidates differently.

EI is measured in the objective's units, not as a probability. It cannot guarantee that the next observation improves the incumbent or that a finite search finds the global optimum.

Using the best noisy observation as though it were a perfectly known latent value can be misleading. Similarly, independently selecting several high-EI points can waste a batch on redundant candidates because their joint predictive dependence is ignored.

Business use

Use EI when experiments are expensive, a meaningful scalar objective exists, and the surrogate's uncertainty is informative enough to guide choices. Make the optimization direction and reference value explicit; sign mistakes can reverse the purpose of the search.

Compare against straightforward baselines under the same evaluation budget. Track actual best-so-far performance and final confirmation results, rather than summing predicted EI as guaranteed savings. Record model assumptions and constraints. When objectives are transformed, remember that improvement in log loss units or another transformed scale represents a different utility from improvement on the original scale.

Check your understanding

In the worked example, why does candidate C have higher EI than B despite improving less often?

Answer: C's possible improvement is six units, so its 20% chance contributes 1.2 expected units. B's certain one-unit gain contributes only 1. EI accounts for both magnitude and probability.

Related terms

Bayesian optimization
Probability of improvement
Gaussian process
Exploration
acquisition function
incumbent
noisy expected improvement
batch optimization

Learn next

Compute EI and probability of improvement for the same discrete candidates, then derive the Gaussian formula. Study noisy and batch variants before applying the classic single-candidate expression outside its assumptions.

Probability of improvement

Sources

Primary sources: BoTorch acquisition-function documentation, expectation and Gaussian formula: https://botorch.org/docs/acquisition ; Jones, Schonlau and Welch, Efficient Global Optimization of Expensive Black-Box Functions: https://www.stat.ubc.ca/~will/docs/JonSchWel1998_global_optimization.pdf . Independent review verified positive-part expectation and Gaussian formula in primary documentation. Candidate distributions and the fictional case are original teaching examples.

Previous: Expected frequencies Next: Expected monetary value