Can measuring one option teach us about another?

Yes, when the model links their unknown performances through correlated beliefs or shared parameters.

Must observations be noiseless?

No. Noise is incorporated in the predictive observation and posterior update.

Does KG maximize information entropy reduction?

Not generally. It targets improvement in the value of the final decision, which can differ from reducing uncertainty overall.

AI Glossary / Probabilistic machine learning / Advanced

Knowledge gradient acquisition

Definition

A knowledge gradient acquisition function scores a possible measurement by the expected improvement it would produce in the value of the best final decision. In Bayesian optimization, it commonly measures the expected increase in the largest posterior mean after one additional observation.

What it means

Suppose several alternatives have uncertain performance and only a limited number of experiments are affordable. The next experiment need not be the alternative that currently looks best. It may be an alternative whose result would change which option should ultimately be selected.

Knowledge gradient, or KG, evaluates that learning opportunity. It looks ahead to possible measurement outcomes, updates the beliefs for each outcome, and asks how valuable the resulting best decision would be on average.

The standard criterion is a one-step value-of-information calculation. It is optimal for its stated one-measurement decision problem, but repeatedly using it does not automatically solve every longer-horizon experimental-design problem optimally. Its value also depends on the prior, observation model and final decision objective.

How it works

For a maximization problem, begin with the largest current posterior mean across eligible alternatives. For each candidate experiment, consider its predictive distribution of possible observations.

For each possible observation, update the posterior and find the largest updated mean. Average that best updated value over the observation distribution, then subtract the current best mean. The difference is the knowledge gradient for that experiment.

Correlated beliefs allow one measurement to update several alternatives. Observation noise affects how much can be learned. Costs can be incorporated through a specifically chosen cost-sensitive rule or decision objective. The word gradient here refers to incremental decision value; it does not mean an ordinary derivative of the unknown performance function.

Three practical examples

A simulator allocates additional runs to configurations whose uncertain performance could change the final configuration choice.
A materials experiment selects the next candidate by modeling how its measurement would inform both that candidate and related ones.
A model-tuning study chooses an expensive evaluation by its expected effect on the best configuration recommended after evaluation.

Think of it this way

Think of choosing which envelope to open before selecting a prize. The most interesting envelope is the one whose information could improve your final choice, not necessarily the one with the most unpredictable contents.

The analogy assumes a specified belief about what the envelopes contain and how the information affects the choice. Knowledge gradient does not know their actual contents in advance, and curiosity without decision value is not its objective.

Worked example

Distinguishing useful uncertainty from large uncertainty

Consider an original finite example with a risk-neutral final choice. Alternative A has known value 6. Alternative B has value 2 or 10, each with probability one half. Alternative C has value −100 or 0, also equally likely. Assume B and C are independent and that one measurement reveals the measured alternative exactly.

Before measuring, the posterior means are 6, 6 and −50. The best current expected value is therefore 6.

Measuring B leads to a final best value of 6 if B is revealed as 2, or 10 if it is revealed as 10. The expected best value is 8, so KG(B) = 8 − 6 = 2.

Measuring C never changes the best available value: even its better outcome is below 6. Thus KG(C) = 0. Yet C's variance is 2,500, compared with B's variance of 16. The most uncertain option is not the most useful measurement.

If measuring B costs 0.5 in the same utility units and stopping is allowed, its net one-step gain is 1.5. This discrete teaching example illustrates the KG principle; it is not the correlated-normal calculation used in the cited technical paper.

Illustrative case study

Fictional case: allocating simulation runs

A fictional warehouse team compares several scheduling rules using a noisy simulator. It maintains a probabilistic model of each rule's mean performance and expects related rules to share some behavior.

The team evaluates candidate simulation runs using their expected effect on the final recommended rule. It checks whether the assumed correlations are supported and whether measurement variability has been separated from uncertainty about the mean.

The team compares KG with a simple allocation baseline over repeated simulated studies. It records final selection quality and total computation, rather than claiming success merely because each chosen experiment has a high acquisition score. No operational improvement is assumed in this fictional case.

Common misconceptions and mistakes

Knowledge gradient is not the same as sampling the largest posterior variance. Uncertainty matters through its potential to improve the final decision.

It is also not generally identical to expected improvement. Their definitions can coincide in particular settings, but standard KG explicitly considers the value of the updated recommendation and can accommodate noisy observations and information shared across alternatives.

An expected value of information is not a promise that every realized observation improves the reported best mean. An unfavorable observation can lower it. The nonnegative gross expected gain in the standard unconstrained decision setting comes from having the option to retain a previous decision, not from every outcome being good news.

Business use

KG is relevant when experiments are costly, the ultimate recommendation matters, and the team can maintain a credible uncertainty model. It links data collection to a concrete decision rather than to uncertainty reduction alone.

Specify eligible final choices, observation costs, noise assumptions and the performance quantity being optimized. Include acquisition-computation costs in the experiment budget. Test sensitivity to prior correlations and compare repeated final decisions with simpler strategies. For batch experiments, constraints or multiple remaining stages, use a formulation that actually represents those conditions rather than silently assuming the simplest one-step setting.

Check your understanding

In the worked example, what is the gross KG of measuring the already known alternative A?

Answer: Zero. Its value is already known, and measuring it supplies no additional information under the stated assumptions.

Related terms

Bayesian optimization
Expected improvement
acquisition function
value of information
posterior mean
experimental design
ranking and selection

Learn next

Calculate KG in a small noisy observation model, then introduce correlation between alternatives. Compare the selected experiment with the one chosen by maximum variance and by immediate expected reward.

Sources

Primary technical paper: Frazier, Powell and Dayanik, The Knowledge-Gradient Policy for Correlated Normal Beliefs: https://people.orie.cornell.edu/pfrazier/pub/CorrelatedKG.pdf

Author's broader resource: https://warrenpowell.org/optimal-learning/

The finite three-alternative calculation and scheduling-study case are original teaching examples.

Previous: Knowledge extraction Next: Knowledge integration