AI Glossary / Statistics and probability / Advanced
Bayesian history matching
Definition
Bayesian history matching is an uncertainty-aware approach to identifying simulator input settings that have not been ruled out as inconsistent with observed data. It commonly uses Bayesian or Bayes linear emulators to screen large input spaces efficiently. The aim is to eliminate implausible regions while accounting for observation error, simulator discrepancy, and uncertainty in the emulator.
What it means
A simulator may be expensive to run and contain many uncertain inputs. Finding one matching setting does not identify all alternatives or validate the simulator.
History matching asks a screening question: which input regions can be rejected because their predicted outputs are too far from observations, even after relevant uncertainties are considered? The remaining region is often called not ruled out yet, or NROY.
That wording matters. A retained setting might predict an acceptable match, or the emulator might simply be too uncertain to reject it. Membership does not supply a posterior probability, establish that the setting is true, or guarantee an acceptable match when the actual simulator is run.
How it works
Begin with a domain of possible inputs and a designed collection of simulator evaluations. Fit and validate an emulator, which provides fast predictions and uncertainty estimates for outputs at untested input settings.
Connect observations to the real system through an observation-error term, and connect the real system to the simulator through a model-discrepancy term. These represent different uncertainties and should not be merged without justification.
For a single output, a common implausibility measure divides the absolute difference between predicted output and observation by the square root of total relevant variance. With zero-mean, mutually uncorrelated terms, the denominator combines emulator variance, observation-error variance, and discrepancy variance. More general structures require appropriate covariance treatment, and stochastic simulators need their intrinsic variability represented.
Reject settings exceeding a chosen implausibility cutoff, then place new simulator runs in the retained region. Successive waves improve the emulator and may rule out further space. Multiple outputs require a stated aggregation rule. Cutoffs, diagnostics, stopping criteria, and the treatment of uncertainty must be documented; a conventional threshold is not a universal posterior credibility guarantee.
Three practical examples
- Engineering simulation: screen combinations of material or operating parameters against measured performance without running the expensive simulator everywhere.
- Environmental modeling: investigate which settings remain compatible with several observed system summaries under explicit measurement and model uncertainties.
- Agent-based modeling: constrain plausible input regions using observed outcomes while accounting for both simulator randomness and imperfect representation of behavior.
Think of it this way
Think of crossing unsuitable locations off a map using imperfect clues. Uncrossed locations still need investigation. Better clues permit further screening, while uncertain clues should not justify confident exclusions.
Worked example
Suppose an observed output is 100. At candidate setting A, an emulator predicts 114 with variance 4. Observation-error variance is 9 and model-discrepancy variance is 3. Assume the three uncertainty components are uncorrelated and use the rule to reject when implausibility exceeds 3.
The total variance is 4 + 9 + 3 = 16, giving standard deviation 4. Candidate A's implausibility is |114 - 100| / 4 = 3.5, so it is ruled out under this specification.
Candidate B predicts 108 with the same uncertainty. Its implausibility is 8 / 4 = 2, so it remains not ruled out yet.
Now reconsider A at an earlier stage when its emulator variance was 16 rather than 4. Total variance would be 16 + 9 + 3 = 28, and implausibility would be 14 / sqrt(28), approximately 2.646. A would then be retained because the emulator was less certain. Reducing emulator uncertainty can therefore justify rejection without changing the predicted mean. It does not mean the underlying simulator setting physically became worse.
Illustrative case study
Fictional case: Moss Engineering uses an expensive airflow simulator with uncertain parameters. Several settings fit one observed pressure reading, but engineers also need compatibility with temperature and flow measurements.
They construct emulators, specify measurement and model-discrepancy uncertainties, and screen the parameter domain in waves. Reserved simulator runs reveal that an early emulator understates uncertainty near one boundary, so the team revises it before relying on exclusions there.
The report describes the surviving region and constraining observations. It does not convert the fraction of surviving points into a posterior probability. Engineers evaluate robustness across multiple compatible settings instead of relying on one apparently best fit.
Common misconceptions and mistakes
Do not confuse history matching with ordinary optimization. Finding a minimum discrepancy and identifying a non-implausible region answer different questions.
Do not call the NROY region a posterior distribution. It is a screened set under stated criteria, not automatically a probability weighting over inputs.
Do not omit model discrepancy merely because a simulator is detailed. Complexity does not imply an exact representation of the real system.
Do not reject points using an unvalidated or overconfident emulator. Once a region is excluded, later waves may never explore it. Conversely, very large uncertainty can retain many unsuitable settings, so retention should not be oversold as demonstrated agreement. Failure to find any surviving region warrants investigation of the simulator, observations, and uncertainty assumptions.
Business use
The method can reduce simulation effort while keeping uncertainty visible, especially when decisions need robustness across multiple parameter combinations compatible with observations.
Business planning should budget for emulator diagnostics, additional simulator runs, uncertainty elicitation, and sensitivity to cutoffs. The output is most useful when users understand what has been excluded, why, and what remains unresolved. If a later decision needs posterior probabilities, an additional justified inferential stage may be required rather than treating the screened region as a completed probability analysis.
People also ask
Does history mean the observations must form a time series?
No. Historical or observed summaries can be compared even when they are not a chronological sequence.
Does a low implausibility prove a good match?
No. It can reflect high uncertainty as well as a close predicted mean.
Can the method use several outputs?
Yes, with a defined rule for combining their evidence and handling relevant dependence.
Check your understanding
Why was candidate A retained with larger emulator variance?
Answer: The prediction was not precise enough to rule it out under the chosen cutoff.
What uncertainty remains even with a nearly exact emulator?
Answer: Observation error, model discrepancy, and any other explicitly relevant uncertainties can remain.
Related terms
- Bayes linear statistics
- Gaussian process
- Sensitivity analysis
- Emulator
- model discrepancy
- implausibility
- uncertainty quantification
- simulator calibration
Learn next
Study emulator validation before implementing repeated screening waves. Then compare history matching with full Bayesian calibration and examine how discrepancy assumptions affect the surviving parameter region.
More in Statistics and probability
- Bayes theorem - Bayes theorem, usually written Bayes' theorem, is a probability identity relating the probability of a hypothesis after observing evidence to its prior...
- Bayesian experimental design - Bayesian experimental design selects an experiment by maximizing expected utility under a probabilistic model of unknown quantities and possible observations.
- Bayesian game - A Bayesian game is a game-theoretic model in which players have incomplete information about relevant features of the interaction, such as another player's...
- Bayesian inference - Bayesian inference is the process of updating probability distributions for unknown quantities using observed data and a specified statistical model.
- Bayesian learning - Bayesian learning is an approach to learning from data that updates a probability distribution over hypotheses or model parameters using Bayes' theorem.
- Bayesian model reduction - Bayesian model reduction is a method for obtaining evidence and posterior quantities for alternative probabilistic models from an already fitted parent...
Work in finance? Get the free CFO AI Prompt Pack: 7 prompts for finance leaders.
Sources
Iskauskas, Vernon, Goldstein, and colleagues, Emulation and History Matching Using the hmer Package, especially Section 2.2 on implausibility and waves: https://www.jstatsoft.org/article/download/v109i10/4587 Williamson and Vernon, Efficient uniform designs for multi-wave computer experiments, author abstract on the not-ruled-out-yet region: https://arxiv.org/abs/1309.3520
Previous: Bayesian hidden Markov model Next: Bayesian inference