Is uncertainty the same as risk?

No. Risk also concerns the consequences and context of possible outcomes.

Can a model be confidently wrong?

Yes. Its confidence estimate may be poorly calibrated or unreliable in unfamiliar conditions.

Does disagreement among models measure all uncertainty?

No. Similar models can share assumptions and blind spots.

AI Glossary / Governance, ethics and safety / Advanced

Uncertainty

Definition

Uncertainty is the presence of unresolved possibilities about an outcome, quantity, explanation, or state of the world given the available information. In AI, it can concern the data, a model, its prediction, or the conditions in which a system operates. Expressing uncertainty helps distinguish what evidence supports from what remains unknown.

What it means

A prediction is often a single answer, but several outcomes may remain plausible. An estimated delivery time of four days, for example, can hide very different possibilities: almost every delivery might take four days, or deliveries might frequently take two or six.
Uncertainty differs from realized error. A cautious prediction can turn out correct, and a confident prediction can turn out wrong. The uncertainty statement describes what was supported before the outcome became known.
The word also covers different questions. Uncertainty about a population average is not the same as uncertainty about one future observation. A useful report says what is uncertain and what its number or range represents.

How it works

A common distinction separates aleatoric uncertainty, associated with variability in observations under the chosen information and model, from epistemic uncertainty, associated with limited knowledge of the model or phenomenon. Additional relevant evidence can reduce some epistemic uncertainty. More repetitions of the same data do not necessarily remove observational variability.
These categories depend on the information available. Adding a useful measurement can make an apparently unpredictable outcome more predictable, so “irreducible” should not be read as independent of every possible change to the problem.
Methods may express uncertainty through probabilities, predictive distributions, intervals, or variation among models. Their interpretation depends on assumptions. A displayed confidence score is not automatically a reliable probability of correctness. Calibration checks whether probability claims correspond to observed frequencies in relevant evaluations; it does not make every individual prediction correct.

Three practical examples

An image classifier receives a blurred photograph. Several labels remain plausible because the input lacks useful detail. A clearer photograph may help more than repeatedly processing the same image.
A demand model encounters a product unlike those in its training data. Its past performance offers limited evidence about this case. The team considers additional data and evaluation instead of assuming that a confident score establishes familiarity.
A planning tool gives both a central delivery estimate and a range of plausible times. A manager considers the consequence of delay when choosing a schedule, rather than treating the central estimate as a promise.

Think of it this way

Think of a weather forecast that distinguishes “rain is possible” from a definite claim. A probability can be useful even though tomorrow produces only one observed outcome. The quality of the probability claim is assessed across relevant occasions, not by demanding that uncertainty disappear whenever one event occurs.

Worked example

Consider a constructed evaluation of 200 binary classifications. For 100 predictions, the system assigns an 80% probability to its chosen class. For the other 100, it assigns 60%.
In the first group, 80 predictions are correct. In the second, 60 are correct. Observed correctness rates are therefore 80/100 = 80% and 60/100 = 60%, matching the reported probabilities in these two groups.
Overall accuracy is (80 + 60)/200 = 70%. The 80% group still contains 20 incorrect predictions. Calibration at this grouping level does not mean certainty about each member, and this finite test does not prove calibration for every subgroup or future condition.
Now imagine a second evaluation in a changed setting. Only 65 of the 100 predictions assigned 80% are correct. The observed rate is 15 percentage points below the claim. The team should investigate the difference, sample uncertainty, and changed conditions rather than assuming that the original result transfers unchanged.
All counts are invented to illustrate interpretation. They are not evidence about a particular product or a universal acceptance threshold.

Illustrative case study

Fictional company Birch Planning introduces a demand assistant that displays one forecast per product. Staff begin using those values as fixed purchasing targets, although the available history differs greatly between established and newly launched products.
The team revises its reporting to explain what the forecast represents and to show relevant uncertainty information. It evaluates the uncertainty estimates on held-out periods and examines performance separately for the product situations that matter operationally.
Buyers use the information alongside the consequences of holding excess stock or running short. The company also identifies situations in which the model's uncertainty estimates have not been validated.
This fictional case illustrates using uncertainty to support judgment. It does not claim that adding a range automatically improves decisions or that the same method remains reliable under every change in demand.

Common misconceptions and mistakes

“A confident output must be correct.” Confidence claims require evaluation.
“Uncertainty means the system is useless.” A well-characterized uncertain prediction can still inform decisions.
“More data removes every uncertainty.” The relevance and information content of the data matter.
“A narrow interval is always better.” Narrowness is unhelpful if the interval fails to represent the stated uncertainty reliably.
“Calibration proves safety.” Consequences, coverage, and other requirements still need assessment.

Business use

Ask what an uncertainty measure means, how it was obtained, and under which conditions it was checked. Distinguish a model-generated score from an empirically evaluated probability claim. Avoid presenting unsupported precision as assurance.
Connect uncertainty to a decision process. Determine when additional information, review, or a different action would be useful, considering the consequences of errors. Monitor whether the evidence supporting uncertainty estimates remains relevant after changes in data, users, or operating conditions.

Check your understanding

Why can 20 predictions in the 80% group be wrong?

Answer: An 80% probability does not promise correctness for every prediction.

What does the second evaluation suggest?

Answer: The earlier probability claim may not match the changed setting and needs investigation.

Why identify the uncertain quantity?

Answer: Uncertainty about an average differs from uncertainty about an individual outcome.

Related terms

Probability
Calibration
Prediction interval
Confidence interval
epistemic uncertainty
aleatoric uncertainty
distribution shift
risk

Learn next

Take a prediction used in a business decision. State precisely what is uncertain, identify evidence for any probability or range, and describe an action that would change if uncertainty were greater.

Probability

Sources

Aalto Dictionary of Machine Learning, uncertainty: https://aaltodictionaryofml.github.io/ADictML_Main.pdf Kendall and Gal, What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?, 2017: https://arxiv.org/abs/1703.04977 Guo and colleagues, On Calibration of Modern Neural Networks, 2017: https://arxiv.org/abs/1706.04599

Previous: Uncertain inference Next: Uncertainty Bias