Is this the same as representation bias?

The concepts overlap, but uncertainty bias emphasizes how prediction uncertainty can favor better-represented groups.

Does a confidence score measure uncertainty reliably?

Not necessarily; its interpretation requires evaluation.

Should every group have identical deferral rates?

That is not a universal rule. Assessment depends on the task, consequences, and alternatives.

AI Glossary / Governance, ethics and safety / Advanced

Uncertainty Bias

Definition

Uncertainty bias, in the supplied Artificial Intelligence Ontology sense, occurs when a predictive system favors groups represented more strongly in its training data because predictions for them involve less uncertainty. The concern is how unequal information and the use of uncertainty can translate into unequal treatment or access to benefits.

What it means

Uncertainty itself is not necessarily a mistake. A model may reasonably be less certain about situations for which it has little useful evidence. The problem arises when the surrounding decision process turns that information gap into a persistent disadvantage without adequately examining or addressing it.
For example, a service might offer an immediate answer only when confidence exceeds a threshold. If uncertainty is higher for a poorly represented group, its members may receive more deferrals. Whether that is beneficial or harmful depends partly on what happens after deferral.
This usage is distinct from a general preference for certainty in human decision-making. It also differs from statistical bias, which can describe an estimator's expected deviation from a target.

How it works

Training coverage influences what a model can learn, although raw sample counts alone do not determine uncertainty. Data quality, relevant variation, modeling assumptions, and differences between training and use also matter.
A decision rule then uses the model's output or confidence. It may prioritize familiar cases, reject uncertain cases, or send them elsewhere. This can create different service experiences across groups even when the same numerical threshold applies to everyone.
Feedback can reinforce the pattern if the system mainly collects useful outcomes for cases it already accepts. Neither feedback nor disadvantage follows from every uncertainty-based rule.
Research on selective classification shows that abstaining on uncertain predictions can improve average accuracy while worsening some group disparities. The practical assessment must examine both predictions and abstentions, including the quality of any alternative service.

Three practical examples

A multilingual help system has much more training material for one language than another. Its confidence rule offers immediate answers more often in the well-represented language. The team examines whether users sent elsewhere actually receive timely assistance.
A learning platform confidently recommends courses to users whose histories resemble its training data. New users with unfamiliar backgrounds receive fewer suggestions. The organization investigates whether limited evidence is being treated as evidence of limited suitability.
A document tool routes uncertain cases to specialists. That routing could improve outcomes if specialist support is effective. If the queue is poorly resourced, the same policy could instead impose repeated delays on users whose documents are less familiar to the model.

Think of it this way

Think of a guide who knows one neighborhood well and another poorly. Being honest about that difference is useful. Offering help only in the familiar neighborhood, without improving knowledge or arranging another source of help, can leave the less familiar neighborhood underserved. The issue is the consequence of the information gap, not honesty about uncertainty.

Worked example

Consider a constructed support evaluation with 100 requests from Group A and 100 from Group B. Assume that unequal training coverage contributes to lower model confidence for B. A common threshold produces automatic answers for 90 A requests and 50 B requests; the remaining requests are deferred.
Automatic-answer coverage is 90% for A and 50% for B, a difference of 40 percentage points. Suppose 81 of A's automatic answers and 45 of B's are correct. Accuracy among answered requests is 81/90 = 90% and 45/50 = 90%.
Equal accuracy on answered requests therefore coexists with very different coverage. Across all submitted requests, correct automatic answers account for 81% of A requests and 45% of B requests. These describe automatic service, not final resolution.
If deferred requests receive effective assistance, the final outcome gap may be smaller or different. If they receive none, the consequences are more concerning. The example specifies a possible mechanism; observed coverage differences alone would not prove that mechanism in real data.

Illustrative case study

Fictional company Cedar Help introduces a confidence threshold for its answer assistant. Its dashboard reports high accuracy among automatically answered questions. Staff later notice that requests written in less common formats are frequently deferred.
The team separates coverage from conditional accuracy and follows deferred requests through the review process. It finds that some users wait longer because reviewers lack suitable reference material. The team investigates the training and evaluation coverage of those formats, improves the support route, and tests the revised workflow.
The company retains uncertainty reporting while reconsidering what the system does with that information. This fictional case illustrates investigating unequal service rather than simply suppressing uncertainty or lowering every threshold. It does not imply that those changes are sufficient for all settings.

Common misconceptions and mistakes

“Uncertainty is the bias.” Uncertainty can be legitimate; the concern is the mechanism and resulting disadvantage.
“More examples always solve it.” Additional data must be relevant and useful, and the workflow may also need changes.
“A common threshold guarantees equal service.” Confidence distributions and deferral outcomes can differ.
“Abstaining is always harmful.” An effective alternative may protect users from errors.
“Equal accuracy among answered cases proves fairness.” It can conceal differences in coverage and downstream outcomes.

Business use

Evaluate who receives an automatic decision, who is deferred, and what happens next. Compare relevant groups using sufficient evidence, and report uncertainty in the evaluation itself when sample sizes are small.
Investigate the cause before choosing a remedy. Potential improvements include better data coverage, more appropriate models, improved uncertainty estimates, and better alternatives for deferred cases. Test whether a change improves the complete service rather than merely its headline accuracy. Document unresolved limitations and avoid interpreting limited evidence about a person as evidence against that person.

Check your understanding

What does the worked example's equal 90% accuracy conceal?

Answer: A 40-percentage-point difference in automatic-answer coverage.

Why follow deferred cases?

Answer: Final outcomes depend on the alternative process.

What further evidence would establish the cause of a real disparity?

Answer: Evidence connecting data coverage, model uncertainty, decision rules, and outcomes.

Related terms

Uncertainty
Representation Bias
Sampling bias
Calibration
epistemic uncertainty
selective classification
abstention
fairness assessment

Learn next

Map an uncertainty-based decision rule. Record its coverage, conditional accuracy, and downstream outcomes for relevant groups. Identify which observations would support or challenge the proposed explanation for any differences.

Uncertainty

Sources

Artificial Intelligence Ontology, Uncertainty Bias: https://raw.githubusercontent.com/berkeleybop/artificial-intelligence-ontology/main/aio.owl NIST SP 1270, Towards a Standard for Identifying and Managing Bias in Artificial Intelligence: https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.1270.pdf Jones and colleagues, Selective Classification Can Magnify Disparities Across Groups, ICLR 2021: https://arxiv.org/abs/2010.14134

Previous: Uncertainty Next: Uncomfortable science