Should reviewers always follow AI to avoid selective adherence?

No. That could create overreliance. Review should be based on relevant evidence and the tool's limitations.

Can this happen without deliberate discrimination?

Yes. Prior beliefs can influence scrutiny and interpretation without a conscious intention to be unfair.

Does a group difference in overrides prove this bias?

No. Differences in case mix, advice quality, or available evidence must be considered before drawing that conclusion.

AI Glossary / Governance, ethics and safety / Intermediate

Selective Adherence Bias

Definition

Selective Adherence Bias is the tendency to accept algorithmic advice more readily when it agrees with pre-existing beliefs or stereotypes, while discounting comparable advice that challenges them. The concern is not simply that people sometimes override AI; it is that their pattern of acceptance is influenced by irrelevant prior beliefs rather than consistent evaluation of evidence.

What it means

A reviewer may describe an AI score as objective when it confirms an expectation, then call a similar score unreliable when it contradicts that expectation. The same tool is being assigned different credibility depending on whether its advice fits the reviewer's prior view.
This can undermine the assumption that adding a person automatically corrects algorithmic bias. Human oversight can help, but its effectiveness depends on how people use the advice. Selective adherence is related to confirmation bias and differs from indiscriminate overreliance: the person is not following every recommendation, but following some for the wrong reason.

How it works

The process begins with a prior expectation about a case or group. Algorithmic advice is then interpreted through that expectation. Confirming advice may receive less scrutiny, while conflicting advice faces a higher burden of proof. The resulting decisions can preserve or amplify the original belief.
To investigate the mechanism, compare responses to advice while accounting for its quality, relevance, and other legitimate differences between cases. Controlled studies can vary relevant cues systematically. Operational audits should inspect acceptance, overrides, reasons, and outcomes. A disparity in acceptance rates alone is a signal for investigation, not proof that a specific psychological mechanism caused it.

Three practical examples

A reviewer accepts negative AI assessments of a group they distrust while demanding more evidence for equally supported negative assessments of another group.
A manager adopts demand forecasts that support an existing product preference but dismisses comparable forecasts favoring a competing proposal.
A quality assessor treats an AI warning as decisive for a supplier with a poor reputation, yet overlooks the same warning for a familiar supplier without checking the underlying evidence.

Think of it this way

Think of using a magnifying glass only on evidence that disagrees with your expectations. The evidence supporting your view passes with less inspection. The analogy is limited because different cases can legitimately warrant different scrutiny; the bias concerns inconsistent standards tied to irrelevant beliefs, not every difference in how information is evaluated.

Worked example

In a fictional controlled exercise, reviewers see 40 recommendations consistent with a prior expectation and 40 inconsistent recommendations. The exercise holds recommendation quality and relevant case evidence comparable. Reviewers follow 30 of the consistent recommendations and 18 of the inconsistent ones.
The adherence rates are 30 divided by 40, or 75%, and 18 divided by 40, or 45%, a 30-percentage-point difference. This pattern is compatible with selective adherence under the stated design. It does not establish the effect's size in real operations. Researchers would still examine random variation, the manipulation, alternative explanations, and whether the exercise reflects actual decisions. If evidence quality differed between the two sets, the same raw difference could have a legitimate explanation.

Illustrative case study

At fictional Pine Services, a supervisor notices that staff invoke the AI system's authority inconsistently when reviewing supplier cases. The organization introduces a structured record of the evidence used, the recommendation, and the reason for accepting or overriding it. Reviewers discuss matched training scenarios that challenge prior expectations.
The team evaluates whether the procedure improves consistency rather than assuming that training removes bias. The fictional case illustrates an oversight design question; it does not claim a measured reduction in discrimination or suggest that staff should follow AI recommendations automatically.

Common misconceptions and mistakes

Not every override is biased. A person may have relevant information the model lacks, detect an error, or apply a rule that the tool does not implement. Similarly, accepting advice that matches a prior view is not inherently wrong if the evidence supports it.
Avoid collapsing selective adherence into automation bias. General overreliance and selective acceptance are different mechanisms and may coexist. Research findings also vary by setting and participant group. The original public-sector studies do not justify a claim that every user or profession behaves identically, or that algorithmic advice always intensifies stereotypes more than human advice.

Business use

The concept matters whenever AI advises people rather than making the final decision. Evaluation should include the combined human-AI process, not only model accuracy. Organizations can examine consistent evidence standards, meaningful override reasons, reviewer training, and patterns across comparable cases. The objective is justified reliance: accepting sound advice and challenging unsound advice for reasons that withstand scrutiny.

Check your understanding

A reviewer rejects an AI recommendation because a documented input is wrong. Is that necessarily selective adherence?

Answer: No. A relevant evidence-based correction is legitimate. The concern arises when equivalent advice is accepted or rejected according to prior beliefs or stereotypes rather than comparable evidence standards.

Related terms

Confirmation bias
Automation bias
human oversight
algorithmic advice
motivated reasoning
human-AI interaction
fairness

Learn next

Examine a small set of acceptance and override decisions. Compare the stated evidence standards across cases, including cases that challenge expectations. Identify what information would distinguish a justified override from selective scrutiny before interpreting differences as bias.

Sources

Sources: Artificial Intelligence Ontology, Selective Adherence Bias: https://raw.githubusercontent.com/berkeleybop/artificial-intelligence-ontology/main/aio.owl . Alon-Barkat and Busuioc, Human–AI Interactions in Public Sector Decision Making: https://doi.org/10.1093/jopart/muac007 . Original author preprint: https://arxiv.org/abs/2103.02381 . NIST SP 1270: https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=934464 .

Previous: Selectional restriction Next: Self-attention