AI Glossary / Governance, ethics and safety / Advanced
Algorithmic aversion
Definition
Algorithmic aversion is reluctance to rely on an algorithm, particularly when people reject its recommendations after seeing it make mistakes despite evidence that it performs better than the available human alternative. The concept concerns how people respond to imperfect decision aids. It does not mean that every objection to AI is unreasonable or that algorithms deserve automatic trust.
What it means
A useful model can lose support after one memorable failure, while familiar human mistakes attract less attention. The important comparison is whether equivalent evidence receives equivalent treatment. A person might forgive a colleague's inaccurate forecast but regard a similar machine error as proof that the entire system is unusable.
However, average accuracy alone cannot settle suitability. Someone may reasonably reject a model because its errors are unusually costly, its data are inappropriate, or its use undermines privacy. Understanding algorithmic aversion requires separating these substantive concerns from an inconsistent standard of performance.
How it works
In influential forecasting experiments, Dietvorst, Simmons and Massey found that observing errors could reduce confidence in algorithms more sharply than confidence in human forecasters. Participants sometimes preferred the worse-performing human option even after observing the algorithm's comparative advantage. These findings describe particular experimental settings, rather than a universal response to every AI application.
A later study series found that allowing participants limited adjustments to algorithmic forecasts increased their willingness to use them. This suggests that some control can matter for acceptance. It does not establish that unrestricted overrides always improve results.
For an organization, the practical investigation starts with comparable evidence: what each option predicted, what happened, and what consequences followed. Surveying confidence alone cannot reveal whether distrust is justified by performance.
Three practical examples
- Stock planning: A buyer abandons a forecasting tool after it misses a holiday spike, although the buyer's own recorded forecasts have larger errors across comparable weeks. The comparison should include the spike's business cost.
- Travel planning: A commuter dismisses a route planner after one delay but continues using a familiar route that is usually slower. Unrecorded concerns, such as accessibility, could still explain that choice.
- Service scheduling: A supervisor rejects automated workload estimates after a visible mistake. Reviewing both estimates and subsequent outcomes helps distinguish disproportionate distrust from a genuine failure to represent local conditions.
Think of it this way
Imagine judging two weather forecasters. One memorable wrong prediction should prompt scrutiny, but it does not by itself show which forecaster is generally more useful. Fair comparison requires the same recordkeeping and the same standards for both. You would also distinguish an inconvenient missed shower from a dangerous missed storm warning.
Worked example
Consider five fictional forecasts measured using absolute error in units. The algorithm's errors are 2, 3, 1, 4 and 10. The human's errors are 5, 6, 4, 7 and 8.
The algorithm's total error is 20, giving a mean absolute error of 20 / 5 = 4 units. The human's total is 30, giving 30 / 5 = 6 units. On this measure, the algorithm reduces average error by (6 - 4) / 6, or approximately 33.3%.
The final case is nevertheless worse for the algorithm: 10 rather than 8. Rejecting it solely because that failure feels unacceptable, while overlooking the human's larger overall errors, would warrant investigating algorithmic aversion.
The calculation assumes that absolute errors are the relevant comparison. If the last case has much greater consequences, a different cost measure could favor the human. Five observations also provide weak evidence about future performance. The example identifies a comparison problem, not proof of irrationality.
Illustrative case study
Fictional retailer Willow Supplies introduces demand forecasts for its planners. One obvious overestimate becomes the main topic of discussion, and several planners return to personal estimates. Management initially treats the reaction as resistance to change.
An analyst records both forecasts before outcomes are known and reviews error by product group. The review finds useful automated forecasts for routine products but poor coverage of promotions. Planners receive a way to flag missing promotion information and record the reasons for adjustments.
The company then evaluates the combined process, including overrides. Its objective is appropriate reliance: use the tool where evidence supports it and investigate its limits elsewhere. The case illustrates a diagnostic approach; it does not claim a measured improvement from a real deployment.
Common misconceptions and mistakes
“People always dislike algorithmic advice.” Responses depend on the task, experience, presentation and available alternatives.
“Any refusal to use AI is algorithmic aversion.” Concerns about suitability, rights, accountability or harmful errors can be well founded.
“One impressive accuracy score settles the issue.” Aggregate performance can conceal consequential failures and unsuitable comparison data.
“Human control automatically fixes the problem.” Overrides can introduce errors or hide systematic problems. Their effect needs evaluation, and users need meaningful information rather than a decorative approval button.
Business use
The concept helps teams investigate disappointing adoption without blaming users. Establish a comparable baseline, review consequential errors, and explain what the tool can and cannot support. Record why people accept or override advice and assess subsequent outcomes.
Success should mean better decisions under appropriate safeguards, rather than the highest possible acceptance rate. A low override rate can reflect excessive trust just as easily as excellent performance.
People also ask
Is this the same as automation bias?
No. Automation bias concerns excessive reliance on automated advice; algorithmic aversion concerns reluctance to rely on it. Both can undermine appropriate judgment.
Does explaining the model eliminate aversion?
Not necessarily. An explanation may address one concern while leaving poor performance or unsuitable data unresolved.
Must users accept the more accurate option?
Accuracy is one consideration. Consequences, constraints and the legitimacy of using the system also matter.
Check your understanding
Why compare human and algorithmic mistakes on the same cases?
Answer: Otherwise, differences in task difficulty can distort the comparison.
Does the worked example prove the algorithm should always be selected?
Answer: No. It compares one metric on five cases under stated assumptions.
What should a team measure after introducing overrides?
Answer: The quality and consequences of the resulting decisions, not merely user satisfaction or tool usage.
Related terms
- Automation bias
- Mean Absolute Error
- Human-in-the-loop
- AI governance
- calibrated trust
- human oversight
- decision support
- model evaluation
Learn next
Study automation bias alongside algorithmic aversion, then examine evaluation measures that reflect unequal error costs. Practice comparing recorded forecasts before considering an intervention intended to increase adoption.
More in Governance, ethics and safety
- AI principles - AI principles are high-level commitments intended to guide how artificial intelligence is developed, deployed and used.
- AI risk management - AI risk management is the ongoing work of identifying, assessing, responding to and monitoring risks associated with developing or using AI.
- AI safety - AI safety is the study and practice of preventing or reducing harm arising from AI systems and their use.
- Alignment - In AI, alignment is the aim of making a system's behavior consistent with intended goals, human values and relevant constraints.
- Alignment method
- Amplification Bias - Amplification bias occurs when processing by an AI system increases a skew or association relative to a stated reference distribution.
Sources
Dietvorst, Simmons and Massey, Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err, original forecasting experiments. https://marketing.wharton.upenn.edu/wp-content/uploads/2020/07/Dietvorst-Simmons-Massey-2014.pdf Dietvorst, Simmons and Massey, Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them, experimental research on limited adjustments. https://faculty.wharton.upenn.edu/wp-content/uploads/2016/08/Dietvorst-Simmons-Massey-2018.pdf The numerical comparison, practical examples and Willow Supplies case are original teaching illustrations.
Previous: Algorithmic amplification Next: Algorithmic inference