Does the model need to retrain continuously?

No. Periodic updates can create the same loop when later training data were shaped by earlier system decisions.

Is more feedback always a solution?

No. More observations collected through the same selective process can reinforce the distortion rather than reveal missing alternatives.

Does using click-through rate eliminate the problem?

Not necessarily. It accounts for exposure counts in one way, but position effects, user selection, and other confounding factors may remain.

AI Glossary / Governance, ethics and safety / Intermediate

Feedback Loop Bias

Definition

Feedback Loop Bias occurs when a system's outputs influence the behavior or observations that later become its learning data, reinforcing a distorted pattern. For example, recommendations affect what users see, and the resulting clicks may then be treated as independent evidence of preference. The concern is not feedback itself, but failing to account for how the system helped create the evidence it subsequently learns from.

What it means

A deployed model does not always observe an unchanged world. It may decide which items are shown, which cases receive review, or which options are available. Those decisions affect what outcomes can be recorded.

If the next training round treats the recorded outcomes as though every alternative had an equal opportunity to be observed, the model can strengthen an initial preference or blind spot. Items shown less often produce less feedback, while heavily exposed items accumulate more evidence.

The loop can operate through periodic retraining as well as live online updates. A model need not change after every interaction for its earlier decisions to shape the dataset used by a later version.

How it works

The cycle begins with a model or rule influencing an action, such as displaying a recommendation. People respond within the options they encounter, producing clicks, purchases, ratings, or other recorded outcomes. These observations are collected for evaluation or retraining.

Bias can arise when the learning process overlooks the exposure policy or other selection effects. The updated model then favors patterns partly produced by the earlier system, and its new actions generate similarly selected data.

Investigating the loop requires tracing which outcomes could be observed and why. Useful evidence may include exposure records, model versions, decision logs, and appropriately designed comparisons. Corrective methods depend on the setting; simply retraining more often can amplify the same problem if the data-generating process remains unexamined.

Three practical examples

1. A recommendation engine repeatedly promotes already visible items because their larger click totals are mistaken for stronger underlying preference.

2. A review-prioritization model learns mainly from cases it previously sent to reviewers, leaving little evidence about cases it consistently skipped.

3. A help assistant's suggested questions influence what users ask, and the resulting question logs are later treated as a complete picture of what users originally needed.

Think of it this way

Think of a shopkeeper placing one product at eye level, observing that it sells more, and concluding that it should receive even more prominent placement. The placement helped produce the sales evidence used to justify the next placement.

The analogy does not mean popularity is always artificial or that visible products are never better. It shows why observation and intervention must be distinguished. A real investigation needs evidence about alternative explanations and what customers would have encountered under different arrangements.

Worked example

Suppose two fictional help articles, A and B, each receive a click from 10% of users who see them under otherwise comparable conditions. The system shows A 1,000 times and B 100 times. Expected clicks are therefore 100 for A and 10 for B.

A retraining rule based only on total clicks might infer that A is ten times as attractive and give it still more exposure. Yet the stated click-through rates are equal: 100/1,000 = 10% and 10/100 = 10%.

This toy example isolates an exposure-count effect. Real click-through rates can also differ because of position, user selection, and presentation, so dividing by impressions does not automatically remove all bias. The calculation shows why raw counts are insufficient evidence of intrinsic preference, not a complete causal correction or a claim about real users.

Illustrative case study

Fictional case: Fern Library recommends learning resources. Staff notice that new resources receive little exposure and therefore almost no ratings, while older resources dominate the next model's training data.

The team records which resources were displayed, their positions, and the recommendation policy that selected them. It designs a limited evaluation with reviewed alternatives to learn whether the apparent popularity pattern reflects user value or repeated exposure.

This fictional response investigates the feedback mechanism rather than deleting popular resources. It makes no claim that every concentration of attention is harmful or that one experiment resolves all long-term effects.

Common misconceptions and mistakes

Do not equate all learning from user feedback with bias. Feedback can be valuable when its collection and interpretation suit the task.

Do not treat missing outcomes as negative outcomes without justification. A person cannot click an item they were never shown, and an unreviewed case may have an unknown rather than a clean result.

Avoid assuming a static offline test reveals all deployment effects. The system may change future behavior and observations. Also avoid claiming a feedback loop proves deliberate manipulation; the pattern can emerge unintentionally from ordinary optimization and incomplete logging.

Business use

Feedback-loop analysis matters for recommendations, prioritization, personalization, and any AI workflow that shapes its own future evidence. Without it, a dashboard can show improving prediction of the system's selected observations while missing declining coverage or user value.

Document how actions influence data collection, retain relevant exposure information, and evaluate outcomes beyond short-term engagement where the task requires it. When changing the model, consider whether historical feedback was generated under a different policy. A new learner may inherit distortions from earlier decisions even if its own training procedure is technically correct.

Check your understanding

One item has ten times as many clicks but also ten times as many impressions. Does that establish a stronger user preference?

Answer: No. The click-through rates may be equal, and other exposure conditions can also matter. Raw click totals alone do not separate preference from the system's display decisions.

Related terms

Selection bias
Online learning
Retraining
Concept drift
Causal inference
Feedback loop
exposure bias
recommendation system
performative prediction

Learn next

Learn exposure bias and selection bias next, then map one application's action-to-observation cycle. Identify which outcomes are missing because of previous decisions before selecting an evaluation or correction method.

Selection bias

Sources

Artificial Intelligence Ontology, FeedbackLoopBias class: https://raw.githubusercontent.com/berkeleybop/artificial-intelligence-ontology/main/aio.owl

Chaney, Stewart, and Engelhardt, How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility: https://arxiv.org/abs/1710.11214

The exact ontology definition and primary paper's abstract were read. The paper reports simulation findings, not a universal outcome for all recommenders. The exposure arithmetic, workflow guidance, and Fern Library case are original teaching illustrations.

Previous: Feedback loop Next: Feedback Network