Does randomization solve it?

Random assignment supports a fair comparison of assigned groups, but restricting analysis to later adherers can reintroduce selection concerns.

Can adjustment remove it?

Adequate adjustment can help under causal and measurement assumptions; unmeasured differences may remain.

Does a negative-control association prove the exact bias?

No. It can reveal concerns but requires its own assumptions and does not automatically quantify the distortion in the main result.

AI Glossary / Statistics and probability / Intermediate

Healthy user bias

Definition

Healthy user bias occurs when people who choose a preventive intervention differ from nonusers in other health-related behaviors or characteristics that also influence the outcome. An observational comparison may then credit the intervention for benefits partly caused by those differences. The core problem is confounding: intervention uptake and outcome share other causes. The term does not mean that all users are healthy, that prevention is ineffective, or that observational evidence should automatically be rejected.

What it means

Choosing to use something can reveal information about the person making the choice. Someone who adopts one health-related service might already follow other routines, have better access to care, or face fewer barriers to participation.

For AI evaluation, a similar reasoning problem arises when enthusiastic adopters of a wellness tool have better outcomes than nonusers. Their prior habits may help explain the difference. This analogy concerns evaluation design; it does not establish that any particular product suffers from the bias.

How it works

Suppose a pre-existing characteristic influences both intervention use and subsequent outcomes. Comparing users with nonusers without adequately accounting for that characteristic mixes its association with the intervention's possible effect.

An analysis should define eligibility, treatment initiation, follow-up, outcome, and the intended causal comparison before looking at results. Researchers can examine baseline differences, measure relevant behaviors, choose a suitable active comparator, and assess sensitivity to unmeasured confounding. A new-user design addresses additional timing and selection issues but is not a universal cure.

The related healthy adherer effect concerns continued adherence rather than initial uptake. These distinctions matter because decisions made after treatment starts can themselves depend on changing health.

Three practical examples

Researchers comparing preventive-service users with nonusers check whether the groups differed in other preventive behaviors before follow-up.
An employer evaluating a voluntary wellness program asks whether participants were already more active before joining.
A developer evaluating an AI habit coach distinguishes outcomes after random assignment of access from outcomes among people who voluntarily became frequent users. Frequency of use may reflect motivation and available time as well as the tool's usefulness.

Think of it this way

Think of comparing the gardens of people who buy a particular watering can with those who do not. Buyers may already spend more time gardening, improve their soil, and remove weeds regularly. Better gardens do not, by themselves, tell you how much improvement the watering can caused. You need a comparison that addresses those prior differences.

Worked example

Construct a fictional population with two baseline profiles, called lower risk and higher risk. Assume the intervention has no effect within either profile. The outcome risk is 2% for the lower-risk profile and 10% for the higher-risk profile, regardless of intervention use.

Among 1,000 users, suppose 800 are lower risk and 200 higher risk. Expected outcomes are 800 x 0.02 + 200 x 0.10 = 36, giving 3.6% risk.

Among 1,000 nonusers, suppose 200 are lower risk and 800 higher risk. Expected outcomes are 200 x 0.02 + 800 x 0.10 = 84, giving 8.4% risk.

The crude risk ratio is 3.6/8.4 = 0.4286, apparently favoring users despite the assumed zero effect. Standardizing both groups to an equal mixture gives 0.5 x 2% + 0.5 x 10% = 6% in each group. This construction isolates one measured confounder; actual populations rarely reduce to two perfectly measured profiles.

Illustrative case study

Fictional case: Lumen Wellbeing finds fewer reported sick days among regular users of its AI coaching service. Its initial dashboard describes the association as a program benefit.

An analyst discovers that regular users also reported more exercise and fewer access barriers before enrollment. The team revises its evaluation to collect baseline measures, define a comparable start date, and separate availability of the service from subsequent adherence.

It presents adjusted observational findings with their assumptions and proposes a randomized rollout where feasible. The company can still describe engagement and user experiences, but it avoids attributing every difference in absence to the service. No numerical effectiveness claim is implied by this fictional case.

Common misconceptions and mistakes

Do not equate a large dataset with an unbiased comparison. More records can narrow uncertainty around a systematically distorted estimate. Matching on age and sex alone may leave important behavioral differences unresolved.

Do not interpret healthy user bias as a character judgment about nonusers. Costs, disability, scheduling, access, and clinical circumstances can influence uptake. Also distinguish confounding from limited generalizability: an unrepresentative study population raises a different question from whether the treatment comparison within that population is biased.

Business use

This concept helps teams assess vendor claims, voluntary employee programs, and AI-supported engagement services. Before converting an observed association into a return-on-investment forecast, ask how participants were selected and which differences existed before the intervention.

Predictive models may accurately identify people who will have favorable outcomes while providing weak evidence about what would improve those outcomes. A model that predicts participation does not automatically identify the effect of participation. Evaluation documentation should keep the prediction target and causal question distinct.

Check your understanding

Users have a 3.6% outcome risk and nonusers 8.4%, but their risk profiles differ. Can we conclude the intervention reduced risk by about 57%?

Answer: No. The worked example produces exactly that crude association when the intervention has no effect. A causal conclusion needs a design and analysis that address the differences responsible for uptake and outcomes.

Related terms

Confounding
Selection bias
Propensity scores
Causal inference
healthy adherer effect
active comparator
propensity score
negative control

Learn next

Next, learn about target-trial specifications and baseline adjustment. Then compare intention-to-treat and adherence-based questions, including the extra assumptions needed when treatment use changes during follow-up.

Intention-to-treat

Sources

Sources: Shrank, Patrick and Brookhart, Healthy User and Related Biases in Observational Studies of Preventive Interventions (2011): https://doi.org/10.1007/s11606-010-1609-1 . Publisher abstract and indexed primary-article methodological passages were consulted: https://pmc.ncbi.nlm.nih.gov/articles/PMC3077477/ . Direct full-page retrieval encountered browser verification. The numerical example, gardening analogy and business application are original explanatory constructions; the arithmetic was independently checked.

Previous: Healing brush Next: Heaps's law