AI Glossary / Causal inference and experimentation / Advanced
Propensity scores
Definition
Propensity scores are the probabilities of receiving a particular treatment or exposure conditional on observed pretreatment characteristics. For a binary treatment, the score is commonly written as P(A = 1 | X). In causal analysis, they help construct comparisons between treated and untreated groups that are more comparable on measured baseline information.
What it means
People or organizations often receive an intervention for reasons related to their likely outcomes. A support program may be offered more often to accounts already experiencing difficulties. Comparing recipients with everyone else can therefore mix the program's effect with pre-existing differences. A propensity score summarizes the observed assignment pattern; it is not the probability that the intervention will work. It also cannot automatically account for important characteristics that were never measured or were recorded after treatment began.
How it works
Define the treatment, target effect and relevant baseline covariates. Estimate treatment probabilities using a suitable model, then apply a method such as matching, weighting or stratification. Inspect overlap and whether the resulting comparison balances important measured covariates. Rosenbaum and Rubin established the score's balancing role under appropriate conditions. Causal interpretation still requires assumptions such as no unmeasured confounding for the chosen adjustment set, consistency and adequate treatment overlap. A high-performing treatment classifier alone is not the objective; the quality of the resulting causal comparison matters.
Three practical examples
- A hypothetical customer-support study compares outcomes for accounts offered an additional service with comparable accounts not offered it, using baseline information available before the offer.
- An operations researcher examines a voluntary training program while accounting for measured differences between participating and nonparticipating teams.
- A policy-analysis team checks whether treated and untreated units overlap sufficiently before attempting a propensity-score-based estimate, declining to extrapolate where comparable units are absent.
Think of it this way
Think of trying to compare participants who had similar chances of entering a program for the reasons you recorded. That can make the comparison more useful, but it does not recreate a randomized trial. An unrecorded reason for participation may still distinguish the groups and affect their outcomes.
Worked example
Suppose a binary-treatment study uses unstabilized inverse-probability weights for a population average effect. A treated unit with estimated propensity 0.25 receives weight 1/0.25 = 4. An untreated unit with propensity 0.25 receives weight 1/(1−0.25), approximately 1.33. A treated unit with propensity 0.80 receives weight 1.25. These invented calculations illustrate why units receiving an uncommon treatment given their covariates can receive larger weights. They do not compute a treatment effect or validate the model. Extremely small treatment probabilities can create unstable weights, requiring careful investigation of overlap, estimation choices and the population to which a conclusion applies.
Illustrative case study
A fictional software company evaluates an optional onboarding service. Analysts recognize that struggling accounts were more likely to receive it, so raw retention differences are not interpreted as causal effects. They define baseline covariates, estimate propensity scores and inspect balance after applying a chosen adjustment method. Some accounts have no credible comparison group, prompting a narrower analysis and a clear change in the target population. The team also examines sensitivity to unmeasured factors and reports uncertainty. The study provides assumption-dependent evidence rather than claiming that a single score has removed every source of bias.
Common misconceptions and mistakes
A propensity score predicts treatment assignment, not the outcome and not the individual treatment effect. Matching on an estimated score does not guarantee balance in a finite sample; check the actual covariates. Including variables affected by treatment can introduce bias. A model with excellent assignment prediction may reveal poor overlap rather than a good causal design. Trimming or restricting observations can change the population being studied. Finally, measured balance does not demonstrate that unmeasured confounding is absent, so causal claims must remain tied to the design and assumptions.
Business use
Propensity-score methods can support observational evaluations when randomized assignment is unavailable and relevant baseline information is credible. Start with a clear intervention, comparison and outcome timing. Involve subject-matter expertise to choose adjustment variables, rather than selecting them only through predictive performance. Report overlap, balance, uncertainty and any excluded population. Compare conclusions across defensible specifications and explain what could still bias the result. The business value lies in a better-designed comparison, not in using a sophisticated score to make an observational association sound definitive.
People also ask
Does a score of 0.8 mean an 80% chance of benefit?
No. It describes estimated treatment probability conditional on the included baseline covariates.
Can matching fix unmeasured confounding?
Not automatically; the missing information remains a potential source of bias.
Are propensity scores only for matching?
No. They are also used in weighting, stratification and other estimators.
Check your understanding
Why would an untreated unit with propensity 0.9 have a large inverse-probability weight?
Answer: Remaining untreated is unusual under its estimated assignment probability, giving weight 1/0.1 = 10 in the stated scheme.
Why check covariate balance after adjustment?
Answer: Estimated scores and finite samples may not produce the intended comparability. Explain why restricting to an overlapping population can improve credibility while changing the question being answered.
Related terms
- Confounding
- Causal estimand
- covariate balance
- propensity-score matching
- inverse probability weighting
- positivity
Learn next
Define a treatment effect using potential outcomes, then study exchangeability and overlap. Compare matching and weighting for the same target population. Learn balance diagnostics and sensitivity analysis before interpreting an adjusted estimate as evidence for an intervention decision.
More in Causal inference and experimentation
- Precision-recall curve - A precision-recall curve shows how a classifier's precision and recall change as its decision threshold varies.
- Preferential Bayesian optimization - Preferential Bayesian optimization is a sequential search method that uses comparisons between alternatives to seek a preferred solution.
- Probabilistic Programming - Probabilistic programming is an approach to expressing statistical models as programs that represent uncertainty through probability distributions.
- Proper accuracy scoring rule - A proper accuracy scoring rule evaluates probability forecasts so that reporting the true outcome probabilities achieves the best expected score.
- Proxy metric - A proxy metric is a measurable quantity used as an indirect indicator of an outcome or quality that is harder to measure directly.
- Random experiment - A random experiment is a process with an uncertain outcome drawn from a defined set of possible outcomes, called its sample space.
Sources
Rosenbaum and Rubin, The central role of the propensity score in observational studies for causal effects: https://doi.org/10.1093/biomet/70.1.41 Rubin and Waterman, Estimating the Causal Effects of Marketing Interventions Using Propensity Score Methodology: https://arxiv.org/abs/math/0609201 The weights and onboarding case are original illustrations, not a completed causal study or a claim of program effectiveness.
Previous: Proof-number search Next: Proper accuracy scoring rule