AI Glossary / Causal inference and experimentation / Advanced
Causal positivity
Definition
Causal positivity is the requirement that the treatment choices needed for a causal comparison have positive probability within the relevant covariate groups in the target population. For a binary treatment and a population average effect, this commonly means that the probability of treatment lies strictly between zero and one for relevant covariate values.
What it means
To compare alternatives fairly, the data-generating process must allow the alternatives being compared. If a company gives a program to every senior employee and no junior employee, it has no within-seniority comparison for that program. A prediction model may still output numbers for the missing combinations, but those numbers rely on extrapolation. Positivity concerns support for the causal question, not whether the intervention has a positive or beneficial effect. The required support also depends on the particular target intervention and population.
How it works
Researchers consider treatment assignment conditional on an appropriate set of measured variables. Structural violations occur when a treatment is impossible for a relevant group; practical problems arise when it is possible but rarely represented in the available sample. Diagnostics can inspect assignment patterns, propensity distributions and influential weights. Near-zero probabilities can make inverse-weighted estimates unstable. Responses may include better data collection, changing the target population or choosing a supported intervention contrast. Truncating weights can reduce instability but introduces a tradeoff; it does not create the missing evidence or automatically restore identification.
Three practical examples
- An AI assistant is available only to one department. A comparison across departments cannot rely on within-department variation in access if none exists.
- A delivery experiment includes both routing policies in every participating depot. The design supports comparison better than allocating one policy exclusively to all large depots and the other to all small depots.
- A voluntary course is taken by almost every experienced employee but almost no new employee. Both options technically occur, yet the rare combinations may contribute very little reliable information.
Think of it this way
Think of comparing two recipes when your records contain only recipe A in one kitchen and only recipe B in another. A kitchen difference may be inseparable from the recipe difference without further information. More detailed arithmetic cannot fill the absent cells. Before analyzing outcomes, look at whether the comparisons your question requires actually exist.
Worked example
In an original dataset, 100 senior staff include 99 treated and one untreated person, while 100 junior staff include 20 treated and 80 untreated people. Using the simple within-group treatment proportions, the untreated senior person's inverse comparison probability is 1/(1−0.99)=100. An untreated junior person's corresponding weight is 1/(1−0.20)=1.25. The senior comparison is therefore extremely sensitive to one observation. If all 100 senior staff were treated because of a fixed policy, the observed group would have no untreated comparison at all. These calculations illustrate sparse support; estimated sample proportions alone do not prove the underlying population probabilities or resolve confounding.
Illustrative case study
A fictional customer-service business evaluates a new coaching system. Senior agents almost always receive coaching because they handle difficult cases. Analysts inspect the assignment data before comparing outcomes and discover very few uncoached senior agents. They explain that the desired all-agent effect would depend heavily on those few cases or on model extrapolation. Management considers a future rollout with a defensible comparison and a narrower current question for groups with adequate support. The report describes the narrower population explicitly. This case illustrates a possible response to limited support, not an empirical finding or a rule that one remedy always works.
Common misconceptions and mistakes
Positivity does not mean that both groups must be the same size. Unequal probabilities can be acceptable when there is sufficient support for the question. It is also distinct from exchangeability: having treated and untreated cases does not remove confounding. A smooth propensity model can conceal a deterministic business rule by outputting small nonzero probabilities. Conversely, an empty cell in a small sample does not necessarily prove a structural impossibility. The data, assignment process and target population should be considered together rather than relying on a single plot or threshold.
Business use
Positivity checks can prevent organizations from making broad causal claims based on unsupported comparisons. They also reveal useful design questions before a rollout: which groups will receive each option, and will the intended evaluation retain meaningful comparison data? When support is weak, analysts can explain the practical consequences in terms of unstable estimates, extrapolation and a narrower population of relevance. A decision-maker should know whether a reported effect applies to all intended users or only to the subset whose alternatives were represented.
People also ask
Does randomization guarantee good support in every realized subgroup?
No. Small samples can still produce sparse or empty combinations.
Can a large dataset have positivity problems?
Yes. A fixed assignment policy can exclude a treatment for a group regardless of total sample size.
Does trimming observations leave the estimand unchanged?
Not generally. Restricting the analyzed population can change the target and should be reported.
Check your understanding
Why was the untreated senior weight so large?
Answer: The estimated probability of that observed treatment condition was only 0.01.
What distinguishes a structural violation from sparse data?
Answer: One concerns an unavailable treatment under the assignment process; the other concerns limited representation.
Why can a well-fitting outcome model still be problematic?
Answer: It may extrapolate into unsupported treatment-covariate combinations.
Related terms
- Propensity scores
- Overlap
- propensity score
- inverse probability weighting
- causal identification
- extrapolation
- target population
Learn next
Create a table of treatment counts across relevant baseline groups. Examine how the target question changes if unsupported groups are excluded. Then study weight diagnostics and sensitivity to truncation while keeping structural and practical limitations distinct.
More in Causal inference and experimentation
- Causal language modeling - Causal language modeling is a training task in which a language model predicts the next token using only the preceding context.
- Causal link - In partial-order AI planning, a causal link records that one action establishes a condition needed by another action.
- Causal network - A causal network is a model that represents variables and their causal relationships, commonly using a directed graph together with probability...
- Causality - Causality concerns relationships in which changes in one part of a system can produce changes in another.
- Causation - Causation is a cause-and-effect relationship in which one event, condition or intervention contributes to producing another.
- Character N-gram F-score - Character N-gram F-score evaluates generated text by combining precision and recall of matching character sequences against reference text.
Sources
Causal Learning Handbook, Positivity, Overlap, and Data Support: https://amertens.github.io/causal-learning-handbook/03-02-positivity-overlap.html Petersen and colleagues, Diagnosing and responding to violations in the positivity assumption: https://pmc.ncbi.nlm.nih.gov/articles/PMC4107929/ The employee counts, weight calculation and coaching case are original educational illustrations.
Previous: Causal network Next: Causality