AI Glossary / Causal inference and experimentation / Advanced
Causal estimand
Definition
A causal estimand is the precisely defined causal quantity that an analysis aims to learn. It describes a contrast between outcomes under specified interventions for a specified population. An estimand is the target; an estimator is a method for calculating an estimate of that target from data.
What it means
A question such as “Does this AI tool help?” leaves several choices unresolved. Help whom, compared with what, measured how and over which period? Defining the estimand turns that broad question into something that can guide a study. For example, the target might be the average difference in task completion time over four weeks if all eligible teams were offered the tool versus if none were offered it. That is a different target from the effect among frequent users or the effect on output quality.
How it works
Specify the units and population, intervention and comparison, outcome, time horizon and summary measure. Express the contrast using potential outcomes or another suitable causal language. For a binary treatment, E[Y(1)−Y(0)] is an average treatment effect in the population over which the expectation is taken. Identification asks whether the target can be expressed using available data under stated assumptions. Estimation then chooses a suitable procedure. A carefully defined estimand can remain unidentified; a precise question does not guarantee that a particular dataset can answer it.
Three practical examples
- A support company targets the average effect of offering automated reply suggestions on resolution time across all eligible tickets, with human staff retaining final control.
- A training team targets the effect of a course among employees who enrolled. That population differs from all employees who could have enrolled.
- An online service compares renewal probabilities under two specified reminder policies over a 30-day window. Changing the window to one year changes the causal target, even if the same users are studied.
Think of it this way
Think of the estimand as the destination written into a journey plan. The estimator is the route, and the estimate is where the calculation places you using the available information. Two routes may aim for the same destination. Conversely, two highly accurate routes can lead to different places if their destination descriptions differ. Naming a popular method does not tell readers which question it answers.
Worked example
Suppose a fictional population contains 40 experienced and 60 newer staff. Stipulate average effects of a training offer on task time of minus 2 minutes for experienced staff and minus 5 minutes for newer staff. The population average effect is 0.4(−2)+0.6(−5)=−3.8 minutes. Now consider a treated population containing 80% experienced and 20% newer staff, and additionally assume the same average effect applies within each experience group among treated staff. Its average effect is 0.8(−2)+0.2(−5)=−2.6 minutes. Both calculations can be correct because their target populations differ. These invented effects are supplied for arithmetic; they are not identified from observed before-and-after records.
Illustrative case study
A fictional agency asks analysts to evaluate an AI drafting tool. Leadership initially mixes three questions: the effect of offering access, the effect of regular use and the effect among employees who already volunteer to try it. The team writes a separate estimand for each. It chooses the offer effect as the primary question for a randomized rollout and describes the others as requiring additional design or assumptions. The report retains the prespecified target even when adoption varies. This scenario illustrates how defining the estimand can prevent a promising usage comparison from silently replacing the original business question.
Common misconceptions and mistakes
An estimand is not a fitted coefficient by default. A regression coefficient may correspond to a causal target only under appropriate assumptions and interpretation. Nor are estimand, estimator and estimate interchangeable words. Another mistake is changing the population after discovering weak overlap without explaining that the target changed. Restricting analysis can be sensible, but readers need to know whom the result now concerns. Teams also sometimes specify an intervention vaguely, combining materially different versions under one label and making the intended contrast unclear.
Business use
A clear estimand aligns analysts, product owners and operational teams before data collection. It reveals whether the proposed evaluation addresses the actual decision and whether the necessary comparison is feasible. It also helps explain disagreements between studies: different populations, outcome windows or intervention versions may produce different valid answers. A useful project brief puts the causal target in ordinary language next to its formal expression. That makes it easier to notice when a software setting or eligibility filter changes what the analysis estimates.
People also ask
Can several estimators target the same estimand?
Yes. Different methods may estimate the same quantity under their respective conditions.
Is an estimand always an average effect?
No. Targets can include risks, ratios, distributions, subgroup effects or policy values.
Should it be chosen after model fitting?
The substantive target should guide the analysis; exploratory changes should be clearly identified rather than presented as prespecified.
Check your understanding
Why did the worked averages differ?
Answer: They used different population compositions.
What additional assumption supported the treated-population calculation?
Answer: That the stipulated within-group effects also applied among treated staff.
What separates identification from estimation?
Answer: Identification links the causal target to observable information under assumptions; estimation calculates from a sample.
Related terms
- Estimand
- Estimate
- Average treatment effect
- estimator
- target population
- causal identification
Learn next
Rewrite a broad business question as a precise intervention contrast. Name the population and time horizon. Then compare an effect among all eligible units with an effect among treated units and identify what evidence each requires.
More in Causal inference and experimentation
- Box-Benkhen experiment - A Box - Behnken experiment uses a response-surface design to study how several controllable factors affect an outcome and to fit a quadratic model.
- Brier score - The Brier score measures the quality of probability predictions by averaging squared differences between predicted probabilities and observed outcomes.
- Calinski-Harabasz Index - The Calinski-Harabasz Index is an internal clustering score that compares separation between cluster centers with variation inside clusters, adjusted for...
- Causal graphs - Causal graphs are diagrams that represent assumptions about causal relationships among variables.
- Causal inference - Causal inference is the process of drawing conclusions about how outcomes would change under specified interventions.
- Causal language modeling - Causal language modeling is a training task in which a language model predicts the next token using only the preceding context.
Sources
Causal Learning Handbook, causal roadmap workshop: https://amertens.github.io/causal-learning-handbook/workshop/00-short-course.html CausalPy documentation, estimands: https://causalpy.readthedocs.io/en/stable/knowledgebase/estimands.html The population calculations and agency scenario are original illustrations; the subgroup effects are assumed for teaching purposes.
Previous: Causal analysis Next: Causal graphs