Does hiding the protected attribute ensure this criterion?

No. Its effects may remain in other features.

Can it be tested without assumptions?

Not generally. It relies on a causal model, including potentially unobserved factors.

Does passing it prove legal compliance?

No. A statistical fairness criterion does not determine all applicable legal or organisational requirements.

AI Glossary / Causal inference and experimentation / Advanced

Counterfactual fairness

Definition

Counterfactual fairness is a causal-model-based fairness criterion asking whether a prediction for an individual would remain the same in distribution under a hypothetical change to a protected attribute, while keeping the individual's relevant background factors fixed in the model.

What it means

The idea concerns a comparison between the actual person and a modeled counterfactual version of that same person, rather than only comparing group-level outcome rates. It was introduced as a formal fairness framework by Kusner, Loftus, Russell and Silva.
The comparison depends on a causal model connecting protected attributes, observed features and underlying factors. If a protected attribute affects other features, simply changing its column while freezing every other observed input may not represent the intended counterfactual.
This is one fairness definition, not a universal certificate of ethical or lawful behavior. The meaning of hypothetical changes to socially significant attributes and the acceptability of different pathways require substantive judgment.

How it works

Specify a structural causal model and identify the protected attribute, prediction and background variables. Use the observed case to reason about the relevant background factors. Then consider interventions assigning alternative values to the protected attribute and propagate their consequences through the model.
Compare the resulting prediction distributions conditional on the observed information. Under the formal criterion, they should be invariant across the specified attribute changes. Practical methods may construct predictors from suitable latent or causally unaffected information. The result depends on assumptions that may not be identifiable from observational data, so sensitivity to alternative plausible causal models is central to a credible assessment.

Three practical examples

A research team tests a predictor in a synthetic causal model where the true data-generating mechanisms are known.
An audit studies whether a model's output changes through pathways from a protected attribute under explicit causal assumptions.
A fairness workshop compares counterfactual fairness with group metrics to show how different definitions answer different questions.

Think of it this way

Think of comparing two versions of the same story while changing one specified background characteristic and consistently updating the events that depend on it. The output should not differ under the criterion being tested. The analogy is limited: real people's identities are not simple switches, and the model's imagined alternative is an assumption-dependent construction rather than an observable second life.

Worked example

Consider a purely mathematical example with binary attribute A and background variable U. Suppose X=2A+U, and the prediction is initially X. For a case with A=1 and X=7, the model implies U=5. Changing A to 0 while holding U fixed changes X to 5, so the original predictor changes from 7 to 5.
Now consider the constructed predictor X−2A. In the stated model this equals U, so it remains 5 under either value of A. This illustrates counterfactual invariance in a deliberately simple structural model.
The example is not a deployment recipe. It assumes the causal equation is correct and U is recovered exactly. Using protected-attribute information in a real decision can raise separate policy and legal questions, and satisfying this mathematical criterion would not establish that the resulting decision system is fair in every relevant sense.

Illustrative case study

Fictional case: Indigo Research evaluates a synthetic allocation model and initially tests fairness by swapping an attribute label while leaving all other features unchanged. A reviewer points out that some features are descendants of the attribute in the team's own causal model.
The researchers revise the counterfactual calculation to propagate those changes and compare results under alternative plausible graphs. They document which conclusions depend on unobserved background assumptions. The exercise becomes a study of a specific fairness criterion, not a claim that the model is ready for a consequential real-world allocation task. No fairness guarantee is asserted for an actual population.

Common misconceptions and mistakes

Removing the protected-attribute column does not guarantee counterfactual fairness because other features may carry its influence. Conversely, a naive column swap does not necessarily represent the required causal counterfactual.
The criterion is not identical to demographic parity, equalized odds or individual similarity-based fairness. Its evaluation depends on a causal model and assumptions about latent variables. A model may satisfy one fairness definition while failing another. Treating all causal influence from an attribute as unacceptable is itself a substantive choice, and more specialized path-based definitions answer different questions.

Business use

Counterfactual fairness can structure advanced fairness research and audits when the causal assumptions are explicit and relevant stakeholders can examine them. It encourages teams to look beyond simple omission of sensitive fields and consider how historical or social processes shape other inputs.
Before using the criterion, define the decision context, protected attributes and what the counterfactual comparison is intended to mean. Record alternative plausible models and assess whether conclusions change. Combine this analysis with outcome evaluation, procedural review and domain-specific requirements. Report the exact criterion assessed rather than labeling a system simply “fair” based on one mathematical test.

Check your understanding

A fairness test changes A but holds a feature X fixed even though the model says A causes X. Is that automatically a valid counterfactual-fairness test?

Answer: No. The intended counterfactual generally needs to propagate the intervention through the causal model while preserving the appropriate background factors. A column swap may test something else.

Related terms

Structural causal model
Protected attribute
Demographic parity
Equalized odds
Latent variable
Fairness in machine learning
counterfactual
path-specific effect

Learn next

Learn structural counterfactuals before applying this criterion. Reproduce a small synthetic example with known mechanisms, then change an assumption and observe how the fairness conclusion changes. Compare the question answered by this criterion with the questions answered by group fairness metrics.

Group fairness

Sources

Matt J. Kusner, Joshua R. Loftus, Chris Russell and Ricardo Silva, Counterfactual Fairness: https://arxiv.org/abs/1703.06856 Conference paper: https://papers.nips.cc/paper_files/paper/2017/file/a486cd07e4ac3d270571622f4f316ec5-Paper.pdf Primary paper abstract/search excerpts consulted. The structural example is original; full mathematical, contextual and editorial review remain pending.

Previous: Counterfactual explanation Next: Countermeasure