Can a randomized study contain deliberate confounding?

Yes; blocked or fractional designs may alias selected effects while preserving the contrasts of primary interest.

Does adjustment always solve the problem?

No. It requires suitable information and assumptions, and exact overlap can prevent separate estimation.

Is this the same as causal confounding?

The ideas are related, but experimental aliasing and confounding bias in observational causal inference are not identical definitions.

AI Glossary / Statistics and probability / Intermediate

Confounded

Definition

Variables or effects are confounded when their contributions cannot be distinguished adequately in the comparison or design being used. In the straightforward experimental sense, two factors may change together so that an observed difference cannot be attributed separately to either one. In designed experiments, confounded can also describe effects that share the same estimable contrast, including deliberate aliasing between treatment and block effects.

What it means

Suppose a company tests a new AI tool only in one office while another office uses the old tool. Tool and office vary together. If outcomes differ, the comparison may reflect the tool, office conditions, or both. The data do not contain the cross-comparisons needed to separate these explanations without additional assumptions.
Confounded is a description of an identification problem, not a synonym for confusing, noisy, or unimportant. A large dataset can measure a combined difference very precisely while leaving its separate components unresolved. The key issue is what variation the design actually provides.

How it works

Identify the effect the study aims to estimate and list other factors that differ across the compared conditions. Inspect whether relevant combinations are represented. If every new-tool case occurs in one office and every old-tool case in another, the two indicators carry identical information.
In a regression, exact overlap can make separate coefficients unidentifiable. Software may drop a column, impose a constraint, or report a singularity; none of these actions creates the missing comparison. Near overlap may permit estimation but with instability and heavy reliance on modeling assumptions.
Plan independent variation where feasible. Randomize tool assignment within offices, include both tools in multiple settings, or use a suitable blocked or factorial design. Counterbalancing may address order effects when carryover and other assumptions permit.
Some experimental designs deliberately confound selected higher-order interactions with blocks to make a study feasible. Such designs can still estimate prioritized effects, provided their alias structure is understood. State which effects are inseparable and what assumptions support interpretation. Do not treat an effect estimate as uniquely attributable when the design only identifies a combination.

Three practical examples

A new training method is used only by experienced employees, while beginners receive the old method, entangling method and experience.
A manufacturing process setting changes at the same time as the raw-material supplier, leaving their separate contributions unresolved in a simple before-and-after comparison.
A fractional factorial experiment intentionally aliases selected interactions with other effects, requiring careful interpretation of its contrasts.

Think of it this way

Think of two lights controlled by the same switch. Observing brightness after flipping that switch shows their combined change. It does not reveal how much each light contributed. More repetitions of the same switch movement can sharpen the combined measurement without creating a separate control for either light.

Worked example

Consider an original design with two binary indicators: A for the new workflow and B for the afternoon shift. All old-workflow observations occur in the morning, so A=B=0; all new-workflow observations occur in the afternoon, so A=B=1. The group means are 20 and 26 completed tasks.
Under an additive model, expected output is beta0 + betaA times A + betaB times B. Because A=B in every observed row, this reduces to beta0 + (betaA + betaB) times A. The comparison estimates a combined difference of six tasks.
Several coefficient pairs fit the same means: betaA=6 and betaB=0; betaA=0 and betaB=6; or betaA=2 and betaB=4. The data do not select among them. Repeating these same two combinations thousands of times would not resolve the exact confounding.
Adding observations with the new workflow in the morning and the old workflow in the afternoon creates the missing combinations. With an appropriate randomized design, replication, and model, separate effects can then be investigated. If interactions are possible, the analysis must consider them rather than assuming additivity without justification.

Illustrative case study

In a fictional company, Riverbend Processing compares a new document system introduced at a newly equipped branch with the old system at an established branch. The initial report attributes the difference in processing speed entirely to the software.
A reviewer notes that branch, equipment, staff composition, and software changed together. The team revises the claim to describe the observed package difference. It plans a further comparison that uses both systems under more comparable conditions and measures relevant operational differences.
This fictional case illustrates recognizing and redesigning a confounded comparison. It does not imply that the software had no effect; the original comparison simply does not isolate that effect.

Common misconceptions and mistakes

Confounded does not mean no effect exists. A small p-value cannot untangle effects that the design never separates. More data from the same perfectly overlapping conditions do not solve exact confounding. Statistical adjustment cannot manufacture missing treatment combinations. Correlation between predictors is not always complete confounding, although it may create estimation difficulties. Deliberate confounding in a designed experiment is not automatically a mistake, but its limitations must be explicit.

Business use

Businesses need to recognize confounded comparisons when evaluating tools, policies, staffing changes, or process improvements. The practical benefit is avoiding investment decisions based on an attribution the evidence cannot support. Ask which factors changed together and what comparisons are missing. Where separation is impossible, describe the intervention package honestly and avoid assigning its result to one component without further evidence.

Check your understanding

In the worked example, what does the six-task difference identify under the additive model?

Answer: The sum of workflow and shift contributions.

Why can neither contribution be uniquely estimated?

Answer: The indicators are identical in every observed row.

What additional variation would help?

Answer: Observations of both workflows in both shifts, collected under a design suited to the question.

Related terms

Confounding
Aliasing (factorial experiments)
Factorial design
Blocking
Collinearity
Observational study
aliasing
identifiability
randomization
interaction
treatment comparison

Learn next

Learn experimental design and estimable contrasts, then inspect a small design matrix for identical or dependent columns. Compare accidental confounding with planned aliasing in fractional factorial designs. Next study causal confounding, where the central question concerns whether observed groups support a valid intervention comparison.

Confounding

Sources

David Howell, Statistical Methods for Psychology glossary, Confounded: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html NIST/SEMATECH e-Handbook, A Glossary of DOE Terminology: https://itl.nist.gov/div898/handbook/pri/section7/pri7.htm NIST/SEMATECH e-Handbook, Confounding (also called aliasing): https://www.itl.nist.gov/div898/handbook/pri/section3/pri3343.htm

Previous: Conformalized quantile regression Next: Confounder