Can there be several control groups?

Yes; different comparators can answer different questions.

Is every controlled study randomized?

No. A comparison group can exist without random assignment.

Does a large control group remove bias automatically?

No. Sample size improves precision under a model but does not repair an unsuitable comparison or inconsistent measurement.

AI Glossary / Statistics and probability / Intermediate

Control group

Definition

A control group is the comparison group used to assess the outcome of an intervention or experimental condition. It may receive the existing approach, a placebo or sham condition where appropriate, an alternative intervention, or no intervention, depending on the question. Its purpose is to provide a relevant reference for interpreting what happens in the treatment group.

What it means

Outcomes can change even without the intervention being evaluated. Customers may become more experienced, demand may shift, or staff may improve through practice. A control group helps distinguish the intervention comparison from those other influences when the design and analysis support it.
The label alone does not guarantee comparability. A group chosen because its members declined an intervention may differ from participants in motivation or need. Random assignment can create comparable groups in expectation, but it does not guarantee identical observed characteristics or eliminate problems arising after assignment.

How it works

Define the intervention, outcome, population, follow-up period, and comparison that matters. A new workflow versus the existing workflow answers a different question from the same workflow versus doing nothing. Describe the control condition precisely enough that another team could reproduce it.
Where feasible and appropriate, assign eligible units randomly. Choose the correct unit: individuals, teams, locations, or other clusters. If units can influence one another, account for spillovers or contamination rather than treating everyone as independent. Use comparable outcome definitions and measurement schedules across groups.
Plan how to handle nonadherence, missing outcomes, and departures from the assigned condition. An intention-to-treat comparison estimates the effect of assignment under the study conditions; an effect among people who actually used the intervention may require additional assumptions and analysis.
Compare groups using an effect measure suited to the outcome, such as a mean difference, risk difference, or risk ratio. Report uncertainty and practical significance. A treatment group's before-and-after improvement alone is not the same as its improvement relative to a suitable control.
Concurrent controls often help with changes over time. Historical controls can be useful in some settings, but differences in populations, conditions, and measurement make their interpretation more demanding. Explain the design rather than treating all control groups as equally informative.

Three practical examples

An AI assistant trial compares a new assistance feature with the existing interface for randomly assigned eligible users.
A training evaluation compares a revised course with the organization's current course over the same period.
A manufacturing experiment compares a new setting with an established reference setting while using the same response measurements.

Think of it this way

Think of a control group as the comparison lane in a fair test. Observing only the lane with the new approach cannot reveal how much progress would have happened anyway. The comparison is useful only when the lanes represent a defensible contrast and are measured consistently.

Worked example

Suppose four hundred eligible users are randomly assigned equally to a new onboarding process or the existing process. All outcomes are observed after the same period. In the new-process group, 144 of two hundred complete onboarding. In the control group, 120 of two hundred do so.
Completion proportions are 144/200 = 0.72 and 120/200 = 0.60. The risk difference is 0.12, or twelve percentage points. The risk ratio is 0.72/0.60 = 1.20, meaning the observed completion proportion is twenty percent higher relative to the control proportion.
For an illustrative independent-binomial normal approximation, the standard error of the difference is sqrt[(0.72 x 0.28)/200 + (0.60 x 0.40)/200], approximately 0.04699. An approximate 95% interval is 0.12 plus or minus 1.96 x 0.04699, or about 0.0279 to 0.2121.
This interval is roughly 2.8 to 21.2 percentage points. It does not mean every user's chance improved by twelve points, and it does not establish performance in a different population. Clustering, missing outcomes, or a different randomization scheme would require an analysis reflecting those features.

Illustrative case study

Fictional case: Rowan Support evaluates an AI drafting tool. It initially proposes comparing volunteers with colleagues who decline access. Analysts explain that enthusiasm and experience could differ between those groups, complicating the interpretation.
The team instead runs an appropriately randomized pilot, specifying the existing drafting workflow as the control condition. It measures response quality and time in both groups and records whether users actually engage with the tool. Managers review the assignment-based comparison alongside adherence and operational context. The pilot's control group strengthens the comparison without making every observed difference attributable to the tool under all circumstances.

Common misconceptions and mistakes

A control group need not receive nothing. The reference can be an active existing practice. Random sampling and random assignment solve different problems and should not be confused. Baseline similarity does not prove that a nonrandomized comparison is unconfounded. Do not compare a statistically significant change in one group with a nonsignificant change in the other and infer a significant difference between groups; estimate the relevant comparison directly.

Business use

Control groups help organizations evaluate new systems, training, and operational changes before broad adoption. Choose a comparison that reflects the decision: replacing an existing practice, adding a feature, or changing a service level. Measure meaningful outcomes and possible tradeoffs in both groups. Document assignment, exposure, attrition, and spillovers so leaders understand what the estimated difference represents.

Check your understanding

In the example, is the improvement twelve percent or twelve percentage points?

Answer: The absolute difference is twelve percentage points; the relative increase is twenty percent.

Would comparing only treatment users before and after give the same evidence?

Answer: No.

What should change if entire teams are randomized?

Answer: The design and analysis must account for team-level assignment and dependence.

Related terms

Selection bias
Spillover (experiment)
Treatment group
randomized controlled trial
active comparator
placebo
intention-to-treat analysis
spillover
risk difference

Learn next

Compare active, no-intervention, and historical control designs for one business question. Then study randomization units, adherence, missing data, and how to estimate a direct between-group effect with suitable uncertainty.

Missing data

Sources

NIST/SEMATECH e-Handbook, glossary, control group: https://www.itl.nist.gov/div898/handbook/glossary.htm NIST/SEMATECH e-Handbook, Completely randomized designs: https://www.itl.nist.gov/div898/handbook/pri/section3/pri331.htm

Previous: Control function (econometrics) Next: Control limits