Must matched observations come from the same person?

No. They may come from different units paired on relevant characteristics, although the supplied glossary uses the narrower repeated-subject example.

Does matching always improve precision?

No. It helps when the pairing captures useful outcome-related similarities and the design is analyzed appropriately.

Can I use an independent-samples test anyway?

That usually ignores important dependence. Choose a method that matches the actual design and outcome.

AI Glossary / Statistics and probability / Beginner

Matched samples

Definition

Matched samples are samples whose observations have a meaningful correspondence across conditions or groups. The matching can arise because the same unit is measured more than once, or because different units are deliberately paired using relevant characteristics measured independently of the outcome comparison. The supplied psychology glossary emphasizes repeated observations of the same subject; broader statistical usage also includes matched distinct subjects. The connection between observations must be preserved in analysis.

What it means

In an independent-samples comparison, one group's observations have no special one-to-one connection with the other group's observations. In a matched comparison, each observation has a designated partner. That partner may be another measurement on the same person or a comparable person selected under a matching rule.

Matching aims to make comparisons more informative by accounting for relevant differences between units. It does not guarantee that groups are identical in every respect, and it does not automatically turn an observational study into a randomized experiment. The reason for the match matters as much as the existence of a pair identifier.

How it works

Define the unit of analysis, the conditions being compared, and the basis for pairing before using outcomes to judge results. For repeated measurements, link records reliably to the same unit. For distinct units, select suitable matching variables and document the algorithm, restrictions, and treatment of unmatched observations.

For a simple two-condition numerical comparison, calculate a difference within each pair. Analyze the distribution of these differences using a method appropriate to the outcome and design. More complicated matching, repeated observations, or clustered pairs may require a model beyond a basic paired test.

Keep the direction of subtraction consistent. Plan for missing members of pairs and assess whether excluding incomplete pairs changes the population represented. In an experiment where each participant receives both conditions, consider order, learning, and carryover effects rather than assuming matching alone removes them.

Three practical examples

Employees complete a task before and after training, with records linked to each employee.
An AI benchmark compares two systems on the same prompts, preserving prompt-level pairs when analyzing score differences.
An observational study pairs establishments with similar baseline size and sector before comparing their outcomes under different practices.

Think of it this way

Think of comparing two shoes within each person's pair rather than comparing a box of left shoes with an unrelated box of right shoes. The correspondence tells you which measurements belong together. In a statistical study, however, matching two different people is less exact than matching the two shoes: unmeasured differences may still remain.

Worked example

Consider an original example with four employees. Their before-training scores are 10, 20, 30, and 40; the corresponding after-training scores are 12, 19, 34, and 43. The within-person differences, after minus before, are 2, −1, 4, and 3.

The mean difference is (2 − 1 + 4 + 3)/4 = 2. The squared deviations from that mean are 0, 9, 4, and 1, summing to 14. The sample standard deviation of differences is sqrt(14/3), approximately 2.160.

Now imagine the after scores are accidentally reversed before pairing. The apparent differences become 33, 14, −11, and −28. Their mean remains 2 because the overall before and after totals have not changed, but their sample standard deviation becomes approximately 26.920.

This demonstrates why correct linkage matters even when a headline difference of means is unchanged. The uncertainty and individual-change interpretation depend on the pairs. These four observations are a teaching example, not enough to justify a general claim that training caused improvement.

Illustrative case study

Fictional case: PromptWillow evaluates two AI writing assistants on the same collection of customer requests. The team initially compares the two score lists without checking their order. A review finds that one list was sorted by difficulty and the other by request identifier.

Analysts restore the correct request-level correspondence, calculate paired score differences, and inspect whether improvements vary by request type. They also recognize that several requests came from the same customer, which may introduce dependence across pairs. The final analysis accounts for the evaluation design rather than treating every score as an unrelated observation.

Common misconceptions and mistakes

Equal sample sizes do not make samples matched. Pairing observations arbitrarily after data collection can produce a misleading analysis. Matching on an outcome or on a variable affected by the intervention can also change the question and introduce bias.

Repeated measurements from one person are related, but different pairs are not necessarily independent if they share a household, team, or customer. Matching distinct units on observed characteristics does not eliminate unmeasured confounding. A before-and-after improvement may reflect learning, time trends, or other changes besides the intervention being studied.

Business use

Businesses can use matched comparisons to evaluate product changes, training, or AI systems while reducing variation attributable to the units being compared. Testing both systems on the same tasks often gives a clearer picture of where differences arise than separate task collections do.

Preserve stable pair identifiers, record the matching rationale, and report how many observations were excluded or left unmatched. Describe the population represented by the retained matches. In observational work, assess balance on matching variables and consider remaining confounding; in experiments, use randomization where feasible to strengthen causal interpretation.

Check your understanding

Two groups each contain 100 customers, but no customer or matching rule connects observations across groups. Are these matched samples?

Answer: No. Equal group sizes alone do not create correspondence. A matched design needs a defensible link between each observation and its partner or matched set.

Related terms

Paired data
Confounding
Carryover effect
repeated measures
matched-pairs design
within-pair difference
matching
paired t test

Learn next

Learn paired analysis and the assumptions of the matched-samples t test. Then study matching in observational research, including balance, overlap, and the distinction between observed comparability and causal exchangeability.

Matched-samples t test

Sources

Howell, psychology glossary, matched samples: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html

OECD Glossary of Statistical Terms, broader matched-samples definition: https://www.oecd.org/content/dam/oecd/en/publications/reports/2008/09/oecd-glossary-of-statistical-terms_g1gh9ad7/9789264055087-en.pdf

NIST, paired versus unpaired correspondence: https://www.itl.nist.gov/div898/handbook/eda/section3/eda353.htm

The score example is original and independently calculated. PromptWillow is fictional.

Previous: Matched filter Next: Matched-samples t test