Must every participant receive every treatment?

No; repeated measurements can follow units over time within separate treatment groups.

Does a repeated measures design always need ANOVA?

No; paired analyses, mixed models, marginal models and other methods may be appropriate.

Can missing visits simply be ignored?

That depends on the analysis and missingness mechanism; removing incomplete participants can reduce precision and introduce bias, while more flexible models still require defensible assumptions.

AI Glossary / Statistics and probability / Intermediate

Repeated measures design

Definition

A repeated measures design obtains multiple observations from the same experimental units or participants, often across time or under several conditions. The repeated observations are linked through their shared unit and should not automatically be treated as independent. This design can reveal within-unit change and distinguish it from differences between units. Repeated measurements occur in experiments and observational studies; their presence alone does not establish random treatment assignment or justify a causal conclusion.

What it means

When the same person, machine or site is measured repeatedly, its own earlier or alternative-condition measurement can provide a useful comparison. Stable differences between units may then be accounted for rather than mistaken for treatment or time effects. However, the shared unit also creates dependence. Four readings from each of ten people are not equivalent to readings from 40 unrelated people. The number of units, number of occasions and assignment mechanism all influence what the study can estimate.

How it works

Define the experimental unit, response, measurement occasions and treatment structure. Plan the schedule and, where appropriate, randomize or counterbalance treatment order. Preserve identifiers linking observations to units and record missing measurements. Choose an analysis that reflects within-unit dependence, such as paired differences for two occasions or a suitable mixed model for richer designs. Model time and treatment-by-time patterns when they answer the research question. Check assumptions about covariance, missingness and outcome distribution rather than applying one automatic repeated-measures test.

Three practical examples

Example 1: An AI interface study measures task completion time for each participant using two interfaces with counterbalanced order. Example 2: A maintenance team observes vibration on the same machines every week. Example 3: A learning program records each student's assessment score at several planned stages. These designs preserve within-unit information. Practice effects, equipment aging and unrelated changes over time can still affect the results, so repeated measurement does not eliminate the need for thoughtful design.

Think of it this way

Think of following several individual paths rather than photographing different crowds at each stop. You can see who changed and how, instead of seeing only a sequence of group averages. But observations along one person's path share a history. Treating every point on that path as a new independent traveler would exaggerate the amount of independent information and hide how the observations were connected.

Worked example

Suppose four independent units have baseline values 10, 20, 30 and 40, and follow-up values 12, 23, 31 and 44 in the same order. Their changes are 2, 3, 1 and 4, with mean change 2.5. The squared deviations of those changes from 2.5 sum to 5, so their sample variance is 5/3 and their sample standard deviation is approximately 1.291. Under independent, comparable unit-level changes, the estimated standard error of the mean change is 1.291/square root of four, approximately 0.6455. The dataset contains eight measurements but four independent units, not eight independent change observations. This calculation describes average observed change; without an appropriate comparison and assignment design, it does not show that an intervention caused that change. Four units also provide very limited information about distributional assumptions.

Illustrative case study

In this fictional case study, Silvergate Services evaluates an AI drafting interface. Each participant completes tasks with both interface versions, and the team counterbalances their order to reduce a systematic practice advantage. It records participant identifiers, task difficulty and interruptions. Analysis compares performance within participants while allowing for relevant task effects. The team reports the observed difference and uncertainty, and notes that a small convenience sample may not represent the wider workforce. It does not count every repeated task as an unrelated participant.

Common misconceptions and mistakes

Common mistakes include ignoring within-unit correlation, confusing the number of rows with the number of independent units, and interpreting any before-and-after change causally. Another mistake is assuming traditional repeated-measures ANOVA is suitable regardless of missing data or covariance structure. With more than two within-unit levels, some univariate tests require sphericity or an appropriate correction; mixed models use their own assumptions. Counterbalancing can help with order effects but cannot automatically remove irreversible carryover or time-varying confounding.

Business use

Businesses use repeated measures designs for usability, training, equipment monitoring and customer experience studies. The approach can improve efficiency when within-unit comparisons remove substantial stable variation, but it may increase burden and dropout. AI evaluations should keep participant or device identifiers through analysis and split data carefully when repeated records could leak across training and evaluation sets. Plan the estimand, follow-up schedule and handling of missing observations before collecting data, rather than choosing the most favorable comparison afterward.

Check your understanding

How many independent units are in the worked example?

Answer: Four, despite eight recorded measurements.

What is the mean within-unit change?

Answer: 2.5 units.

Why might using both interfaces in a fixed order create a problem?

Answer: Practice, fatigue or carryover may be confused with the interface difference.

Does a clear change over time prove an AI intervention caused it?

Answer: No; the comparison design must support that causal interpretation.

Related terms

Related terms include longitudinal data
paired observations
within-subject factor
crossover design
covariance structure
mixed-effects model
sphericity and pseudoreplication

Learn next

Learn next how paired comparisons use the variance of within-unit differences. Then explore random intercepts, time trends and alternative within-unit covariance structures. Study order effects and missing-data mechanisms before planning a multi-occasion experiment. Plot individual trajectories as well as group averages to see patterns that a single overall mean can conceal.

Missing data

Sources

Sources: Penn State STAT 502, Introduction to Repeated Measures, https://online.stat.psu.edu/stat502/Lesson11; Penn State STAT 510, Longitudinal Analysis/Repeated Measures, https://online.stat.psu.edu/stat510/Lesson10. These teaching sources support the design and correlation principles; the paired calculation and fictional interface study are original examples with their assumptions stated.

Previous: Repeated game Next: Repeated state