AI Glossary / Statistics and probability / Intermediate
Mixed model designs
Definition
Mixed model designs commonly refer to studies combining at least one between-subjects factor with at least one within-subjects, or repeated-measures, factor. The phrase can also describe designs analyzed with both fixed and random effects. These meanings are related but not identical: one describes how observations are organized, while the other describes the statistical model used to represent them.
What it means
Imagine comparing two training programs while measuring each participant before and after training. Program is a between-subjects factor because different people receive different programs. Time is a within-subjects factor because the same people are measured repeatedly.
The analysis must recognize that two observations from one person are connected. A mixed-effects model can represent this connection through person-specific random effects and, where appropriate, residual correlation. However, the presence of repeated observations alone does not identify the best model.
How it works
Start by defining the experimental units, assignment process, repeated conditions, and outcome. Record participant identifiers so repeated measurements remain linked. A conventional mixed-design ANOVA separates between-person comparisons from within-person comparisons and their interaction.
The interaction asks whether the pattern across repeated conditions differs between groups. For a two-group, two-time-point study, this can be expressed as a difference between the groups' mean changes. Main effects answer other questions and may be difficult to summarize when an interaction is substantial.
A linear mixed-effects formulation might include fixed coefficients for group, time, and their interaction, plus a random intercept for each participant. Random slopes can represent variation in individual changes when supported by the design and data. Residual structures may capture additional time dependence.
Fixed effects describe specified mean relationships. Random effects represent variation across modeled units or clusters through a distribution; the word random does not mean the software selects predictors randomly. The target of inference and sampling structure help determine their treatment.
Check the model's outcome distribution, variance assumptions, covariance structure, and diagnostics. Classical repeated-measures ANOVA can require sphericity for within-subject factors with more than two levels. Mixed-effects models allow other covariance structures, but those structures also require justification.
Unequal measurement schedules and some missing outcomes can be accommodated by suitable likelihood-based models under appropriate missingness assumptions. This is not permission to ignore informative dropout, inadequate clusters, or systematic measurement differences.
Three practical examples
- Staff training: compare two teaching methods while measuring the same employees at baseline and follow-up.
- AI assistance: assign people to interface versions, then measure their performance across several task difficulties.
- Branch operations: model repeated monthly service measures with shared effects of policy and variation among branches.
Think of it this way
Think of a choir performing several songs. Some comparisons concern different choirs; others concern the same choir across songs. Treating every performance as an unrelated choir loses the structure. A model can additionally describe common musical patterns and differences among choirs.
Worked example
Consider a fictional balanced study with two independent groups measured before and after training. Group A has mean scores of 50 before and 64 after. Group B has means of 50 before and 53 after.
The mean changes are 14 and 3 points, so the difference in changes is 14 - 3 = 11 points. With group B and baseline as reference categories, a saturated group-by-time mean model has intercept 50, group coefficient 0, time coefficient 3, and interaction coefficient 11. These coefficients reproduce all four means.
The four means alone do not determine a standard error or p-value. Sample sizes, within-person variation, and dependence are still needed.
Separately, suppose a random-intercept model has person-level variance 9 and independent residual variance 16, with no other random effects. Each observation's variance is 25, and two observations from the same person share covariance 9. Their implied correlation is 9/25 = 0.36. This illustrates one covariance assumption, not a fit to the four means.
Illustrative case study
Fictional case: Harbor Learning compares an AI coaching program with existing training and measures each worker on three occasions. The analyst identifies program as a between-person factor and occasion as a repeated factor.
A mixed-effects analysis includes the program-by-occasion interaction and evaluates the plausibility of its covariance assumptions. The report distinguishes estimated learning patterns from uncertainty and discusses workers who missed later assessments. The company does not count three observations per worker as three independent workers.
Common misconceptions and mistakes
Do not confuse a mixed design with mixed methods research combining qualitative and quantitative evidence. Also distinguish mixed-effects models from mixture models, which represent combinations of component distributions.
A random intercept does not automatically capture every correlation pattern. Repeated observations can have correlations that vary with time separation, and individuals may have different slopes.
Do not infer an interaction merely because one group's change is significant and another's is not. Assess the difference in changes directly. An observed interaction is not automatically causal; assignment, confounding, attrition, and the study question still matter.
Business use
Businesses use these designs when they want to compare groups while following the same customers, workers, devices, or branches over time. Planning should address the number of independent units, measurement schedule, expected attrition, and relevant interaction. Additional repeated measurements can improve information, but they do not simply substitute for additional independent people or clusters.
People also ask
Does mixed always mean fixed and random effects?
No. In design terminology it can mean between-subjects and repeated factors.
Can a mixed-effects model analyze a mixed design?
Yes, when its mean and covariance assumptions suit the question and data.
Does it solve missing-data bias automatically?
No. Its validity depends on missingness and modeling assumptions.
Check your understanding
In a program-by-time study, which factor is repeated?
Answer: Time, when the same participants are observed at each occasion.
What does the interaction describe?
Answer: How the pattern over time differs between programs, rather than simply whether either program changes.
Related terms
- Sphericity
- repeated measures
- between-subjects design
- mixed-effects model
- random intercept
- interaction
- covariance structure
- missing at random
Learn next
draw the study's hierarchy of observations and participants, then compare a group-by-time mean model with alternative plausible covariance structures.
More in Statistics and probability
- Minimization (clinical trials) - Minimization in clinical trials is a sequential allocation method that seeks to keep treatment groups balanced on selected participant characteristics.
- Minimum chi-square estimation - Minimum chi-square estimation chooses model parameters to minimize a chi-square discrepancy between observed data and model predictions.
- Misuse of p-values - Misuse of p-values means calculating, selecting, or interpreting p-values in ways that support conclusions the statistical procedure does not justify.
- Mixed-data sampling - Mixed-data sampling, usually abbreviated MIDAS, is a regression approach for relating time series observed at different frequencies.
- Mixed sampling - Mixed sampling is a sampling plan that combines two or more basic selection methods.
- Mixture distribution - A mixture distribution is a probability distribution formed by combining component distributions with specified probabilities, called mixing weights.
Sources
Sources: David Howell, University of Vermont, glossary entry on mixed model designs: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html . Statsmodels, Linear Mixed Effects Models: https://www.statsmodels.org/stable/mixed_linear.html . Both pages were opened; the glossary's two meanings are explicitly distinguished. Numerical illustration and fictional case are original.
Previous: Mixed methods Next: Mixed Precision