Is it suitable when variables outnumber observations in a group?

The ordinary nonsingular covariance calculation is not; a different approach requires justification.

Does rejection identify the differing matrix entries?

No. Follow-up examination is needed to understand which variances or covariances differ.

AI Glossary / Statistics and probability / Advanced

Box's M test

Definition

Box's M test is a statistical test of whether several populations have the same covariance matrix. It compares covariance information estimated separately within groups with a pooled estimate. The classical test relies on multivariate normality and independent observations, and commonly uses a corrected statistic with an approximate chi-square reference distribution.

What it means

A covariance matrix describes both the variability of individual measurements and how those measurements vary together. Two groups can have similar average values while differing substantially in this structure. Box's M test addresses that structure rather than equality of group means.
In an AI evaluation, groups might represent operating environments, and measurements might include two continuous performance indicators. A shared covariance assumption says that their variances and covariances are the same across environments. The test asks whether the observed covariance differences provide evidence against that assumption under its statistical model.

How it works

Estimate a sample covariance matrix within each group. Pool these matrices using weights equal to each group's sample size minus one. Compare the logarithm of the pooled determinant with the corresponding weighted sum of the group log determinants. Larger discrepancies provide evidence against a common covariance matrix.
For g groups, p variables, and total sample size N, the uncorrected statistic is M = (N-g) log|Sp| minus the sum of (ni-1) log|Si| across groups. Here Sp is the pooled matrix, Si is group i's matrix, and vertical bars denote a determinant.
A commonly used correction rescales M before comparison with a chi-square distribution having (g-1)p(p+1)/2 degrees of freedom. Covariance matrices must be nonsingular for this ordinary calculation. Having more observations than variables in each group is necessary, but exact linear dependencies can still make a matrix singular.

Three practical examples

A research team compares two continuous error measurements across operating environments before assessing whether a shared covariance model is plausible.
An analyst preparing a multivariate analysis of variance examines whether response variability and relationships differ across treatment groups, alongside plots and distributional diagnostics.
A classification study investigates a common-covariance assumption associated with linear discriminant analysis. The team also evaluates predictive performance rather than choosing a classifier from one diagnostic p-value alone.

Think of it this way

Imagine each group's measurements as a cloud of points. The clouds may share a center yet differ in width, height, or tilt. Box's M test concerns the covariance structure describing those features. Moving a whole cloud without changing its shape would change its mean, but not its covariance matrix.

Worked example

Consider two invented groups, each containing 21 independent observations on two continuous variables. Suppose their sample covariance matrices are S1 = [[1, 0], [0, 4]] and S2 = [[4, 0], [0, 1]]. The first variable is more variable in group 2; the second is more variable in group 1.
Both matrices have determinant 4 and trace 5. Equal determinants and equal total variance therefore do not establish equal covariance matrices. Their directions of greater spread differ.
Each group has 20 covariance degrees of freedom. The pooled matrix is (20S1 + 20S2)/40 = [[2.5, 0], [0, 2.5]], whose determinant is 6.25.
The uncorrected statistic is M = 40 log(6.25) - 20 log(4) - 20 log(4), approximately 17.8515. For this two-group, two-variable example, the usual correction coefficient is c = (13/18)(1/20 + 1/20 - 1/40), approximately 0.0541667.
The corrected statistic is (1-c)M, approximately 16.8845, with (2-1) times 2 times 3 divided by 2 = 3 degrees of freedom. The chi-square approximation gives a p-value of approximately 0.000746. Under the assumed model and a prespecified 0.05 threshold, this provides evidence against equal population covariance matrices.
The calculation uses hypothetical sample summaries, not observations from an actual business. Its interpretation is conditional on the model assumptions. If the data were strongly nonnormal, the same small p-value would need careful investigation rather than an automatic covariance-only explanation.
As a check, if both sample covariance matrices were S1, the pooled matrix would also be S1. The log-determinant terms would cancel and M would equal zero. That algebraic agreement still would not prove that the unknown population matrices are exactly equal.

Illustrative case study

Fictional company Cedar Analytics studies two continuous quality measurements for an AI-assisted workflow across three sites. Average scores look similar, but scatterplots suggest that the measurements co-vary differently at one site.
The analyst calculates Box's M test and inspects group distributions, sample sizes, outliers, and measurement procedures. A small p-value prompts further investigation of both covariance differences and possible departures from normality. The team compares models that allow different covariance structures and evaluates their practical behavior. It does not describe the diagnostic as proof that one site's workflow is inferior. This is a fictional teaching scenario.

Common misconceptions and mistakes

The test does not test equality of means. Groups can have different means and equal covariances, or equal means and different covariances.
A nonsignificant result does not prove the common-covariance assumption. Limited sample size can reduce sensitivity.
Box's M is sensitive to departures from multivariate normality. Outliers and skewed distributions can affect its interpretation, so it should not be treated as an assumption-free detector.
Equal determinants do not imply equal matrices, as the worked example shows. Likewise, testing each variable's variance separately ignores off-diagonal covariance relationships.

Business use

Use the test as one diagnostic when a planned multivariate model relies on shared covariance across groups. Review covariance plots and the scale of the differences, not only significance. Document sample sizes, missing-data handling, distributional concerns, and the consequences of choosing a more flexible model.

Check your understanding

Do S1 and S2 have equal determinants?

Answer: Yes, both are 4.

Are they equal matrices?

Answer: No.

Why can the pooled determinant differ?

Answer: Pooling combines the different directional variance patterns.

Related terms

Covariance matrix
Linear discriminant analysis
Determinant
homogeneity of covariance
multivariate normality
MANOVA
chi-square approximation

Learn next

Inspect covariance ellipses and study alternatives that accommodate heterogeneous covariance. Distinguish testing model assumptions from assessing whether a model performs adequately for its intended purpose.

Sources

Friendly and Fox, heplots documentation, Box's M-test for Homogeneity of Covariance Matrices: https://friendly.github.io/heplots/reference/boxM.html . Implementation formulas: https://github.com/friendly/heplots/blob/master/R/boxM.R . Worked matrices and company scenario are original illustrations.

Previous: Box plot Next: Boyce-Codd Normal Form