Can two matrices use different distance scales?

Yes, but both measures must make sense for the intended comparison. Pearson correlation is unchanged by positive rescaling.

Does the test require geographic distances?

No. The matrices can describe other meaningful pairwise dissimilarities.

Is a partial Mantel test equivalent to controlling confounding in regression?

No. Its interpretation and validity need separate justification.

AI Glossary / Statistics and probability / Advanced

Mantel test

Definition

A Mantel test assesses association between two matrices of pairwise distances or dissimilarities measured on the same set of objects. A common version calculates the correlation between corresponding off-diagonal distances and evaluates it by permuting object labels in one matrix. It is designed for a question about relationships between dissimilarities. It is not a general replacement for regression, an ordinary correlation test on independent observations, or a universal correction for spatial dependence.

What it means

Suppose each of several products has both a textual description and a customer-rating profile. One matrix might describe how different each pair of descriptions is; another might describe how different each pair of rating profiles is. The question concerns whether pairs that differ more in one representation also differ more in the other.

Although n objects generate n(n − 1)/2 distinct pairwise distances, these distances are not independent observations. Several distances share each object. That dependence is why simply applying an ordinary correlation significance calculation to the flattened distances is generally inappropriate.

How it works

Align the object identifiers in the two matrices. Choose distances that match the substantive question, and use only one triangle of each symmetric matrix, excluding the diagonal. Compute a chosen association statistic, commonly Pearson correlation or a rank correlation.

Under a justified exchangeability assumption, permute object labels in one matrix by applying the same permutation to its rows and columns. Recalculate the statistic after each relabeling. This preserves the internal matrix structure while changing its correspondence with the other matrix.

Compare the observed statistic with the resulting permutation distribution using the prespecified alternative. For random permutations, document the Monte Carlo p-value convention and number of draws. Restrictions on permutations may be needed for a valid design. They must follow the null hypothesis and sampling structure, not merely produce a desired result.

Three practical examples

An AI researcher explores whether distances among document embeddings correspond to independently measured differences in document topics.
A product analyst compares two deliberately chosen measures of dissimilarity among the same product set.
A research team evaluates correspondence between two relationship matrices, after checking that object-label permutations are valid for the study design.

Think of it this way

Think of two maps connecting the same named places, with each map assigning a length to every connection. The test asks whether long connections on one map also tend to be long on the other. It scrambles place names consistently, rather than cutting the individual connections apart and shuffling their lengths independently.

Worked example

Consider an original arithmetic example with four labeled objects at positions 0, 1, 3, and 7 on a line. Their six distances, in the pair order AB, AC, AD, BC, BD, CD, are 1, 3, 7, 2, 6, and 4.

For illustration, let the second distance vector be exactly twice the first: 2, 6, 14, 4, 12, and 8. The Pearson Mantel correlation is 1 because multiplying every distance by a positive constant leaves correlation unchanged.

Enumerating all 4! = 24 object-label permutations of one matrix gives exactly one correlation at least as large as the observed value, allowing for numerical rounding. Under the stipulated unrestricted permutation null, the exhaustive upper-tail p-value is 1/24, approximately 0.0417. The identity permutation is included.

This construction verifies the arithmetic, not an empirical discovery. The second matrix was deliberately created from the first, so presenting the result as evidence from independent real measurements would be misleading. Six pairwise distances also do not turn four objects into six independently sampled objects.

Illustrative case study

Fictional case: CatalogBridge compares semantic distances from an AI model with distances derived from customer descriptions of product differences. The analyst first discovers that the matrices list products in different orders and aligns their identifiers.

The study then finds that several products are variants within the same family. The team revisits the sampling design and permutation assumptions before interpreting a p-value. It reports the descriptive correlation while investigating a suitable inferential approach. The example shows that a clear-looking matrix comparison can still require substantial design work before it supports a statistical conclusion.

Common misconceptions and mistakes

Do not shuffle individual distance entries independently: doing so destroys the shared-object structure. Do not include diagonal zeros or both symmetric triangles as if they supplied additional independent evidence.

A high Mantel correlation does not establish causation or validate either distance measure. Squaring it does not generally give the proportion of variation explained in the original raw variables. Spatially autocorrelated data can undermine standard permutation inference, and a partial Mantel test does not automatically solve that problem. Questions about raw measurements may be better addressed directly with an appropriate model.

Business use

For businesses, the method can support carefully designed comparisons of relationship structures, such as whether two ways of organizing a product collection agree. Its value depends heavily on the relevance of the distances and the units that could legitimately be relabeled.

Record preprocessing, scaling, identifier matching, excluded objects, permutation restrictions, and the chosen alternative. Report the statistic separately from its p-value. For model selection, use independent validation where possible: repeatedly choosing embeddings or distance definitions to maximize correspondence can overstate evidence from the same dataset.

Check your understanding

Why must the rows and columns receive the same permutation?

Answer: Each row and column refers to an object. Applying the same relabeling preserves the distances belonging to each object pair and the symmetry of the matrix. Independent row and column shuffles would change that structure.

Related terms

Permutation test
Distance matrix
dissimilarity
exchangeability
spatial autocorrelation
Pearson correlation
representational similarity
partial Mantel test

Learn next

Study exchangeability and restricted permutations, then compare questions about raw variables with questions about distances. Learn how the choice of distance and preprocessing changes what a matrix association actually means.

Sources

Vegan, official Mantel documentation: https://vegandevs.github.io/vegan/reference/mantel.html

Scikit-bio, official Mantel reference: https://scikit.bio/docs/latest/generated/skbio.stats.distance.mantel.html

Legendre, Fortin, and Borcard, "Should the Mantel test be used in spatial analysis?": https://doi.org/10.1111/2041-210X.12425

The numerical example is original, with all 24 permutations independently enumerated. CatalogBridge is fictional.

Previous: Manner of articulation Next: Many-task computing