AI Glossary / Statistics and probability / Advanced

Multivariate outliers

Definition

Multivariate outliers are observations whose combinations of values are unusual relative to the joint pattern in several variables. An observation can be unusual jointly even when none of its individual values is extreme.

What it means

The reference matters: unusual relative to which population, group, time period, or fitted model? A statistical flag identifies a case for investigation. It does not establish that the observation is erroneous or should be deleted.

How it works

One approach measures distance from a reference center while accounting for covariance. Squared Mahalanobis distance is (x-mu)-transpose times the inverse covariance times (x-mu). Large values indicate departure from the reference's elliptical pattern.
Estimated centers and covariances can themselves be distorted by unusual observations. Robust covariance methods can reduce this sensitivity under suitable conditions. The method and threshold must fit the data structure; one global elliptical reference may be inappropriate for distinct groups or curved patterns.

Three practical examples

A fictional sensor system checks whether two readings form an unusual pair, even though both readings pass their separate range checks.
A fictional service team investigates requests with an unusual combination of input length and processing time.
A fictional data-quality exercise distinguishes a rare but genuine operating condition from a mismatched pair of measurements taken at different times.

Think of it this way

A person's coat size and sleeve length may each be ordinary in isolation, yet their combination can be unusual for a particular sizing system. Looking only at the separate ranges misses the relationship between measurements.

Worked example

Consider an original two-dimensional reference with mean (0, 0), variance one in each coordinate, and correlation 0.9. Its covariance matrix has rows (1, 0.9) and (0.9, 1). Values usually move together under this reference.
The squared Mahalanobis distance for a point (x, y) is (x squared minus 1.8xy plus y squared)/0.19. The denominator is the covariance determinant, one minus 0.9 squared.
Compare A=(1, -1) and B=(2, 2). Each coordinate of A is only one standard deviation from its own mean. Each coordinate of B is two standard deviations from its mean. Separate range checks could therefore make B look more unusual.
For A, the joint distance calculation is (1+1.8+1)/0.19=20. For B, it is (4-7.2+4)/0.19=80/19, approximately 4.211. A is much farther from the joint pattern because its coordinates move in opposite directions.
The same result can be understood by rotating the coordinates. The direction (x+y)/sqrt(2) has reference variance 1.9; the perpendicular direction (x-y)/sqrt(2) has variance 0.1.
A has zero displacement along the first direction and squared displacement two along the second. Dividing by the corresponding variance gives 2/0.1=20. B has squared displacement eight along the first direction and zero along the second, giving 8/1.9, approximately 4.211.
Suppose, specifically for this illustration, the reference is a bivariate normal population with known mean and covariance. A squared-distance threshold of approximately 9.210 corresponds to a one-percent upper-tail probability. A exceeds it and B does not.
This calibration depends on those assumptions. Substituting parameters estimated from the same small dataset does not automatically preserve the one-percent interpretation. The example's threshold is not a universal rule for every distance calculation.
Even with the stated known reference and independent observations, screening 1,000 ordinary observations at a one-percent flag rate produces an expected ten flags. The probability of at least one is one minus 0.99 to the power 1,000, approximately 99.996 percent. Some flags are expected under the reference model itself.
Next change measurement units by multiplying the second coordinate by 100. A becomes (1, -100), B becomes (2, 200), and the covariance becomes rows (1, 90) and (90, 10000).
When the points and covariance are transformed consistently, their squared Mahalanobis distances remain 20 and approximately 4.211. Simply changing units should not change this assessment. Keeping the old covariance after changing the measurements would be an inconsistent calculation.
For a separate sensitivity exercise, use a broader reference covariance with rows (4, 2) and (2, 4), retaining mean zero. A's squared distance is now one, while B's is 4/3. These values describe a different reference pattern, not a corrected version of the original calculation.
The comparison shows why a distance score cannot be interpreted without its reference. If an estimated covariance becomes broader because of contamination, an unusual observation may appear less distant. Here the second matrix was chosen manually to expose that sensitivity; it was not estimated from a contaminated sample.
Finally, imagine two tight clusters centered near (-5, 0) and (5, 0). Their pooled center can be (0, 0), even if few observations occur near that location. A distance measured only from that pooled center assigns zero to the empty middle. A method that respects the separate groups would ask a different and more useful question about local unusualness.

Illustrative case study

Fictional company Maple Sensors flags a paired reading that passes both separate range checks. Investigators compare it with the relevant operating group and inspect the measurement timestamps.
They discover that one reading belongs to a later sampling interval. The team corrects the pairing after verifying the source records and records the reason for the change. Other flagged observations remain because they represent valid conditions. This fictional case illustrates investigation rather than automatic deletion.

Common misconceptions and mistakes

An ordinary marginal value can participate in an unusual combination.
A statistical outlier is not necessarily a data error.
A reference fitted to contaminated data can hide unusual cases.
A threshold calibrated for known parameters need not apply unchanged to estimated parameters.
One elliptical reference can miss the structure of multiple groups.

Business use

Use flags to prioritize review and connect them to an operational question. In the fictional sensor setting, retain evidence about the reference group, score, inspection, and any correction. Evaluate whether the review process finds meaningful issues rather than judging it only by how many points it flags.

Check your understanding

Which point is farther from the original joint pattern?

Answer: A.

What is its squared distance?

Answer: Twenty.

Why can ordinary data produce flags?

Answer: The reference model assigns some probability beyond the threshold.

Related terms

Learn next

Compare global and local anomaly scores, then examine robust covariance estimation and threshold calibration.

Covariance

Sources

David Howell, Statistical Methods for Psychology glossary: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html . Scikit-learn, Robust covariance estimation and Mahalanobis distances relevance: https://scikit-learn.org/stable/auto_examples/covariance/plot_mahalanobis_distances.html . Numerical comparisons and company scenario are original.

Previous: Multivariate normal distribution Next: Multivariate procedures