Does every method require normal data?

No; assumptions depend on the procedure.

AI Glossary / Statistics and probability / Advanced

Multivariate statistics

Definition

Multivariate statistics studies data in which several variables are recorded together for each observational unit. It examines joint patterns, relationships, uncertainty, and structure that separate one-variable analyses may miss.

What it means

The field includes methods for describing data, reducing dimensionality, modeling several responses, comparing outcome profiles, and investigating relationships between sets of variables. The appropriate method depends on the question, measurement types, and study design.

How it works

Arrange measurements so that their pairing is preserved, usually with observations in rows and variables in columns. Examine the variables and their relationships before selecting a method. Mean vectors, covariance matrices, and correlation matrices provide useful summaries, but do not describe every aspect of a joint distribution.
Examples of multivariate methods include principal component analysis, factor analysis, canonical correlation, and multivariate regression. Their assumptions and purposes differ. A large data matrix does not by itself determine which analysis is suitable.

Three practical examples

A fictional sensor study examines temperature and pressure together because their relationship may reveal operating conditions that separate averages conceal.
A fictional service team models several response measures from the same requests and evaluates combinations relevant to scheduling.
A fictional questionnaire study investigates whether many related measurements can be summarized without losing distinctions important to its purpose.

Think of it this way

A musical chord depends on which notes sound together. A list of how often each note appears does not fully describe the chords. Likewise, preserving combinations of measurements matters when studying a joint pattern.

Worked example

Consider an original four-row dataset with two variables. The paired observations are (1, 1), (2, 3), (3, 2), and (4, 4). Both columns contain the numbers one through four, so both means are 2.5.
For either column, the squared deviations from the mean sum to five. Dividing by n-1=3 gives sample variance 5/3, approximately 1.667.
The paired cross-products of centered values are 2.25, -0.25, -0.25, and 2.25. They sum to four, giving sample covariance 4/3. Dividing that covariance by the product of the two sample standard deviations gives correlation 0.8.
The sample covariance matrix therefore has rows (5/3, 4/3) and (4/3, 5/3). Its diagonal describes the separate spreads, while its off-diagonal entries preserve information about the pairing.
Now change the pairing while keeping the first column unchanged. Use second-column values 4, 2, 3, and 1, producing pairs (1, 4), (2, 2), (3, 3), and (4, 1).
The second column still contains exactly the same four values. Both means and both variances are unchanged. The centered cross-products are now -2.25, 0.25, 0.25, and -2.25, summing to minus four. Covariance becomes -4/3 and correlation becomes -0.8.
Separate means, ranges, and variances cannot distinguish these datasets, yet their joint relationships point in opposite directions. Sorting columns independently would destroy the original pairing and could manufacture a relationship that was never observed.
For the first dataset, the row sums are 2, 5, 5, and 8. Their mean is five and their sample variance is six. The row differences are 0, -1, 1, and 0, with mean zero and sample variance 2/3.
For the rearranged dataset, the sums are 5, 4, 6, and 5, giving variance 2/3. Its differences are -3, 0, 0, and 3, giving variance six. Pairing changes uncertainty in combinations even when separate spreads stay fixed.
A principal-component calculation gives another view of the first dataset. After centering, take the direction that adds the two coordinates and divides by sqrt(2). Its sample variance is half the variance of the sum, or three.
The perpendicular direction subtracts the coordinates and divides by sqrt(2). Its variance is half the difference variance, or one third. These two directions are the covariance matrix's eigenvectors, with eigenvalues three and one third.
The first direction accounts for 3/(3+1/3)=0.9, or 90 percent, of the total sample variance. Retaining it gives a one-dimensional summary of most variation in this small dataset. It still discards the remaining ten percent, which might matter for a particular decision.
This percentage describes these observed measurements under their chosen scaling. It does not establish a latent cause, guarantee predictive accuracy, or prove that future data will have the same structure.
Suppose the second variable is expressed in units ten times larger. Its variance becomes 100 times 5/3, or 500/3. Its covariance with the first becomes 10 times 4/3, or 40/3. Correlation remains 0.8, but an analysis based on the unstandardized covariance now gives the second variable much more numerical weight.
The choice between original units and standardization should therefore reflect the measurement meaning and analysis purpose, not be treated as an invisible formatting step.
Finally, add a third column defined as the sum of the first two. It contributes no new independent measurement information. The enlarged covariance matrix is singular because the columns satisfy an exact relationship. More columns do not necessarily mean more distinct information.

Illustrative case study

Fictional company Birch Monitoring keeps temperature and pressure readings attached to their original sampling intervals. Analysts compare separate summaries with paired plots and joint calculations before choosing a monitoring approach.
When a proposed data-preparation step sorts each column independently, reviewers reject it because it would destroy the observed relationship. Their report documents units, pairing, and the purpose of each summary. This fictional case illustrates that sound multivariate work begins with preserving what the observations mean.

Common misconceptions and mistakes

Similar marginal summaries do not imply similar joint behavior.
Correlation describes a relationship, not a causal explanation.
A component explaining much variance may omit information important to a task.
Changing units can change covariance-based analyses.
Redundant columns do not create additional independent evidence.

Business use

Choose the joint question before selecting a method. In the fictional monitoring project, an operational decision may depend on a combination of readings rather than either reading alone. Check whether a summary preserves the distinctions needed for that decision and whether its assumptions remain suitable.

Check your understanding

What changed when the second column was rearranged?

Answer: The pairing and covariance.

What percentage of variance does the first component explain in the original example?

Answer: Ninety percent.

Does the sum column add independent information?

Answer: No.

Related terms

Learn next

Compare covariance and correlation analyses, then select a multivariate method around a clearly stated question.

Sources

NIST/SEMATECH, Elements of Multivariate Analysis: https://www.itl.nist.gov/div898/handbook/pmc/section5/pmc54.htm and Mean Vector and Covariance Matrix: https://www.itl.nist.gov/div898/handbook/pmc/section5/pmc541.htm . Statsmodels, Multivariate Statistics: https://www.statsmodels.org/stable/multivariate.html . Dataset, calculations, and company scenario are original.

Previous: Multivariate regression Next: Multivariate Student distribution