Does “multivariate” mean many separate bell curves?

It means a joint distribution over several variables. Separate normal marginals alone are insufficient to establish a multivariate Gaussian.

Can the variables be correlated?

Yes. Off-diagonal covariance entries describe their linear co-variation, subject to the matrix being valid.

Must the covariance matrix be invertible?

The ordinary full-dimensional density uses an invertible positive definite matrix. Degenerate Gaussian distributions exist with singular covariance but require a different treatment of their support.

AI Glossary / Graphs and symbolic AI / Intermediate

Multivariate Gaussian

Definition

A multivariate Gaussian, also called a multivariate normal distribution, is a probability distribution for a vector of numerical variables considered jointly. It is specified by a mean vector and a covariance matrix. It generalizes the familiar bell-shaped normal distribution while allowing variables to vary together, rather than assuming that every coordinate changes independently.

What it means

The mean vector gives the center in each coordinate. The covariance matrix describes variation within coordinates and co-variation between them. In two dimensions, the ordinary nonsingular distribution has elliptical contours of equal density; their orientation reflects the covariance structure.
This is a model for a whole vector, not merely a statement that each variable separately looks bell-shaped. Normally distributed individual variables do not automatically form a jointly Gaussian vector. Their dependence structure matters. That distinction becomes important when a system scores combinations of measurements that look unremarkable one variable at a time.

How it works

One construction starts with independent standard normal variables, combines them linearly, and adds a mean vector. The transformation controls the resulting spread and dependence. A valid covariance matrix must be symmetric and positive semidefinite; the usual full-dimensional density formula requires positive definiteness.
To evaluate that ordinary density, measure displacement from the mean using the inverse covariance matrix and apply an exponential decay with the appropriate normalization. This accounts for both scale and direction. Singular Gaussian distributions need special treatment because their variation lies in a lower-dimensional set. A numerical library's acceptance of a covariance matrix is not evidence that the distribution fits a dataset well.

Three practical examples

A sensor-analysis team can model two measurement errors jointly when they share a source of variation.
A simulation can generate correlated numerical inputs rather than drawing every variable independently.
An anomaly-detection experiment can compare observations with a fitted Gaussian reference, while checking whether skew, heavy tails, or multiple operating regimes undermine that reference.

Think of it this way

Think of a cloud of dots centered on a map. A round cloud spreads similarly in every direction; a tilted oval suggests that two coordinates tend to move together. The analogy is limited because a Gaussian is an entire probability distribution, including tails beyond any plotted sample, and a cloud that looks oval need not follow it exactly.

Worked example

Construct an invented two-variable Gaussian from independent standard normal variables Z1 and Z2. Define X = 10 + 2Z1 and Y = 20 + Z1 + 3Z2. The mean vector is (10,20).
X has variance 4. Y has variance 1 + 9 = 10 because the independent components contribute separate variances. Their covariance is 2: only the shared Z1 term contributes, giving 2 × 1 × Var(Z1) = 2. Thus the covariance matrix is [[4,2],[2,10]]. Its determinant is 4 × 10 − 2 × 2 = 36, and its first diagonal entry is positive, so this symmetric two-dimensional matrix is positive definite.
The correlation is 2 divided by the square root of 4 × 10, approximately 0.316. The variables are positively related but not locked together. Also, X + Y has mean 30 and variance 4 + 10 + 2 × 2 = 18. Adding the individual variances alone would incorrectly give 14 because it would ignore covariance. These are exact properties of the constructed model, not estimates from observed data.

Illustrative case study

In a fictional packaging line, two sensor readings are monitored separately. Each has a familiar range, but staff notice combinations that seem unusual even though neither reading alone is extreme. An analyst fits a joint reference model and inspects the covariance pattern.
The analysis also reveals two operating modes with different centers. Rather than describing all variation with one Gaussian automatically, the team compares mode-specific models and checks held-out observations. The example illustrates both the usefulness and the limits of joint modeling; it does not report an achieved detection rate or recommend one model for every production process.

Common misconceptions and mistakes

Covariance is not correlation: covariance depends on measurement units, while correlation rescales by standard deviations. A covariance matrix also cannot be assembled from arbitrary plausible-looking numbers; its entries must be jointly compatible with a valid distribution.
Within a jointly Gaussian vector, zero covariance between coordinates implies independence. Outside that setting, zero covariance generally does not establish independence. Finally, probability density at a single point is not the probability of that exact point. For an ordinary continuous Gaussian, probabilities are assigned to regions by integrating density.

Business use

The model offers a compact representation of joint numerical uncertainty. It can support simulation, probabilistic inference, and diagnostic scoring when its assumptions are reasonable. Its manageable mathematics often makes it a useful baseline against which more complex models can be compared.
Business validation should examine the data-generating context, sample size, changing conditions, and the consequences of tail errors. Estimating a large covariance matrix from little data can be unstable. Record preprocessing and units, and assess whether a single joint distribution is appropriate before interpreting a numerical score as operationally meaningful.

Check your understanding

In the worked construction, what is the variance of X − Y?

Answer: Var(X − Y) = 4 + 10 − 2 × 2 = 10. Equivalently, X − Y = −10 + Z1 − 3Z2, whose independent random terms have variances 1 and 9.

Related terms

Covariance matrix
Correlation
Mahalanobis distance
Gaussian distribution
mean vector
joint distribution
Gaussian mixture
positive definite matrix

Learn next

Study covariance transformations next. Replace Y's shared Z1 coefficient with −1 and recompute the covariance and correlation. The variances remain 4 and 10, but covariance becomes −2. Explain how the cloud's tilt changes while each coordinate's individual spread remains unchanged.

Correlation

Sources

SciPy documentation, multivariate_normal, parameters and density notes: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.multivariate_normal.html . Marco Taboga, original lecture on the multivariate normal distribution: https://www.statlect.com/probability-distributions/multivariate-normal-distribution . Relevant definitions and construction discussion were read; the numerical construction is original.

Previous: Multivariate analysis of variance Next: Multivariate kernel density estimation