Does the estimate integrate to one?

Yes, with normalized kernels and weights.

AI Glossary / Statistics and probability / Advanced

Multivariate kernel density estimation

Definition

Multivariate kernel density estimation estimates a joint probability density for several continuous variables by averaging small density functions centered on observed data points.

What it means

It describes where combinations of values concentrate. The estimate is a density surface, not a list of point probabilities. A region's estimated probability comes from integrating the surface over that region.

How it works

Choose a kernel and a positive-definite bandwidth matrix H, place a scaled kernel at each observation, and average the contributions. With Gaussian kernels, H is each kernel's covariance matrix. Diagonal entries control smoothing along coordinates; off-diagonal entries can orient the smoothing.
Bandwidth selection strongly affects the result. Small bandwidths can emphasize sampling noise; large ones can obscure structure. Selection methods include rules of thumb and cross-validation. Higher-dimensional density estimation becomes increasingly difficult.

Three practical examples

A fictional service team explores the joint distribution of response time and request length, looking for common combinations before defining operational groups.
A fictional laboratory compares two measured sensor features using density contours alongside the underlying observations.
A fictional product team examines session duration and interaction count, checking whether an apparent concentration persists under reasonable smoothing choices.

Think of it this way

Place a small mound over every dot on a map, then average their heights. Narrow mounds reveal individual dots; broad mounds blend nearby dots. The mound widths and orientations affect the landscape, so the landscape is an estimate shaped by both data and smoothing.

Worked example

Consider an original teaching dataset with just two observations: (-1, 0) and (1, 0). These two points are deliberately too few for a convincing practical density study, but they make the calculations transparent.
Use equally weighted two-dimensional Gaussian kernels with covariance H equal to the identity matrix. Each kernel therefore has variance one in both coordinates and zero covariance between them.
At any location (x, y), the first kernel contributes exp(-((x+1) squared plus y squared)/2)/(2 pi). The second substitutes x-1 for x+1. The estimated density is half their sum.
At the midpoint (0, 0), both squared distances are one. The estimated density is exp(-1/2)/(2 pi), approximately 0.09653.
At the observed location (1, 0), the squared distances to the two centers are four and zero. The estimate becomes (exp(-2) plus one)/(4 pi), approximately 0.09035. Thus the fitted surface is higher at the midpoint than at either observation under this smoothing choice.
Now halve each kernel's standard deviation to 0.5. The covariance becomes 0.25 times the identity matrix; it does not become 0.5 times the identity matrix. Variance is the square of standard deviation.
At the midpoint, the new density is 2 exp(-2)/pi, approximately 0.08616. At (1, 0), it is (one plus exp(-8))/pi, approximately 0.31842. Narrower kernels make the observed locations much more prominent than the midpoint in this comparison.
If instead each kernel's standard deviation is doubled to two, H becomes four times the identity. The midpoint density is exp(-1/8)/(8 pi), approximately 0.03511. A lower peak here accompanies broader spreading; it does not mean that the fitted distribution contains less than total probability one.
Return to H equal to the identity and calculate the probability of the square from -1 to 1 in both coordinates. For the kernel centered at (-1, 0), the required horizontal displacement lies between zero and two standard deviations. Its probability is approximately 0.47725. The vertical displacement lies between minus one and one, with probability approximately 0.68269.
Within this kernel the coordinates are independent, so the rectangle probability is their product, approximately 0.32581. The other kernel gives the same result by symmetry. Averaging them therefore leaves 0.32581, or about 32.58 percent. This is an integrated probability, unlike the midpoint density of 0.09653.
Each Gaussian kernel integrates to one over the entire plane. Their half-weighted sum also integrates to one. At any single exact point, however, the probability remains zero under this continuous fitted model.
The mixture mean is (0, 0). Its variance in the first coordinate is two: one from variation between the two centers and one from kernel smoothing. Its second-coordinate variance is one, entirely contributed by smoothing because both observed second coordinates are zero.
That last result is a useful caution. The smooth surface spreads vertically even though these observations provide no vertical spread. The bandwidth was specified manually for instruction. Software that estimates its bandwidth from the sample covariance can fail on such lower-dimensional data; SciPy's gaussian_kde documents this limitation. The manual example does not establish that the underlying population really varies vertically.

Illustrative case study

Fictional company Alder Analytics demonstrates the two-point calculation during analyst training. Participants compare the three bandwidth choices and distinguish heights from integrated probabilities.
For a later operational study, the team collects a larger sample, documents the measurement units, and compares smoothing choices before interpreting contours. Its report includes the observed points so readers can distinguish evidence in the data from detail introduced by the estimate. This is a fictional teaching scenario.

Common misconceptions and mistakes

A high density is not a high probability at an exact point.
A smooth surface does not prove that all visible features reflect the population.
In the example, halving standard deviations quarters kernel variances.
A Gaussian kernel does not require the overall fitted density to be one Gaussian distribution.

Business use

Use joint-density exploration to investigate combinations worth examining further. In the fictional service setting, a low-density request could prompt investigation, but it would not by itself prove a fault or justify an adverse decision.

Check your understanding

What is the midpoint density with identity bandwidth?

Answer: About 0.09653.

What is the estimated square probability?

Answer: About 32.58 percent.

Where does the example's vertical variation come from?

Answer: The chosen kernel bandwidth.

Related terms

Kernel
Cross-validation
Curse of dimensionality
bandwidth matrix
joint density
Gaussian mixture
density contour

Learn next

Compare joint densities with marginal densities, then investigate bandwidth selection and boundary effects for constrained variables.

Sources

Eduardo Garcia-Portugues, Nonparametric Statistics, Section 3.1: https://egarpor.github.io/NP-UC3M/kde-ii.html . SciPy, gaussian_kde: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.gaussian_kde.html . Dataset, arithmetic, and company scenario are original.

Previous: Multivariate Gaussian Next: Multivariate Laplace distribution