How is KDE different from a histogram?

It uses smooth kernel contributions rather than counts within fixed bins.

Can density exceed one?

Yes. A density's total area is one, while its height depends on the measurement scale and concentration.

Does KDE work well in every dimension?

No. High-dimensional density estimation can require very large amounts of data.

AI Glossary / AI and machine learning / Intermediate

Kernel density estimation

Definition

Kernel density estimation is a nonparametric method for estimating a probability density from observed data. It places a small smooth contribution around each observation and adds these contributions to form an estimated distribution.

What it means

A histogram groups values into bins. Kernel density estimation, usually shortened to KDE, offers a smooth alternative. Instead of choosing bin edges, it uses a kernel shape and a bandwidth that controls how widely each observation contributes.

The curve shows estimated density, not a direct probability for an exact value. Probability over an interval corresponds to area under the curve. A density can be greater than one when values are concentrated in a narrow range; this does not violate probability rules because total area, rather than peak height, must equal one.

Nonparametric does not mean assumption-free. Kernel choice, bandwidth, boundary treatment and data quality all influence the estimate. KDE is useful for exploration and some analytical methods, but a smooth curve can conceal uncertainty when the sample is small or unrepresentative.

How it works

For one-dimensional data, each observation contributes a rescaled kernel centered on its value. The contributions are averaged, with normalization accounting for bandwidth. A Gaussian kernel produces bell-shaped contributions, though other kernels are possible.

Bandwidth is especially important. A very small bandwidth can produce a jagged estimate that follows individual observations. A very large bandwidth can merge meaningful peaks and hide structure. Automatic rules provide starting points, and validation or domain knowledge may suggest alternatives.

KDE extends to several dimensions, but data requirements grow quickly as dimension increases. Ordinary kernels can also allocate density outside a natural boundary, such as below zero for waiting times. Analysts may transform the data or use a boundary-aware method. Missing observations, sampling weights and dependence require attention before interpreting the resulting curve.

Three practical examples

1. A service analyst plots a density estimate of waiting times to explore whether one average hides several common waiting patterns.

2. A manufacturing team compares measurement distributions across production periods while checking that apparent peaks are stable across reasonable bandwidths.

3. A data-science lesson overlays a histogram and several KDE curves to show how smoothing choices change the visual story.

Think of it this way

Think of placing a small mound of sand at each observed value and looking at the combined landscape. Wider mounds create a smoother landscape. The limit is that a density estimate has mathematical normalization and sampling uncertainty; an attractive shape does not establish that every hill is a real population feature.

Worked example

Take a toy sample with two observations, 0 and 2, and a Gaussian kernel with bandwidth 1. At x = 1, each observation is one bandwidth away. The standard Gaussian density there is approximately 0.242.

The KDE is the average of the two contributions: (0.242 + 0.242) / 2 = 0.242. This value is a density per unit of the measurement scale, not a 24.2% probability that the next observation equals exactly 1.

To estimate the probability of an interval, the curve must be integrated over that interval. With only two observations, the estimate is also highly uncertain; the example demonstrates the calculation rather than a sound practical analysis.

Illustrative case study

A fictional customer-service team sees two peaks in a KDE of response times and concludes that it has two distinct customer groups. An analyst changes the bandwidth slightly and finds that one peak disappears.

The team examines the raw data, timestamps and process stages. It discovers that rounding and a scheduled batch process contribute to the visual pattern. The final report shows the distribution with an explanation of those operational effects rather than inventing customer categories.

The case illustrates that KDE is a way to inspect data, not a mechanism for proving the existence or cause of hidden groups. Visual patterns need supporting evidence.

Common misconceptions and mistakes

Every peak identifies a real subgroup. Peaks can depend on bandwidth, rounding and sample variation.
Density height is the probability of an exact value. Probabilities for continuous variables come from areas over intervals.
Nonparametric means no choices are required. Smoothing and data assumptions remain important.

Business use

KDE can make distributions easier to discuss when a single average would be misleading. It is useful for exploratory dashboards and comparisons of continuous measurements, provided viewers understand what the vertical axis means.

Show sample size, measurement units and the smoothing approach. Compare reasonable bandwidths before presenting a peak as a substantive finding. Use particular care with small samples, bounded quantities and high-dimensional data. If a decision depends on a tail probability or threshold, assess that estimate directly rather than relying on the appearance of the plotted curve.

Check your understanding

A KDE curve has height 0.3 at a waiting time of five minutes. Does that mean a 30% chance of waiting exactly five minutes?

Answer: No. The height is density; an interval probability is obtained from the area under the curve over that interval.

Related terms

Histogram
Kernel
Smoothing
Curse of dimensionality
probability density
bandwidth
nonparametric statistics

Learn next

Read probability density and bandwidth selection. Plot one dataset with several bandwidths and describe which conclusions stay stable and which depend on the smoothing choice.

Smoothing

Sources

SciPy, official gaussian_kde documentation: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.gaussian_kde.html. It describes Gaussian KDE, bandwidth selection and limitations including oversmoothing multimodal data. The two-observation calculation and service-team scenario are invented teaching examples.

Previous: Kernel Next: Kernel embedding of distributions