AI Glossary / Statistics and probability / Intermediate
Empirical cumulative distribution function
Definition
The empirical cumulative distribution function, or ECDF, gives the proportion of observed sample values that are less than or equal to a specified threshold. For a nonempty, equally weighted sample of fully observed numbers, it is a right-continuous step function rising from zero to one. It describes the sample exactly and can estimate the population cumulative distribution under suitable sampling assumptions.
What it means
An ECDF answers questions such as, "What fraction of these requests finished within five seconds?" The horizontal axis shows possible measurement values, and the vertical axis shows the accumulated fraction of observations at or below each value.
Unlike a histogram, it does not require choosing bins. Unlike a fitted normal curve, it does not impose a parametric distributional shape. However, a precise description of the available data is not automatically an accurate description of future observations or the wider population. Sampling, missingness, and changing conditions still matter.
How it works
For observations x1 through xn, define F_n(t) = (1/n) sum I(x_i <= t). The indicator I equals one when its condition holds and zero otherwise. Count qualifying observations and divide by n.
Sort the values to draw the function efficiently. It stays flat between distinct observations and jumps at observed values. A unique observation contributes a jump of 1/n; if k observations share a value, the jump there is k/n. The use of less than or equal to makes the function right-continuous.
Differences between ECDF values give interval proportions with specific endpoints. F_n(b) - F_n(a) counts observations in (a, b], excluding a and including b. The complement 1 - F_n(t) counts observations strictly greater than t. These distinctions matter when measurements contain ties or when a service rule includes its threshold.
The generalized inverse, the smallest t with F_n(t) at least p, provides one empirical quantile convention. Software may instead interpolate between observations, so percentile calculations should state their convention rather than assuming every package returns the same answer.
An ECDF can be calculated for dependent data, but standard uncertainty statements often assume independent, identically distributed sampling. Weighted data require an explicitly weighted construction. Right-censored durations require a suitable survival estimator rather than treating censoring times as completed event times. Confidence intervals at individual thresholds and simultaneous confidence bands also answer different uncertainty questions.
Three practical examples
- A support team reports the fraction of completed requests resolved by each elapsed-time threshold.
- An AI evaluation team compares ECDFs of absolute prediction errors across two models using the same held-out cases.
- An operations analyst inspects whether a new data batch has a different measurement distribution from an earlier reference batch.
Think of it this way
Think of a running tally at a finish line. At each elapsed time, the tally tells you the fraction of observed participants who have finished by then. Each finish increases the tally, and simultaneous finishes create a larger step.
Worked example
Use an original sample of six fully observed response times in seconds: 2, 3, 3, 5, 8, and 9.
At one second, no observations qualify, so F_n(1) = 0. At two seconds, one qualifies, giving F_n(2) = 1/6. At three seconds, both tied values are included, so F_n(3) = 3/6 = 0.5. The function remains 0.5 at four seconds because no additional observation lies between three and four.
At five seconds, F_n(5) = 4/6, approximately 0.667. At nine seconds, all six observations qualify and the function reaches one.
The fraction strictly above three and at most eight seconds is F_n(8) - F_n(3) = 5/6 - 3/6 = 2/6. The observations are five and eight. Under the generalized-inverse convention, the 75th percentile is eight seconds: the ECDF first reaches at least 0.75 there. An interpolated percentile routine could return another value without changing the ECDF itself.
Illustrative case study
Fictional case: Birch Helpdesk compares two service processes using average completion time. Their averages look similar, but customers complain about occasional long waits.
An analyst plots ECDFs for comparable completed requests and reads the proportions meeting several service thresholds. The team also notices that unresolved requests were excluded from the extract. It separates the completed-request description from an analysis of all requests using appropriate censoring information. The plot improves visibility into the recorded distribution, while the data review prevents an overly optimistic claim about overall service performance.
Common misconceptions and mistakes
Mistake: reading ECDF height as the fraction equal to a value. Equality corresponds to the jump size, not the cumulative height.
Mistake: overlooking ties or endpoint conventions. These can materially change threshold counts.
Mistake: treating the sample maximum as a population upper bound. The ECDF reaches one there because no larger value was observed.
Mistake: smoothing the plotted curve and still presenting it as the exact ECDF. Smoothing changes the representation.
Mistake: interpreting a small visual difference as statistically meaningful without considering sample size, dependence, and how the comparison was selected.
Business use
For businesses, ECDFs make service thresholds and error distributions easier to communicate than a single average. They can reveal whether one process performs well for typical cases while performing poorly for a minority. Use comparable populations, consistent units, and visible sample counts. A useful dashboard pairs the distribution with the operational question and explains exclusions, weights, or censoring that affect interpretation.
People also ask
Can an ECDF decrease?
No. Increasing the threshold cannot remove observations already counted.
Does it work only for continuous data?
No. Ordered discrete values and ties are handled naturally.
Is one minus the ECDF the fraction at or above the threshold?
No. With this convention, it is the fraction strictly above it; including equality requires accounting for the jump.
Check your understanding
Why is the jump at three seconds larger than the jump at two?
Answer: Two observations equal three, while only one equals two.
Does F_n(9) = 1 establish that future responses cannot exceed nine seconds?
Answer: No. It describes the observed sample, not a guaranteed population bound.
Related terms
- Cumulative distribution function
- Empirical distribution
- Quantile
- Percentile
- Survival function
- Histogram
- Censoring
- and distribution shift
Learn next
construct an ECDF from a small sample with ties, compare inverse-ECDF and interpolated quantiles, and study uncertainty bands and appropriate treatment of censored observations.
More in Statistics and probability
- Effective degrees of freedom - Effective degrees of freedom quantify the flexibility or statistical complexity used by a fitting procedure, under a specified definition.
- Effective sample size - Effective sample size is the size of a specified reference sample that would provide comparable statistical information or precision for a particular analysis.
- Elliptical distribution - An elliptical distribution is a multivariate probability distribution obtained by shifting and linearly transforming a spherically symmetric random vector.
- Encoder models - Encoder models transform an input into learned internal representations that can support prediction, matching, or extraction.
- Ensemble Kalman filter - The ensemble Kalman filter, or EnKF, is a method for updating estimates of a changing system by combining an ensemble of model forecasts with incoming...
- Entropy - Entropy, in information theory, measures the average uncertainty associated with a probability distribution.
Sources
Sources: Google Machine Learning Glossary, empirical cumulative distribution function: https://developers.google.com/machine-learning/glossary . SciPy official ecdf documentation, including ties and right-censoring support: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.ecdf.html . The six-response example and case study are original illustrations.
Previous: Empirical Bayes Next: Empirical distribution