Must a heavy tail be a power law?

No; the lognormal is a common counterexample.

Does heavy-tailed mean every extreme is common?

No. Extremes can remain rare while being much more likely than a light-tailed model predicts.

AI Glossary / Statistics and probability / Intermediate

Heavy-tailed distribution

Definition

A heavy-tailed distribution assigns substantial probability to extreme values because its tail is not exponentially bounded. Under a common mathematical definition, a nonnegative random variable is heavy-tailed when E[exp(tX)] is infinite for every t > 0. Terminology varies across fields, so an analysis should state its definition and distinguish heavy tails from the looser description of having more extremes than a normal distribution.

What it means

The tail is the part of a distribution far from its central values. Its behavior determines how quickly the probability of exceptionally large outcomes decreases as the threshold rises. A heavy right tail concerns large positive values; distributions can also have heavy left tails or both.
Heavy-tailed does not simply mean a large standard deviation. Rescaling a normal distribution makes it more spread out without changing its light-tailed character. Nor does heavy-tailed automatically mean that the mean or variance is infinite.

How it works

Study the survival function P(X > x), which gives the probability of exceeding a threshold. Exponential tails decrease roughly as exp(−cx) for a positive rate c. A Pareto tail decreases as a power of x, which is eventually much slower.
Specify which definition is intended. Under the exponential-moment definition, lognormal distributions are heavy-tailed even though all positive integer moments are finite. Some applied discussions use narrower definitions involving power laws or infinite variance; those are not equivalent to the broader definition.
For a Pareto Type I distribution with minimum x_m > 0 and tail index α > 0, survival is (x_m/x)^α for x ≥ x_m. The mean is finite only if α > 1 and the variance only if α > 2. Smaller α gives a heavier tail within this family.
The existence of moments affects familiar statistical methods. With a finite mean, an appropriate law of large numbers can still apply even if variance is infinite. The usual independent, identically distributed finite-variance central limit theorem does not apply when variance is infinite. Avoid claiming that all averaging fails for every heavy-tailed distribution.
Inspect data with tail plots, quantiles, and comparisons among plausible models. A few large observations do not uniquely identify a tail family, and a roughly straight line on logarithmic axes does not prove a power law. Threshold choice, sample size, dependence, and truncation can substantially affect inference.
Separate genuine extreme observations from recording mistakes through investigation. Automatically deleting extremes can remove the phenomenon of interest. Robust estimation can reduce sensitivity for some objectives, but it does not eliminate the need to understand the tail when extreme outcomes drive decisions.
Remember that observed data may be bounded by physical limits or collection rules. A truncated distribution can behave like a heavy-tailed model over an observed range while having different asymptotic properties.

Three practical examples

A computing team examines whether unusually long job durations dominate resource planning.
An AI evaluator studies a prediction-error distribution whose rare large errors are poorly summarized by average performance alone.
An operations analyst compares models for transaction sizes when a small number of very large observations materially affect totals.

Think of it this way

Think of a pile containing mostly small stones but occasional enormous rocks. The rocks may dominate the total weight, and a short inspection can easily miss them. The analogy concerns tail behavior, not a universal rule that every sample must contain an extreme.

Worked example

Consider an invented Pareto distribution with minimum one and tail index α = 2. Its mean is two, but its variance is infinite. Compare it with an exponential distribution of rate 0.5, which also has mean two.
For the Pareto model, P(X > 10) = 10^(−2) = 0.01. For the exponential model, P(X > 10) = exp(−5) ≈ 0.006738.
At a threshold of 100, the Pareto tail probability is 0.0001, whereas the exponential tail probability is exp(−50) ≈ 1.93 × 10^(−22). Matching the means therefore does not match extreme-outcome probabilities.
The models have different supports and shapes, and this comparison is not a fit to real data. It isolates the importance of tail assumptions. The finite Pareto mean also shows that infinite variance does not imply an infinite mean.

Illustrative case study

Fictional case: Aspen Computing provisions capacity using a model that matches typical job durations. Several rare long jobs then consume far more resources than anticipated. Analysts investigate data quality, examine the upper tail, and compare plausible duration models using held-out observations and sensitivity analysis. They avoid declaring a power law from a few outliers. This invented case illustrates why central fit and tail fit can support different conclusions.

Common misconceptions and mistakes

Do not equate heavy tails with skewness: a symmetric Student's t distribution can have heavy tails. Do not assume every heavy-tailed distribution has infinite variance; lognormal distributions provide a counterexample.
A large finite sample standard deviation cannot establish whether the population variance exists. Nor does the absence of extreme observations in one dataset prove light tails.
Robust summaries such as medians are useful, but do not replace tail analysis when the total cost is driven by extremes.

Business use

Heavy-tail awareness can improve capacity planning and evaluation of systems where rare large outcomes matter. Businesses should report relevant upper quantiles, data limitations, and sensitivity to tail assumptions alongside averages. AI models trained with squared-error objectives can be especially influenced by large errors, making the chosen loss and evaluation distribution important.

Check your understanding

Why do the example's models differ despite equal means?

Answer: Their tail decay differs.

Which Pareto moment fails at α = 2?

Answer: The variance is infinite.

Why should a definition be stated?

Answer: Different fields sometimes use heavy-tailed for different classes of distributions.

Related terms

Moment generating function
Robust statistics
Tail probability
Pareto distribution
lognormal distribution
extreme value
regular variation
truncation

Learn next

Learn survival functions and moment existence. Then compare Pareto, lognormal, and exponential tails and study the limits of tail estimation from finite samples.

Sources

Sargent and Stachurski, QuantEcon, “Heavy-Tailed Distributions”: https://intro.quantecon.org/heavy_tails.html Foss, Korshunov and Zachary, Heavy Tails in Probability: https://oa.tib.eu/renate/bitstreams/d9bdb154-ebe3-4664-8566-0c1bd4fcc1f9/download

Previous: Heatmap Next: Hebbian learning