Why use a log-likelihood?

Logarithms preserve the maximizing parameter while converting products into sums and reducing numerical underflow.

Can a likelihood be normalized into a posterior?

A Bayesian posterior requires a prior and a finite normalizing integral or sum; likelihood alone does not specify a unique prior.

Does every observation contribute a density value?

No. Discrete outcomes contribute masses, and censored or grouped observations can contribute interval or survival probabilities.

AI Glossary / Statistics and probability / Intermediate

Likelihood function

Definition

A likelihood function is the joint probability mass or density of observed data, treated as a function of unknown model parameters while the data are held fixed. It is commonly written L(theta; data), proportional to p(data given theta). A factor that depends on the observed data but not on theta can be omitted for likelihood-based parameter comparisons within the same setup. The function is not generally a probability distribution over theta. Its construction must reflect what was actually observed, including censoring, dependence and selection where relevant.

What it means

The likelihood function converts a probability model into a tool for learning from data. It assigns different values to candidate parameters according to how they account for the observations under the specified model. Its maximum, shape and relative values support estimation and inference. The shape may be sharp, flat, asymmetric or multimodal, each with consequences for uncertainty and computation. A likelihood function can be perfectly implemented for an inappropriate model, so mathematical optimization does not replace checking the data-generating assumptions.

How it works

Define the observation mechanism and parameter space, then derive the joint distribution of the recorded data. Independent exact observations often contribute probability masses or densities that multiply together. Right-censored survival observations instead contribute survival probabilities, while interval-censored observations contribute probabilities of falling inside intervals. Take logarithms to turn products into sums and improve numerical stability. Optimize or otherwise analyze the resulting function, checking boundaries, identifiability and numerical convergence. Bayesian methods multiply it by a prior; likelihood-ratio procedures compare constrained and unconstrained maxima under suitable conditions.

Three practical examples

Event-rate estimation: build a likelihood from counts and their exposure times rather than treating all counts as directly comparable.
Reliability analysis: combine exact failure times with information from units still operating at the end of follow-up.
Probabilistic machine learning: specify an observed-data objective for fitting model parameters, including the appropriate treatment of missing or censored outcomes. The mathematical form should follow the observation process rather than software convenience alone.

Think of it this way

Think of the likelihood function as a landscape over candidate parameter settings. The observed data determine the landscape, and peaks identify settings that fit those observations relatively well under the model. If some observations are incomplete, their contribution must reflect what is known rather than inventing a precise value. A search algorithm can locate a peak without proving that the landscape itself is the right one.

Worked example

Assume three independent lifetimes follow an exponential model with rate lambda. Two failures are observed at times 2 and 5, while a third unit is right-censored at time 6 because it remains operating when follow-up ends. Under independent, noninformative censoring, the relevant likelihood is [lambda exp(-2lambda)][lambda exp(-5lambda)]exp(-6lambda), or lambda squared times exp(-13lambda), up to an irrelevant factor. The log-likelihood is 2 log(lambda) - 13lambda. Its derivative is 2/lambda - 13, giving the maximum likelihood estimate lambda = 2/13, approximately 0.15385 per time unit. The second derivative is negative. Dropping the censored unit would instead give 2/7, ignoring six units of observed survival exposure. Comparing lambda = 2/13 with 1/13 yields a likelihood ratio of 4/e, approximately 1.4715. This original example assumes a constant hazard and does not validate that assumption.

Illustrative case study

In a fictional case, Spruce Devices estimates failure rates from a pilot in which many units are still functioning at the closing date. An initial analysis uses only failed units. The reliability team rebuilds the likelihood to include each surviving unit's censoring contribution and checks whether the test-ending rule is compatible with the assumed censoring mechanism. It also compares the constant-hazard model with plausible alternatives. The final report separates the numerical estimate from uncertainty and from assumptions about future operating conditions.

Common misconceptions and mistakes

A likelihood function is not formed by multiplying marginal distributions unless the required independence or conditional factorization is justified. Censored observations are not zero failures with zero exposure, nor are their censoring times exact failure times. A maximizer can lie on a boundary, be nonunique or fail to exist. Dropping parameter-dependent terms changes the function and can change the estimate. Density-based likelihoods depend on data units, although correctly formed parameter likelihood ratios are preserved under suitable parameter-independent transformations of the observations.

Business use

Businesses should review likelihood construction as carefully as optimization results. Document the observation unit, exposure, missingness, censoring and dependence assumptions. Inspect whether the selected parameter space represents meaningful operational behavior and whether data identify the parameters sufficiently. Use diagnostic checks, validation and uncertainty assessment before turning estimates into forecasts. When a system reports convergence, verify what objective it optimized and whether alternative starting points or boundary checks reveal a different solution.

Check your understanding

Why does the unit censored at time 6 contribute exp(-6lambda) rather than lambda exp(-6lambda)?

Answer: The observation establishes survival beyond 6, not failure exactly at 6. Its contribution is therefore a survival probability. Adding the lambda factor would incorrectly count an additional observed failure and alter the estimate.

Related terms

Likelihood
Log likelihood
Maximum likelihood estimate
Censoring
Survival function
score function
observed information
identifiability
observation model

Learn next

Practice deriving likelihoods for binomial, Poisson and censored survival data. Then study profile likelihood, nuisance parameters, boundary cases and the conditions required for large-sample likelihood-based intervals and tests.

Likelihood

Sources

NIST/SEMATECH e-Handbook, Maximum likelihood estimation and censored-data likelihoods: https://www.itl.nist.gov/div898/handbook/apr/section4/apr412.htm . Cousins, likelihood-function tutorial: https://arxiv.org/abs/2010.00356 . The censored exponential calculation and fictional device case above are original illustrations.

Previous: Likelihood Next: Likelihood ratio chi- square