Does the theorem say every likelihood ratio is chi-square distributed?

No. It concerns a transformed statistic under a null hypothesis, regularity conditions and a large-sample limit.

Why multiply the log-likelihood difference by two?

That scaling produces the conventional chi-square limiting statistic.

Is this the same as Wilks' lambda in multivariate analysis?

They are related through likelihood-ratio ideas, but the theorem and that named multivariate statistic are distinct concepts.

AI Glossary / Statistics and probability / Advanced

Wilks' theorem

Definition

Wilks' theorem states that, under suitable regularity conditions and a true null hypothesis, twice the difference between maximized log likelihoods of nested models approaches a chi-square distribution as sample size grows. Its degrees of freedom equal the difference in the models' effective parameter dimensions. It supplies the reference distribution for many likelihood-ratio tests.

What it means

A larger model can fit observed data at least as well as a nested smaller model because it has more freedom. The relevant question is whether the improvement is unusually large if the smaller model is actually adequate. Wilks' theorem provides an approximate scale for judging that improvement.

The result is asymptotic: it describes a limiting distribution, not an automatic exact statement for any dataset. Model structure and assumptions determine whether the approximation is justified. A software package returning a chi-square p-value does not itself establish that those assumptions hold.

How it works

Maximize the likelihood under the full model and again under the null constraints. Write their maximized log likelihoods as ℓfull and ℓnull. Calculate T = 2(ℓfull − ℓnull), equivalently minus twice the log of the restricted-to-unrestricted likelihood ratio.

For a regular comparison with q independent restrictions, use a chi-square reference distribution with q degrees of freedom. Conditions include suitable smoothness, identifiable parameters and a regular interior null rather than a problematic boundary. The same idea can construct confidence regions by retaining parameter values whose constrained likelihoods are not too far below the unrestricted maximum. Optimization failures must be resolved before interpreting the statistic.

Three practical examples

1. Customer analytics: An analyst compares a logistic model with and without a prespecified group of predictors. If the nesting and regularity conditions hold, a likelihood-ratio test assesses the added terms jointly.

2. Count modeling: A team tests a specified event-rate value within a suitable likelihood model. The theorem can provide an approximate reference for the loss of fit imposed by that restriction.

3. Research review: A reviewer notices that a variance component is being tested at zero. Because zero is a boundary for a nonnegative variance, the ordinary Wilks reference must be examined rather than applied automatically.

Think of it this way

Think of comparing a flexible measuring tool with the same tool locked into a restricted setting. The flexible version will fit at least as well. Wilks' theorem tells you how much extra fit could arise from ordinary sampling variation under regular conditions. It does not say the flexible tool's additional complexity is always worthwhile.

Worked example

Consider an original example of 100 independent binary outcomes, with 60 successes. Assume a common success probability p. The full-model maximum-likelihood estimate is 0.60, while the null fixes p at 0.50. The binomial coefficient cancels when comparing the two likelihoods.

The statistic is T = 2[60 log(0.60/0.50) + 40 log(0.40/0.50)] ≈ 4.0271. The full model has one free parameter and the null has none, giving one degree of freedom. The chi-square upper-tail approximation is about 0.0448.

At a prespecified 5% significance level, this approximate test rejects p = 0.50. It is not an exact binomial-test calculation and need not agree numerically with an exact test. The observed estimate differs from the null by ten percentage points; whether that difference matters practically requires context. The p-value is not the probability that the null is true or that a particular business intervention caused the observed successes.

Illustrative case study

In this fictional case, Willow Insights compares a baseline response model with a version containing two additional predictors selected before analysis. The likelihood-ratio calculation suggests improved fit. The analyst verifies that both models use the same observations and compatible likelihood definitions, and that both fits converged.

The team then checks held-out prediction and the practical size of the effects. It reports the test as evidence about the specified nested restriction, rather than evidence that the expanded model is universally superior. A separate exploratory search over many alternative predictors is labeled as exploratory and is not silently treated as the same prespecified comparison.

Common misconceptions and mistakes

Do not apply the usual dimension-difference rule to arbitrary nonnested models. Boundary parameters, unidentified components, singular models and some mixture-model comparisons can have different limiting distributions. Small samples can also make a valid asymptotic result a poor numerical approximation.

A likelihood-ratio statistic should be nonnegative when both nested likelihoods are correctly maximized. A negative result can indicate numerical problems or incompatible fits. Penalized objectives are not automatically interchangeable with ordinary maximized log likelihoods. Neither a significant result nor a more flexible model guarantees better out-of-sample predictions, meaningful effect sizes or a credible causal interpretation.

Business use

Wilks' theorem underpins familiar statistical comparisons in business regression, model checking and uncertainty estimation. Its practical contribution is to turn a difference in likelihood fit into a calibrated approximate test when the assumptions are appropriate.

A useful report states both models, the restrictions, observation count, likelihood-ratio statistic, degrees of freedom and approximation used. Explain any boundary or optimization concerns before decision-makers rely on the p-value. For high-impact comparisons, assess approximation quality through a method justified for the model rather than assuming that a large total dataset always supplies enough information about every parameter.

Check your understanding

If the full model has five regular free parameters and the nested null has three, what is the reference degrees of freedom?

Answer: Two, provided the restrictions are independent and the theorem applies.

Can a boundary null be justified merely by increasing sample size?

Answer: No. The limiting distribution itself may differ from the ordinary Wilks result.

Related terms

Likelihood-ratio test
maximum likelihood
nested models
chi-square distribution
regularity conditions
parameter identifiability
boundary parameter
profile likelihood

Learn next

Next, calculate a simple likelihood ratio by hand and compare exact and asymptotic results where both are available. Study profile-likelihood confidence intervals, then examine boundary cases to understand why counting parameters is not sufficient to justify every test.

Sources

Charles Geyer, University of Minnesota, “Likelihood Inference,” explains nested models, the Wilks statistic, regularity requirements and likelihood-based confidence intervals: https://www.stat.umn.edu/geyer/5421/notes/likelihood.html

The binomial calculation and Willow Insights case are original teaching illustrations. The numerical p-value uses the stated asymptotic chi-square reference, not an exact finite-sample distribution.

Previous: Wilcoxon signed-rank test Next: Win statistics