Why multiply the log-likelihood difference by two?

This scaling yields the standard likelihood-ratio statistic with its familiar asymptotic chi-square limit under regular conditions.

Can the statistic be negative?

For correctly optimized nested models using the same likelihood, it should be nonnegative apart from numerical issues.

Does a significant result show a large effect?

No. Statistical evidence depends on sample information as well as effect size, so practical magnitude must be reported separately.

AI Glossary / Statistics and probability / Intermediate

Likelihood ratio chi- square

Definition

Likelihood ratio chi-square is a statistic that compares the maximum likelihood under a restricted model with the maximum likelihood under a larger, nested model. It is calculated as twice the difference in maximized log-likelihoods, equivalently minus twice the logarithm of the restricted-to-unrestricted likelihood ratio. Under suitable regularity conditions and sufficiently large samples, its null distribution is approximately chi-square. In contingency-table analysis, the closely associated statistic G-squared compares observed counts with expected counts under a specified null model.

What it means

A more flexible model usually fits the same data at least as well as a restricted one. The likelihood-ratio statistic measures how much fit improves when restrictions are relaxed. The reference distribution helps judge whether that improvement is unusually large under the null model. The statistic is not itself a probability or an effect size. Its chi-square label describes an asymptotic calibration, not a guarantee that every likelihood-ratio comparison has an exact chi-square distribution. Model nesting, parameter boundaries and sample information matter.

How it works

Specify the null and alternative models, verify that they describe the same observations under compatible likelihoods, and fit both by maximum likelihood. Subtract the null model's maximized log-likelihood from the alternative's and multiply by two. For regular nested models, the reference degrees of freedom usually equal the difference in free parameter counts. In a contingency table, G-squared is 2 times the sum of O log(O/E), with zero-observation contributions interpreted by their limiting value. Check expected counts, structural constraints and sampling design before relying on the approximation.

Three practical examples

Categorical association: compare an independence model for a table with a model allowing association.
Regression analysis: test whether additional parameters improve a nested probabilistic model under an appropriate likelihood.
Reliability modeling: compare a constrained lifetime model with a larger family that contains it. A likelihood-ratio test does not replace assessment of practical effect size, model adequacy or predictive performance.

Think of it this way

Think of a restricted model as a rulebook with fewer adjustable settings. The larger model can fit at least as well, but some improvement occurs merely because it has more freedom. The statistic measures the improvement, and the null calibration asks whether that gain is unusually large for the restrictions being tested. It does not announce that the larger model is the true explanation.

Worked example

Consider a two-by-two table with first row counts 30 and 10 and second row counts 20 and 40. Row totals are 40 and 60, column totals are 50 and 50, and the grand total is 100. Under independence, expected counts are 20,20,30,30. The likelihood-ratio statistic is 2[30 log(30/20) + 10 log(10/20) + 20 log(20/30) + 40 log(40/30)] = 17.2609, using natural logarithms. The ordinary independence comparison has one degree of freedom. Its chi-square approximation gives p approximately 0.0000326. Pearson's chi-square for the same table is 16.6667, showing that the two statistics are related but not identical. This original example assumes independent observations and the usual table model; its small p-value indicates evidence against independence, not the probability that independence is true.

Illustrative case study

In a fictional case, Elm Services compares outcomes across two service channels. Its analyst finds a large likelihood-ratio statistic in a customer table. Before drawing conclusions, the team checks that customers appear only once and investigates differences in case mix between channels. It reports association and relevant outcome proportions without claiming that channel choice caused the difference. When a later dataset includes repeated customers, the team revises the model rather than reusing the independent-count calibration unchanged.

Common misconceptions and mistakes

Likelihood-ratio chi-square and Pearson chi-square are not numerically interchangeable, although they can have similar large-sample behavior. A zero observed count does not require taking log zero directly; its contribution is defined as zero when the expected count is positive. Structural zeros and zero expected counts need separate treatment. Non-nested models do not automatically permit the usual degrees-of-freedom difference rule. Boundary parameters, mixtures, sparse tables and weak identifiability can invalidate ordinary chi-square calibration. Nonsignificance does not prove the null model or establish equivalence.

Business use

Businesses should state the compared models, statistic, degrees of freedom, calibration and data assumptions. Report effect estimates and uncertainty in meaningful units alongside the test result. Consider multiplicity when testing many features or segments, and assess whether observations are independent or clustered. For model development, use predictive validation as well as inferential comparisons. When standard asymptotic conditions are doubtful, seek a justified alternative such as an appropriate exact procedure or model-based bootstrap rather than forcing the statistic into a familiar reference distribution.

Check your understanding

Why do 17.2609 and 16.6667 both arise from the same table?

Answer: The first uses the likelihood-ratio formula involving logarithms, while the second uses Pearson's squared observed-minus-expected discrepancies. They compare the same null in this setting but are different statistics with only asymptotic similarities.

Related terms

Deviance (statistics)
Wilks' theorem
Degrees of freedom
Contingency table
Likelihood-ratio test
G-squared
deviance
Pearson chi-square
nested models
parametric bootstrap

Learn next

Study the relationship between multinomial likelihoods and contingency-table deviance. Then examine regularity conditions, boundary cases and the difference between a hypothesis test, model-selection criterion and held-out predictive comparison.

Sources

NIST/SEMATECH e-Handbook, Likelihood ratio tests: https://www.itl.nist.gov/div898/handbook/apr/section2/apr233.htm . Penn State STAT 504, Two-Way Tables: https://online.stat.psu.edu/stat504/Lesson03 . The table calculation and fictional service case above are original illustrations; the chi-square calibration is explicitly approximate.

Previous: Likelihood function Next: Likelihood tempering