Can a log likelihood ratio be negative?

Yes, for a stated numerator and denominator it is negative when the numerator model gives the data lower likelihood. A particular fitted test statistic may use a nonnegative convention.

Is it the same as log loss?

No. Log loss evaluates predictions through negative log probabilities; a log likelihood ratio compares two likelihoods for the same data.

Does it identify why two corpora differ?

No. It can provide evidence of a frequency pattern under the chosen model, but causes require additional investigation.

AI Glossary / Causal inference and experimentation / Advanced

Log likelihood ratio

Definition

A log likelihood ratio compares how well two statistical models or hypotheses account for the same observed data by taking the logarithm of their likelihood ratio. Equivalently, it subtracts one log likelihood from the other. The sign depends on which model is placed in the numerator. In language analysis, related likelihood-ratio statistics help compare word-frequency patterns or test association. A likelihood ratio is evidence under specified models, not a probability that a hypothesis is true.

What it means

Likelihood evaluates observed data as a function of a model or its parameters. If the same observations are much more likely under one proposed explanation than another, their ratio expresses that relative fit. Taking a logarithm makes multiplicative evidence easier to handle as an additive quantity.

The comparison is meaningful only when the models refer to the same observations and their likelihoods are defined consistently. A more flexible model can fit data better simply because it has more freedom. Testing, model selection, and prediction therefore require attention to assumptions and complexity, rather than automatically treating any positive log-likelihood advantage as a decisive finding.

How it works

Specify the hypotheses and the observation model. In a generalized likelihood-ratio test, fit the permitted parameters under each hypothesis, often comparing a restricted null model with a more flexible alternative. Calculate the two maximized log likelihoods.

One common statistic is twice the alternative-minus-null log-likelihood difference. Another notation defines a null-over-alternative likelihood ratio and takes minus twice its logarithm; these conventions can represent the same statistic. State the orientation and scaling.

Under suitable regularity conditions, a reference distribution can support a significance calculation. Those conditions, sampling assumptions, and degrees of freedom must be checked. A raw likelihood comparison alone does not supply a universal p-value, effect size, or practical decision threshold.

Three practical examples

1. Word association: a corpus analyst compares a model in which two word events are independent with one allowing their occurrence rates to differ by context.

2. Vocabulary comparison: researchers examine whether a term's observed frequency is better explained by one shared rate across collections or by collection-specific rates.

3. Statistical model testing: an analyst compares a restricted model with a nested alternative, keeping the dataset and likelihood definition fixed while checking the test's assumptions.

Think of it this way

Think of comparing two explanations by asking how unsurprising the same evidence would be under each. The log ratio records their relative fit on a convenient scale. The analogy is limited because statistical likelihood requires explicit probability models. An attractive explanation, a persuasive story, or a larger ratio does not automatically establish causation or provide the prior information needed for posterior probabilities.

Worked example

For a simple illustration, observe eight successes in ten independent Bernoulli trials. Compare a null success probability of 0.5 with an alternative probability of 0.8. Using natural logarithms, the alternative-minus-null log likelihood ratio is 8 ln(0.8/0.5) + 2 ln(0.2/0.5).

This equals approximately 1.927. Exponentiating gives a likelihood ratio of about 6.87: the observed count has that relative likelihood under these two probabilities. The binomial combinatorial factor cancels because it is identical in numerator and denominator.

Twice the log ratio is approximately 3.855. This arithmetic does not mean the alternative has a 6.87-to-one posterior probability, nor does it establish a significance claim without a specified test and assumptions. Reversing the ratio changes the log ratio's sign.

Illustrative case study

Fictional case: Pine Corpus compares vocabulary in two collections of support messages. A term receives a large likelihood-ratio statistic, and the team initially describes it as the most important emerging issue. Inspection shows that the score reflects a frequency difference, while business importance depends on what the messages actually describe.

The analyst reports the observed rates and counts alongside the statistic, checks duplicate messages and collection differences, and accounts for the many terms examined. Reviewers inspect examples before deciding whether the pattern deserves action. The case separates statistical evidence of a distributional difference from the meaning and priority of that difference.

Common misconceptions and mistakes

Do not confuse the likelihood of observed data under a hypothesis with the probability of that hypothesis given the data. They are different quantities.

Do not compare numbers without checking whether the author used natural or another logarithm, reversed the ratio, or multiplied by two. Identical words can conceal different conventions.

Avoid treating the statistic as an effect size or applying an asymptotic reference distribution automatically to any dataset. Dependence, sparse counts, parameter boundaries, and repeated testing can affect interpretation. A technically large score may accompany a practically small difference in a large sample.

Business use

Log likelihood ratios can support evidence-based analysis of text patterns and statistical models. They help formalize comparisons that would otherwise rely on raw counts or intuition, especially when the underlying question is clearly specified.

For business reporting, include the competing hypotheses, observation unit, sample sizes, rates, and direction of the difference. Explain what the statistic can and cannot establish. When a vocabulary screen examines thousands of terms, the screening design and follow-up review matter alongside the formula. Decisions should connect the statistical finding with operational evidence instead of equating surprise with importance.

Check your understanding

Model A has log likelihood −120 and model B has −125 on the same data. What is ln(LA/LB)?

Answer: −120 − (−125) = 5. The positive result favors A on likelihood for these observations, without by itself settling model complexity or practical usefulness.

Related terms

Learn next

Write down two simple probability models for the same observations and calculate their log likelihoods. Subtract in both directions and explain the sign. Then distinguish the questions answered by the ratio, an effect-size estimate, and a significance test before applying the method to text counts.

Sources

Primary source: Ted Dunning, Accurate Methods for the Statistics of Surprise and Coincidence, https://aclanthology.org/J93-1003.pdf . The likelihood-ratio framework and text-analysis scope were consulted. This entry explicitly separates the unscaled log ratio from common test-statistic conventions. The Bernoulli calculation and fictional corpus study are original illustrations; no empirical significance or causal claim is made.

Previous: Log likelihood Next: Log-linear models