AI Glossary / Natural language and speech / Intermediate
Log odds ratio
Definition
A log odds ratio is the logarithm of the ratio between two odds. It compares the relative odds of the same event in two groups or conditions. In text analysis, it can help identify words associated more strongly with one collection of documents than with another.
What it means
Probability and odds are related but different. If an event has probability p, its odds are p ÷ (1 − p). A probability of 0.20 therefore gives odds of 0.20 ÷ 0.80 = 0.25, or one to four.
An odds ratio compares odds across two groups. Taking its logarithm produces a scale centered on zero: equal odds give a log odds ratio of zero. A positive value favors the group in the numerator, while a negative value favors the comparison group.
This entry uses natural logarithms. The measure describes an association under the chosen counting and comparison setup. It does not establish why the groups differ, and its numerical value is not a probability or a percentage-point difference.
How it works
Specify the event and the unit being counted. For document presence, the event might be that a document contains a particular word. For token frequency, it might be that a randomly selected token is that word. These definitions produce different denominators and should not be mixed.
Let pA and pB be the event probabilities in groups A and B. The log odds ratio is ln[pA/(1 − pA)] − ln[pB/(1 − pB)]. Reversing A and B changes its sign, while exponentiating the result recovers the odds ratio.
Observed counts can be unstable, especially for rare words. A zero count can make an unsmoothed estimate infinite or undefined. Text-analysis methods may introduce prior counts and account for uncertainty rather than rank terms by raw ratios alone.
The prior-adjusted, variance-standardized statistic discussed in corpus comparison is related to the raw log odds ratio but is not numerically identical to it. Reports should identify which quantity they actually display.
Three practical examples
- A support team compares how often documents mention delivery in two ticket collections, using document presence as its explicitly defined event.
- A language researcher compares word-token proportions across corpora while accounting for their different total token counts.
- A text-analysis dashboard flags strongly associated terms for inspection, showing counts so readers can distinguish broad patterns from rare occurrences.
Think of it this way
Think of comparing two balances between an event and its alternative. Odds describe each balance; the odds ratio compares the balances; the logarithm expresses that comparison around a neutral zero point.
The analogy does not imply that the observations are physically balanced or causally linked. It also does not make a large ratio reliable when it rests on very few observations. The counts and study design remain important.
Worked example
Comparing document mentions
In an invented sample, 20 of 100 group-A documents contain the word delayed, while 10 of 100 group-B documents contain it. Each document contributes once to the presence-or-absence count, regardless of repeated mentions.
The estimated probabilities are 0.20 and 0.10. Their odds are 20/80 = 0.25 and 10/90 ≈ 0.1111. The odds ratio is 0.25 ÷ 0.1111 = 2.25, and the natural-log odds ratio is ln(2.25) ≈ 0.8109.
For comparison, the probability ratio is 0.20 ÷ 0.10 = 2, and the probability difference is 0.10, or ten percentage points. These are three different summaries. Swapping the group order gives a log odds ratio of approximately −0.8109. None of these calculations establishes that belonging to group A caused the word to appear.
Illustrative case study
A fictional customer-experience team compares two sets of product reviews. Its first term list highlights a rare spelling that occurs twice in one set and never in the other. The raw ratio appears dramatic, but reviewers cannot treat that as a stable vocabulary difference.
The team retains the underlying counts, documents its tokenization choices and uses a stated prior-based method for the comparison. It inspects the original passages before assigning a business interpretation: a word may be quoted, negated or used in a different sense.
The lesson is that an association score can guide exploration without becoming a complete explanation. The fictional exercise does not claim a measured improvement from smoothing or identify a universal prior that suits every corpus.
Common misconceptions and mistakes
- An odds ratio is a probability ratio. They generally differ, as the worked example shows.
- Zero log odds ratio means the event never happens. It means equal odds in the comparison.
- A large score proves a reliable difference. Small counts and dependence among observations affect interpretation.
- Association identifies causation or sentiment. Neither follows from the numerical comparison alone.
Business use
Log odds comparisons can help teams explore recurring language across departments, products or time periods. They are most useful when paired with transparent event definitions, sample sizes and examples of the relevant text.
Before comparing periods, check whether collection methods, document lengths or source populations changed. A difference in word usage may reflect those changes rather than a new customer concern. If results will drive decisions, assess uncertainty and review the most influential terms in context. Preserve the distinction between a raw effect measure, a smoothed estimate and a standardized ranking statistic.
People also ask
What does a negative value mean?
The event has lower odds in the numerator group than in the comparison group, under the stated setup.
Can the log odds ratio exceed one?
Yes. It is not bounded to the probability interval; ln(3), for example, exceeds one.
What happens with zero counts?
A raw estimate may be infinite or undefined, requiring an explicit modeling or smoothing choice rather than an unexplained replacement.
Check your understanding
Group A has event probability 0.20 and group B 0.10. Is their odds ratio exactly two?
Answer: No. Two is the probability ratio. Their odds are 0.25 and about 0.1111, giving an odds ratio of 2.25.
Related terms
- Odds
- Probability
- Logit
- Odds ratio
- Smoothing
- Effect size
- corpus comparison
- Dirichlet prior
- sampling uncertainty
Learn next
Build a two-by-two count table and calculate probability difference, probability ratio and log odds ratio separately. Then change a small count and observe why rare-event comparisons need careful uncertainty handling.
More in Natural language and speech
- Linear chain CRF - A linear chain CRF is a conditional random field that assigns probabilities to sequences of labels using a chain of dependencies between neighboring label...
- Linearly separable - Two labeled groups are linearly separable when a single linear decision boundary can place every member of one group on one side and every member of the...
- Locative adverb - A locative adverb is an adverb that expresses where an action or situation occurs or, in a broad spatial use of the term, the direction associated with it.
- Log probabilities - Log probabilities are logarithms of probabilities.
- Logic form - A logic form, in the NLP convention covered here, is a structured representation of a sentence using predicates and shared arguments.
- Logistic function - The logistic function maps a real-valued input to an S-shaped output.
Sources
Jurafsky and Martin, Speech and Language Processing, Chapter 23: https://web.stanford.edu/~jurafsky/slp3/23.pdf. Corpus comparison using log odds ratios, informative priors and variance standardization. The document-presence calculation is an original example and is distinct from the chapter's token-frequency setup.
Previous: Log-odds Next: Log probabilities