AI Glossary / Statistics and probability / Intermediate
Log-linear models
Definition
Log-linear models describe expected counts by making their logarithms linear combinations of parameters. In categorical-data analysis, they represent patterns of association in contingency tables, including tables involving several categorical variables. Main effects describe differences in marginal prevalence, while interaction terms describe associations that those main effects cannot explain. This entry follows the supplied glossary's contingency-table meaning, rather than using the term for every model that contains a logarithm.
What it means
Suppose a table cross-classifies people by service channel and whether an issue was resolved. Each cell contains a count, and the model assigns an expected count to every combination. A two-variable independence model can be written log(mij) = overall term + row effect i + column effect j.
Exponentiating turns the additive expression into multiplicative factors. Adding a row-by-column interaction allows association between the variables. Unlike a typical logistic regression presentation, a log-linear analysis can treat the classification variables symmetrically rather than selecting one as the response. The count is what the model fits.
How it works
Define the table, its sampling scheme, and any impossible combinations before fitting. Specify which main effects and interactions are included. In a hierarchical model, including an interaction also includes its component lower-order terms; for example, a three-way term brings the corresponding two-way terms and main effects.
Estimate expected counts using an appropriate likelihood formulation, often through Poisson regression or iterative proportional fitting. Parameter constraints are needed because unrestricted category effects otherwise provide redundant representations.
Compare fitted and observed counts, inspect residuals, and assess model fit. For three variables, a model containing AB and AC interactions but no BC or ABC interaction expresses conditional independence of B and C given A. This is not necessarily independence after A is ignored. Sparse cells and structural zeros require particular care in inference.
Three practical examples
- A customer-research team examines the joint relationship among subscription tier, region, and preferred support channel.
- An AI evaluation team studies error categories across language and task type, checking whether an association differs between languages.
- A retailer analyses purchase category, promotion exposure, and customer segment without immediately treating one categorical variable as the sole response.
Think of it this way
Think of a table as a grid whose cell heights represent expected activity. Main effects raise or lower whole rows and columns multiplicatively. Interactions reshape particular combinations. The modeling question is how much structure is needed to explain the pattern without simply giving every cell its own unrestricted setting.
Worked example
Consider an original two-by-two table. Channel A has 30 resolved and 10 unresolved cases; channel B has 20 resolved and 40 unresolved cases. Row totals are 40 and 60, column totals are 50 and 50, and the grand total is 100.
Under independence, each expected count equals its row total times its column total divided by 100. The fitted table is therefore 20, 20 in the first row and 30, 30 in the second. Its fitted odds ratio is (20 times 30)/(20 times 30) = 1, whereas the observed odds ratio is (30 times 40)/(10 times 20) = 6.
The Pearson statistic is the sum of squared observed-minus-expected differences divided by expected counts: 5 + 5 + 3.3333 + 3.3333 = 16.6667. The likelihood-ratio statistic is approximately 17.2609. Both compare the independence fit with the observed table, with one residual degree of freedom under the usual sampling assumptions. Adding the full two-way interaction saturates this small table and reproduces all four counts. That perfect fit is not independent evidence of predictive usefulness or a causal channel effect.
Illustrative case study
Fictional case: Beacon Learning categorizes support requests by product tier, region, and resolution status. A two-variable summary suggests that one tier performs poorly. The analyst retains region in a three-way table and compares scientifically motivated interaction structures.
The team discovers that the overall summary hides differing regional compositions. It reports fitted counts and interpretable conditional comparisons, while documenting sparse combinations that make estimates unstable. Managers investigate process differences before changing service policy. The model organizes associations in the collected data; it does not establish that changing a customer's tier would cause a different resolution outcome.
Common misconceptions and mistakes
Do not take logarithms of observed counts and run ordinary least squares as a substitute for a count likelihood. Zero observed counts can be legitimate sampling outcomes, while structural zeros represent impossible combinations and alter the problem.
A main effect is not automatically a causal influence. Omitting a three-way interaction does not mean every pair is independent. A saturated model fits the available table exactly but leaves no residual information for assessing its fit there. Finally, marginal and conditional associations can differ, so always identify which variables are being held fixed.
Business use
These models help businesses understand categorical behavior across several dimensions without relying on many disconnected two-way tables. They can reveal whether a simple association structure is sufficient or whether important interactions need attention.
Define categories consistently and avoid unnecessarily detailed subdivisions that create mostly empty cells. Keep the sampling design and repeated observations in view: a table of counts does not automatically imply independent observations. Present expected counts, uncertainty, and practical comparisons so decision-makers do not have to interpret a long list of coding-dependent coefficients.
People also ask
Are log-linear models only for three or more variables?
No; two-way tables provide useful basic examples, and higher-dimensional tables are a major application.
Are they the same as logistic regression?
They are related, but their usual formulation and interpretation differ.
Does “linear” mean counts change additively?
The linear predictor is on the logarithmic scale; effects on expected counts are multiplicative.
Check your understanding
1. What does the independence model predict for channel A's resolved cell?
Answer: 40 times 50 divided by 100 = 20. 2.
What odds ratio does independence imply?
Answer: One, when the relevant cell counts are positive. 3.
Does a saturated fit prove causation?
Answer: No; it only reproduces the observed table.
Related terms
- Contingency table
- Poisson regression
- Conditional independence
- Odds ratio
- Saturated model
- interaction
- iterative proportional fitting
- structural zero
Learn next
Begin with independence in a two-way table, then study three-way conditional independence and homogeneous association. Practice reading model notation and distinguishing sampling zeros from impossible category combinations before fitting larger tables.
More in Statistics and probability
- Location test - A location test is a statistical hypothesis test about where a distribution is centered.
- Location testing for Gaussian scale mixture distributions - Location testing for Gaussian scale mixture distributions concerns testing a common center when independent observations have errors that can be represented...
- Log likelihood - Log likelihood is the logarithm of a likelihood function, usually using the natural logarithm.
- Logistic regression - Logistic regression is a statistical and machine-learning method for modeling the probability of a categorical outcome.
- Logrank test - The logrank test compares time-to-event distributions between groups while allowing right-censored observations.
- Long-range dependence - Long-range dependence is persistent statistical dependence across widely separated observations in a time series or another indexed process.
Sources
David Howell, Statistical Methods for Psychology glossary. https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html — original glossary scope for log-linear models. R stats, “Fitting Log-Linear Models.” https://stat.ethz.ch/R-manual/R-devel/library/stats/html/loglin.html — primary implementation documentation on iterative proportional fitting, hierarchical margins, conditional independence, and structural zeros. The worked table and fictional case are original.
Previous: Log likelihood ratio Next: Log Loss