AI Glossary / Probabilistic machine learning / Intermediate
Generalized linear mixed model
Definition
A generalized linear mixed model, or GLMM, combines a generalized linear response model with random effects that represent variation among groups or other shared sources of dependence. It can model outcomes such as binary events and counts while accounting for observations clustered within stores, machines, people or locations.
What it means
A basic regression often treats observations as independent after accounting for its predictors. But repeated observations from the same unit may remain related. A store's transactions, for example, can share an underlying propensity that differs from another store's propensity.
A GLMM adds random effects to the linear predictor alongside fixed-effect coefficients. A link function connects that predictor to the conditional response mean. The word mixed refers to fixed and random effects; it does not mean a mixture model that assigns each observation to an unknown component. Random effects are usually assigned a distribution, often Gaussian, rather than estimated as unrelated group coefficients.
How it works
Write the conditional mean as mu and the linear predictor as eta = X beta + Z b. Here beta contains fixed-effect coefficients and b contains random effects. The link function satisfies g(mu)=eta. For a binary response, a common choice is the logit link; for a Poisson count, a common choice is the log link.
A random intercept allows groups to have different baseline predictors. A random slope allows a predictor's association to vary by group. The random-effect distribution controls how much variation is expected and how information is shared across groups. Conditional independence given these effects can coexist with dependence after the effects are integrated out.
Likelihood-based fitting integrates over random effects, often using numerical approximations. Bayesian fitting instead specifies priors and approximates or samples the posterior. The choice affects computation and uncertainty assessment. Predictions for an already observed group and an entirely new group also require different treatment of its random effect.
Three practical examples
1. Store-level conversion: model whether a visit leads to a purchase, with store random intercepts and customer or visit predictors. Account for shared store conditions rather than treating every visit as wholly unrelated.
2. Equipment incidents: model counts of faults across machines, incorporating exposure duration and machine-level variation. A suitable exposure offset prevents longer observation periods from being mistaken for intrinsically higher fault rates.
3. Repeated user responses: model binary task completion across multiple sessions per user. A random intercept can represent persistent user differences, while a justified random slope can represent differing responses to a session feature.
Think of it this way
Think of branches following a common planning model while allowing each branch its own baseline adjustment. Branches with little information can borrow strength from the wider collection. The analogy describes partial pooling. A random effect is a statistical component, not proof that a branch's management caused its estimated difference or that branches were randomly assigned.
Worked example
Consider an illustrative logistic random-intercept model for purchase probability. Let the shared intercept be -1 and the coefficient for a binary feature x be 0.6. For store A, suppose the fitted random intercept is 0.4. Its conditional log odds are therefore -1 + 0.6x + 0.4.
When x=0, the predictor is -0.6 and the probability is 1/(1+exp(0.6)), approximately 0.3543. When x=1, the predictor is zero and the probability is 0.5. The conditional probability difference is about 0.1457, or 14.57 percentage points.
The associated conditional odds ratio is exp(0.6), approximately 1.8221. This is not an 82.21-percentage-point increase in probability. For another store with random intercept -0.4, the same two probabilities are approximately 0.1978 and 0.3100, a difference of 0.1122. The shared log-odds coefficient therefore need not imply the same probability difference across groups. These are stipulated predictions, not estimates from real store data, and they do not establish a causal effect.
Illustrative case study
Fictional case: Alder Retail analyzes visits from stores with very different sample sizes. Its analysts build a binary-response GLMM and initially report store-specific predictions alongside predictions for proposed new stores. A reviewer notices that the latter were calculated as if their random effects had already been learned.
The team distinguishes predictions conditional on known-store information from predictions integrating over uncertainty for a new store. It also evaluates performance by withholding whole stores. The fictional case illustrates matching validation and prediction to the deployment question, not a claimed improvement in sales forecasting.
Common misconceptions and mistakes
Random effects do not automatically correct every source of dependence. A random intercept may leave time dependence or other structure unexplained. Likewise, a Poisson random-intercept model does not guarantee an adequate description of excess zeros, outliers or every form of overdispersion.
Fixed-effect coefficients in a nonlinear GLMM generally describe conditional associations given the random effects. Population-averaged associations can differ. Setting a random effect to zero is not generally equivalent to averaging the inverse-link prediction over its distribution. Few groups, weakly supported random slopes and nearly zero variance estimates can also make interpretation and fitting difficult.
Business use
Choose a GLMM when the grouping structure is substantively meaningful and the response distribution suits the outcome. Record which effects are shared, which vary and how the group distribution is specified. Assess uncertainty in group comparisons instead of presenting noisy rankings as established facts.
Align validation with the intended use: forecasting another observation from an existing group differs from predicting for unseen groups. Inspect convergence diagnostics and sensitivity to random-effect structure or approximation settings. Communicate whether a reported probability is conditional or population averaged, and avoid interpreting observational coefficients as intervention effects without additional assumptions.
People also ask
Must the response itself be Gaussian?
No. Binary and count outcomes are common; a Gaussian random-effect distribution is separate from the response distribution.
What is a random slope?
It allows a predictor's coefficient to vary across groups according to the model's random-effect structure.
Is a GLMM the same as a generalized estimating equation model?
No. They handle clustered outcomes through different modeling approaches and can target differently interpreted associations.
Check your understanding
Why do both example stores have odds ratio 1.8221 but different probability changes?
Answer: They share the same conditional log-odds coefficient but start from different baseline predictors. The logistic transformation is nonlinear.
Related terms
- Generalized linear model
- Generalized estimating equation
- random intercept
- random slope
- partial pooling
- link function
- hierarchical model
- conditional prediction
- generalized estimating equations
Learn next
Practice converting log odds into probabilities, then compare a pooled logistic regression with a random-intercept model. Study how conditional predictions differ from predictions averaged over the random-effect distribution.
More in Probabilistic machine learning
- Generalized advantage estimation - Generalized advantage estimation, or GAE, is a reinforcement learning method for estimating how much better an action was than the value predicted for its...
- Generalized belief propagation - Generalized belief propagation, or GBP, is a family of probabilistic inference methods that exchange messages between regions containing groups of variables.
- Generalized expectation maximization - Generalized expectation maximization, usually called generalized EM or GEM, is a version of expectation maximization that improves the expected...
- Gibbs kernel - The Gibbs kernel is a Gaussian-process covariance function that allows the characteristic length scale to vary with the input location.
- Graphite graph generative model - Graphite is a graph generative modeling framework that combines latent node representations with an iterative graph-neural-network decoder.
- Hamiltonian Monte Carlo - Hamiltonian Monte Carlo, or HMC, is a Markov chain Monte Carlo method for sampling from continuous probability distributions using gradients of their log...
Work in finance? Get the free CFO AI Prompt Pack: 7 prompts for finance leaders.
Sources
lme4 authors, Fitting Generalized Linear Mixed-Effects Models: https://lme4.github.io/lme4/reference/glmer.html
Statsmodels authors, Generalized Linear Mixed Effects Models: https://www.statsmodels.org/stable/mixed_glm.html
Independent review consulted lme4’s conditional-mean and likelihood-integration details and Statsmodels’ Bayesian GLMM methods. Original logistic probabilities and odds ratio verified; retail case fictional.
Previous: Generalized linear classifier Next: Generalized linear model