Can deviance be negative?

A properly defined fitted-versus-saturated likelihood-ratio deviance is nonnegative; other quantities called deviance may use different additive constants.

Does a smaller deviance always mean a better model?

It means better comparable in-sample likelihood fit, but complexity, assumptions and generalization still matter.

Is the deviance difference always chi-squared?

No. The usual reference requires nested models and suitable regularity and large-sample conditions.

AI Glossary / Statistics and probability / Intermediate

Deviance (statistics)

Definition

Deviance is a likelihood-based measure of how much a fitted statistical model falls short of a saturated model that can fit the observed responses exactly. In the common generalized-linear-model convention, it is twice the difference between the saturated and fitted log likelihoods. Smaller deviance indicates better fit on the same data under a comparable likelihood convention. It is not automatically a probability, a percentage error or evidence of useful prediction.

What it means

A saturated model provides a flexible reference rather than a desirable deployment model. It can fit observed data perfectly by using enough parameters, but may offer little useful generalization. Deviance measures the cost of the restrictions imposed by a simpler model. Different distributions produce different deviance formulas, reflecting different likelihoods. Software may report scaled or unscaled versions or omit data-only constants in related likelihood summaries, so the definition must be checked before comparing values from different sources.

How it works

Specify a probability model for the response and fit its parameters. Evaluate the fitted log likelihood and the corresponding saturated-model log likelihood using the same observations and likelihood convention. Their difference, multiplied by two, gives the likelihood-ratio deviance in the standard Poisson or binomial setting. For nested models, subtracting the fuller model's deviance from the reduced model's deviance gives a likelihood-ratio comparison when assumptions are satisfied. Appropriate large-sample conditions can support a chi-squared reference with degrees of freedom equal to the parameter-count difference. This calibration is not universal, especially with sparse data, boundary parameters or model misspecification.

Three practical examples

Count modeling: assess restrictions in a Poisson regression for the number of service requests or observed events.
Binary-response modeling: compare nested logistic models using the improvement in likelihood-based fit.
Diagnostics: examine deviance residuals to identify observations or patterns that deserve closer investigation under the chosen response model.

Think of it this way

Think of comparing a practical map with a perfectly detailed record of one journey. The record can reproduce that journey exactly, while the map uses a simpler structure. Deviance measures the model-based loss of fit from using the simpler representation; it does not tell you whether the perfectly detailed record would guide the next journey well.

Worked example

Consider two hypothetical independent Poisson counts, 2 and 4, with equal exposure. An intercept-only model estimates a common mean of 3. The saturated model instead uses means 2 and 4. Poisson deviance is 2 sum[y log(y/mu) - y + mu], using natural logarithms and treating the zero-count logarithmic term as zero. Here D = 2[2 log(2/3) - 2 + 3 + 4 log(4/3) - 4 + 3] = 0.679596 approximately. The saturated fit has deviance zero. The number 0.679596 is not a 67.96% error rate. With only two small counts, this example illustrates the calculation rather than asserting a reliable chi-squared goodness-of-fit approximation. Repeating the comparison on different data would change the reference likelihood as well.

Illustrative case study

Fictional case: Willow Operations models daily service counts using a Poisson regression. Analysts compare a baseline model with one including a prespecified workload factor and examine the change in deviance. They also inspect residual patterns and whether variability exceeds the Poisson mean-variance relationship. A lower deviance does not settle the decision: the team evaluates later-period predictions and checks whether the extra term is operationally meaningful. This invented case illustrates model assessment, not evidence that a particular business process follows a Poisson distribution.

Common misconceptions and mistakes

Deviance is not the same as the residual sum of squares for every response family, although they are related in normal linear models under appropriate scaling. Lower training deviance can result from adding flexibility and overfitting. Residual deviance does not always have a chi-squared distribution with residual degrees of freedom; individual binary observations and sparse counts require particular care. Overdispersion can invalidate standard Poisson or binomial inference. Comparing deviances across different outcome definitions, observation sets or likelihood constants can be meaningless. A zero deviance for a saturated model does not establish predictive perfection.

Business use

Businesses can use deviance as part of a transparent statistical-model comparison when the likelihood matches the measurement process. It helps distinguish improvements in fit from subjective preference for a more complicated model. Report the model family, observations, parameter restrictions and any dispersion treatment. For AI-related forecasting or classification, combine likelihood-based assessment with held-out performance and errors that matter in practice. A useful model must generalize and support the decision, not merely reduce a training statistic by exploiting additional parameters.

Check your understanding

Why is the saturated deviance zero in the worked example?

Answer: The fitted means equal the observed counts, so each likelihood contribution matches its saturated reference. That removes the likelihood gap on those observations without demonstrating anything about performance on new counts.

Related terms

Likelihood
Saturated model
Generalized linear model
Poisson regression
Logistic regression
likelihood-ratio test
residual deviance
null deviance
overdispersion
deviance residual

Learn next

calculate the Poisson deviance for a small dataset, including a zero count. Compare null and residual deviance for nested models, then study when likelihood-ratio approximations are valid. Examine dispersion and residual diagnostics before interpreting a deviance reduction as sufficient evidence for a model change.

Likelihood

Sources

Penn State STAT 504, Poisson Regression: https://online.stat.psu.edu/stat504/Lesson09 R documentation, Model Deviance: https://stat.ethz.ch/R-manual/R-devel/library/stats/html/deviance.html statsmodels, Poisson deviance documentation: https://www.statsmodels.org/dev/generated/statsmodels.genmod.families.family.Poisson.deviance.html

Previous: Deviance information criterion Next: Deviation scores