Why is variance in squared units?

It averages squared discrepancies.

Is residual standard deviation the same quantity?

It is the square root and returns to the response's original units.

Can residual variance be zero?

A model can fit all training observations exactly, but that does not prove noise-free prediction or valid uncertainty estimates.

AI Glossary / Statistics and probability / Intermediate

Residual variance

Definition

Residual variance describes variation remaining around a model's fitted values. In ordinary least-squares linear regression with a correctly specified mean and constant error variance, the usual estimator is the residual sum of squares divided by the residual degrees of freedom. Its square root is the residual standard deviation, also called the standard error of estimate in some introductory texts.

What it means

A fitted model explains a pattern, but observations generally do not lie exactly on that pattern. Residual variance summarizes the size of the remaining squared discrepancies in the model's response scale.
The estimate is model-dependent. Remaining variation can reflect random noise, omitted structure, measurement error, or misspecification. Calling it residual does not establish that it is irreducible or that no useful predictor could explain it.

How it works

For each observation, subtract the fitted response from the observed response. These differences are residuals. They are calculated from the fitted model and are not identical to the unobserved population errors appearing in a theoretical model.
Square the residuals and add them to obtain the residual sum of squares, often abbreviated RSS or SSE. Squaring prevents positive and negative residuals from canceling and gives greater weight to larger discrepancies.
In a full-rank ordinary least-squares model with n observations and p fitted coefficients, including the intercept when present, the residual degrees of freedom are n minus p. When n is greater than p, divide RSS by n minus p to obtain the usual residual variance estimate. With zero residual degrees of freedom, this estimator is undefined.
Under a correct linear mean model with zero-mean uncorrelated errors of common variance, this degrees-of-freedom adjustment gives an unbiased estimate of that common error variance. Normality is not necessary for this unbiasedness statement, although exact small-sample inferential procedures may require additional assumptions.
The fitted residuals themselves do not all have the same variance even under that model. Their variances depend on leverage, and fitting creates dependence among them. The pooled estimate should therefore not be interpreted as an assertion that every individual residual behaves identically.
Distinguish this estimator from training mean squared error, which divides RSS by n, and from the ordinary sample variance of the residual list, which may use n minus one. These denominators answer different questions and are not interchangeable.
Inspect residual plots before trusting a single pooled summary. Curvature, changing spread, autocorrelation, or influential points may signal that the model or uncertainty calculation needs revision. Weighted, generalized, penalized, and multilevel models require corresponding definitions and degrees-of-freedom treatment.
Use validation data to assess prediction performance. Adding terms can reduce training RSS, but it does not guarantee lower prediction error on new observations. The residual variance estimate and a held-out error metric should be labeled separately.

Three practical examples

AI forecasting: a team summarizes remaining variability after fitting a simple baseline regression.
Quality analysis: engineers examine whether a process model leaves approximately constant scatter across operating conditions.
Experiment analysis: researchers use an error-variance estimate to support standard errors for model coefficients under stated assumptions.

Think of it this way

Think of a fitted line as a path through a cloud of points. Residual variance describes the squared vertical spread left around that path, with an adjustment for the coefficients estimated from the same points. It does not describe how far the line might miss a different cloud tomorrow.

Worked example

Take six observations with predictor values 0, 1, 2, 3, 4, and 5 and responses 6, 6, 8, 12, 13, and 15. The fitted ordinary least-squares line is predicted y = 5 + 2x.
Its fitted responses are 5, 7, 9, 11, 13, and 15. Subtracting them from the observations gives residuals 1, -1, -1, 1, 0, and 0. Their sum is zero, and their predictor-weighted sum is also zero, as expected for this fitted intercept-and-slope model.
RSS is 1 + 1 + 1 + 1 + 0 + 0 = 4. There are n = 6 observations and p = 2 coefficients, so the residual variance estimate is 4/(6 minus 2) = 1 squared response unit. The residual standard deviation is sqrt(1) = 1 response unit.
By contrast, training MSE is 4/6, approximately 0.6667, and training RMSE is approximately 0.8165. These smaller numbers use a different denominator; they do not contradict the residual variance estimate.

Illustrative case study

Fictional case: Ash Operations fits a baseline relationship between workload and processing time. Its analyst reports the residual standard deviation in minutes alongside the squared-unit variance estimate. A residual plot shows increasing spread at high workloads, so the team investigates a model for changing variance before relying on a single uncertainty estimate for all operating conditions.

Common misconceptions and mistakes

Do not confuse residual variance with variance explained by the predictors. A small residual variance also depends on the response units and scale.
Do not divide by the number of predictors without checking the residual degrees of freedom. The intercept counts as a fitted coefficient.
An apparently small training residual variance can coexist with poor generalization or leakage. It is not a complete measure of predictive quality.
Do not assume a fitted residual is an independently observed realization of the population error. Fitting imposes relationships among residuals.

Business use

Residual variance helps organizations quantify unexplained spread and assess whether a model's uncertainty description is plausible. Useful reporting identifies the model, response units, numerator and denominator, assumptions, diagnostic findings, and the distinction between fitting error and new-case prediction performance.

Check your understanding

Why are there four residual degrees of freedom in the example?

Answer: Six observations were used to estimate two coefficients.

Why does training RMSE differ from residual standard deviation?

Answer: The two calculations divide the same RSS by six and four, respectively.

Related terms

Residual
Mean Squared Error
Degrees of freedom
Leverage
residual sum of squares
residual standard deviation
heteroskedasticity
regression diagnostics

Learn next

Learn next how leverage affects individual residual variance, how residual plots reveal misspecification, and how prediction intervals combine estimated error variance with uncertainty about the fitted mean.

Sources

Sources: Howell, Statistical Methods for Psychology glossary, https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html ; NIST, Linear Least Squares Regression certification definitions, https://www.itl.nist.gov/div898/strd/lls/data/LINKS/c-NoInt2.shtml ; NIST/SEMATECH, Least Squares, https://www.itl.nist.gov/div898/handbook/pmd/section4/pmd431.htm . The primary sources establish terminology and least-squares conventions. The six-point calculation and case are original.

Previous: Residual stream Next: Resilience