Is tolerance an outcome-performance score?

No; it describes relationships among predictors.

Should every low-tolerance predictor be deleted?

No. Its substantive role and the analysis objective matter.

Does tolerance detect every dependence?

No. The conventional diagnostic concerns linear relationships in the specified design matrix.

AI Glossary / Statistics and probability / Intermediate

Tolerance

Definition

Tolerance is a regression diagnostic that measures how much of one predictor's variation remains unexplained by the other predictors in the model. In ordinary linear regression with an intercept, it is calculated as 1 minus the R-squared from an auxiliary regression of that predictor on the others. Small tolerance indicates substantial linear overlap.

What it means

A model may include several variables that carry similar information. For example, advertising expenditure and the number of advertisements purchased may move closely together. The model then has limited information for separating their individual associations with the outcome.
Tolerance examines this overlap among predictors. It does not measure prediction accuracy, the percentage of outcome variation explained, or a permissible manufacturing error. This entry uses the collinearity meaning in the supplied glossary.

How it works

Choose one predictor, Xj, from the fitted model's design matrix. Regress it on the remaining predictors using the same relevant observations and an intercept. Calculate the resulting Rj-squared and subtract it from 1.
Repeat the calculation for other predictors. A tolerance of 0.16 means that 16% of Xj's centered variation remains after its linear relationship with the other predictors has been accounted for in that auxiliary regression.
The reciprocal is the variance inflation factor: VIFj = 1/tolerancej. A tolerance near 1 indicates little linear overlap with the other columns; a tolerance near zero indicates that Xj is almost reproducible from them.
For conventional ordinary least squares with homoskedastic independent errors, coefficient variance is sigma-squared divided by the product of Sxx,j and tolerancej, where Sxx,j is the predictor's centered sum of squares. Holding error variance and predictor spread fixed shows why small tolerance increases uncertainty about that coefficient.
Variance and standard error differ. A VIF of 6.25 corresponds to a standard-error multiplier of the square root of 6.25, or 2.5, under that comparison. It does not mean predictions become 6.25 times less accurate.
Inspect the actual design matrix, including indicator variables, interactions, transformations, and missing-data exclusions. Exact dependence can make coefficients unidentified. Software may drop a column or report an aliased estimate instead of a usable tolerance.
Treat screening thresholds as prompts for investigation. The consequences depend on the purpose, sample information, coefficient precision, and whether extrapolation is involved. A universal cutoff cannot decide which scientifically important variable should disappear.
Consider whether the task needs separate coefficients, stable predictions, or a causal adjustment set. Combining redundant measurements, improving the study design, or regularizing a predictive model can be reasonable responses. Removing a confounder merely to improve a diagnostic can introduce a more serious interpretive problem.

Three practical examples

A marketing analyst checks whether spend and impression counts can support separate coefficient interpretations.
A manufacturing team investigates overlapping temperature summaries before attributing output changes to one of them.
A demand-model developer examines near-duplicate regional indicators and confirms that the encoded design matrix has full rank.

Think of it this way

Think of several microphones recording nearly the same voice. More recordings do not necessarily make it easy to identify which microphone contributed a particular sound. Tolerance asks how much distinct linear information remains in one recording after the others are considered.

Worked example

Suppose an auxiliary regression of predictor X on the other predictors gives R-squared = 0.84. Its tolerance is 1 - 0.84 = 0.16, and its VIF is 1/0.16 = 6.25.
For an original numerical illustration, assume an ordinary least-squares model with independent, constant-variance errors, error variance sigma-squared = 4, and centered predictor sum of squares Sxx = 100. The coefficient variance is 4/(100 x 0.16) = 0.25, giving standard error 0.50 when the error variance is treated as known.
Under the comparison with tolerance 1 and the same error variance and predictor spread, coefficient variance would be 4/100 = 0.04 and standard error 0.20. The variance ratio is 6.25 and the standard-error ratio is 2.5. These calculations illustrate precision, not the sign, bias, significance, or causal meaning of the coefficient.

Illustrative case study

Fictional case: Cedar Retail includes monthly promotion spend and promotion impressions in a sales regression. Spend has low tolerance, and its estimated coefficient changes substantially when the observation window changes. The analysts inspect the purchasing relationship and discover that impressions were usually bought at a nearly fixed rate.
They retain a combined exposure measure for one prediction experiment, but keep a separate analysis documenting why the historical data cannot reliably isolate the two contributions. They do not report the diagnostic as proof that promotion has no value.

Common misconceptions and mistakes

Do not confuse predictor R-squared with the outcome model's R-squared. Low pairwise correlations do not rule out a predictor being explained by several others together. Centering variables does not generally remove substantive linear redundancy, although it can change some parameterizations involving interactions. Small tolerance does not automatically bias an ordinary least-squares estimate under its assumptions. It does make individual coefficients harder to estimate precisely.

Business use

Tolerance helps teams assess whether a coefficient-based explanation is supported by enough distinct information. Record the model specification, rows used, affected variables, and consequences for uncertainty. Compare practical prediction performance separately, and avoid presenting a cleaner diagnostic as evidence that the model is fair, causal, or externally valid.

Check your understanding

If R-squared from the auxiliary regression is 0.75, what are tolerance and VIF?

Answer: They are 0.25 and 4.

Under a fixed-error-variance, fixed-predictor-spread comparison, what is the standard-error multiplier?

Answer: It is 2, not 4.

Related terms

Learn next

Practice computing auxiliary regressions, then examine coefficient uncertainty and prediction performance separately. Study rank deficiency before interpreting models containing redundant indicators.

Sources

University of Vermont, Statistical Methods for Psychology glossary, Tolerance: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html SAS, Parameter Estimates for Linear Models, collinearity diagnostics: https://support.sas.com/documentation/cdl/en/sgug/59902/HTML/default/fit_sect26.htm Penn State, Variance Inflation Factors: https://online.stat.psu.edu/stat462/node/180/

Previous: Tokenizer Next: Tone curve