Is delta always Y-error variance divided by X-error variance?

No. That is this entry's convention; some software uses its reciprocal.

Does delta = 1 mean X and Y have equal observed variance?

No. It means equal measurement-error variances in the specified units.

Does high correlation prove agreement?

No. Strong association can coexist with substantial systematic or individual differences.

AI Glossary / Statistics and probability / Advanced

Deming regression

Definition

Deming regression estimates a straight-line relationship when both measured variables contain error. It uses information about the relative measurement-error variances to balance discrepancies in the two directions. The method is often used to compare measurement systems. Unlike ordinary regression of Y on X, it does not treat the observed X values as error-free measurements of the underlying explanatory quantity.

What it means

Suppose two instruments measure related underlying quantities on the same items. Both readings fluctuate because of measurement error. Fitting only vertical deviations can misrepresent the underlying relationship when error in X matters.

Deming regression introduces latent, error-free values along a line and measures how observed points depart from those values in both coordinates. The relative penalty depends on how much error each measurement method is expected to have.

The required variance ratio concerns measurement errors, not the overall observed variation of X and Y across items. Observed variation also includes genuine differences between items. Replicate measurements or external precision information can help identify the relevant error variances.

How it works

A basic model writes observed x as an underlying value plus an X measurement error, and observed y as an intercept plus slope times that underlying value plus a Y measurement error. The standard simple form assumes zero-mean errors, independence, and constant error variances; normality underlies common likelihood and inference treatments.

Define delta here as Var(error in Y)/Var(error in X). Some software uses the reciprocal, so the convention must be checked. The fit minimizes appropriately scaled squared discrepancies in both coordinates. With equal error variances in the stated units, it becomes orthogonal regression, minimizing squared perpendicular distances.

For centered sums Sxx, Syy, and Sxy, and nonzero Sxy, the slope is [Syy−delta Sxx + sqrt((Syy−delta Sxx)²+4 delta Sxy²)]/(2 Sxy). The intercept is mean(y) minus slope times mean(x). Weighted extensions address unequal precision, but require a suitable variance model.

Three practical examples

A manufacturing team compares two gauges that both have measurable repeatability error.
A laboratory investigates systematic differences between two measurement procedures while using independent precision information.
An AI-assisted measurement project compares numerical estimates from an automated imaging system with a reference procedure that also has measurement uncertainty.

Think of it this way

Think of drawing a line through points whose horizontal and vertical positions are both slightly uncertain. Ordinary vertical-distance fitting treats the horizontal positions as fixed. Deming regression allows both coordinates to move toward the underlying line, with movement penalized according to each direction's measurement precision.

Worked example

Consider an original three-pair teaching dataset: (1,1), (2,2), and (3,4). Assume independent measurement errors with equal constant variances in the displayed units, so delta = 1. These assumptions are supplied for demonstration, not estimated from the three pairs.

The means are mean(x) = 2 and mean(y) = 7/3. The centered sums are Sxx = 2, Syy = 14/3, and Sxy = 3. Substituting into the equal-variance formula gives slope = (4+sqrt(97))/9, approximately 1.53876. The intercept is 7/3 − 2×1.53876, approximately −0.74419.

Ordinary least squares of Y on X instead gives slope Sxy/Sxx = 3/2 = 1.5 and intercept 7/3−3 = −2/3. The answers differ because they optimize different discrepancy measures under different assumptions about X error.

An independent check uses the leading eigenvector of the centered two-variable scatter matrix [[2,3],[3,14/3]]. Its Y-to-X component ratio is also approximately 1.53876, as expected for orthogonal regression. This verifies the equal-error-variance calculation. Three invented pairs do not establish instrument agreement or provide a convincing empirical estimate of measurement precision.

Illustrative case study

In this fictional case, Granite Instruments evaluates an AI vision system for estimating component dimensions. A contact gauge supplies comparison readings, but repeated measurements show that the gauge also varies.

The team collects paired measurements across the operating range and separate replicate readings to assess precision. It states the error-variance ratio convention, checks for changing variance and outliers, and examines whether departures from a straight line are important. It also evaluates whether differences meet the application's practical tolerance.

The case is invented to illustrate method comparison. A fitted slope near one and intercept near zero do not alone establish that individual measurements agree closely enough for use.

Common misconceptions and mistakes

Do not substitute the ratio of the observed Y and X sample variances for the measurement-error variance ratio without justification. Between-item variation is not instrument error.

Equal error variances depend on units. Rescaling one variable changes its error variance and therefore requires a corresponding change in the ratio. Leaving the ratio fixed after changing units can change the implied model incorrectly.

Deming regression is not automatically robust to outliers, nonlinear relationships, correlated errors, or unequal precision. It also does not turn an uncertain reference method into ground truth. Finally, fitting an underlying measurement relationship is a different objective from optimizing prediction of a noisy Y reading given an observed noisy X reading; the appropriate method depends on the question.

Business use

Businesses should specify whether they need calibration, prediction, or agreement assessment before choosing a regression method. For Deming regression, record the measurement units, error model, ratio source, and treatment of replicate observations.

Assess sensitivity to plausible variance ratios and inspect whether error sizes change over the measurement range. Report uncertainty in the fitted relationship and compare discrepancies with predefined operational tolerances. A statistically detectable departure may be operationally minor, while a nonsignificant result can reflect inadequate precision. Decisions about substituting one measurement system for another need evidence about individual differences as well as the fitted average relationship.

Check your understanding

Check your understanding: Why does the example use orthogonal regression?

Answer: Equal error variances were assumed.

What independent calculation checked the slope?

Answer: The leading scatter-matrix eigenvector.

Why not estimate delta from total sample variances?

Answer: They include genuine item variation.

What must change after rescaling X?

Answer: Its measurement-error variance and the corresponding ratio convention.

Related terms

Measurement error
Calibration
Repeatability
Ordinary Least Squares
Errors-in-variables model
orthogonal regression
method comparison
variance ratio

Learn next

Compare Deming fits across plausible error ratios. Then study replicate-based precision estimates, weighted Deming regression, and agreement analysis.

Sources

Sources: Bendix Carstensen, MethComp Deming documentation, https://search.r-project.org/CRAN/refmans/MethComp/html/Deming.html. NCSS Statistical Software, Deming Regression, https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/NCSS/Deming_Regression.pdf. Their variance-ratio conventions differ; this entry explicitly uses Y-error variance divided by X-error variance. The numerical fit and fictional instrument case are original.

Previous: Delta rule Next: Demographic parity