Are inducing points just a random subset of observations?

They can be chosen that way, but inducing locations and values have their own modeling role and need not coincide with observed examples.

Does DTC remove observation noise?

No. The zero conditional covariance concerns latent training function values given inducing values.

Can DTC and subset-of-regressors predictions have the same mean?

Yes, under the corresponding construction, while their predictive variances differ because DTC retains the test residual term.

AI Glossary / Probabilistic machine learning / Intermediate

Deterministic training conditional approximation

Definition

The deterministic training conditional approximation, or DTC, is a sparse Gaussian process approximation that makes training function values deterministic conditional on a smaller set of inducing values. It retains the original Gaussian process conditional uncertainty for test values. This distinction reduces computation while preserving a test-point variance contribution omitted by a simpler low-rank approximation.

What it means

A full Gaussian process relates every training value to every other one through a covariance matrix. An inducing-variable approximation summarizes those relationships through a smaller collection of latent function values.
DTC assumes that once those inducing values are known, the training function values follow their conditional means without additional residual variation. That assumption applies to the latent training function, not necessarily to noisy observations. At a test location, the model keeps the conditional residual uncertainty. Training and test points therefore receive different approximation treatments.

How it works

Let u denote inducing values, f the training function values, and K the covariance blocks from the original kernel. Define Q_ff = K_fu K_uu^−1 K_uf. DTC replaces the training conditional covariance with zero while keeping its conditional mean K_fu K_uu^−1 u.
For a test value f_star, DTC retains the exact prior conditional p(f_star | u). With Gaussian observation noise, inference uses Q_ff plus the observation-noise covariance for the training data. Test prediction includes the retained residual variance beyond what inducing values explain.
The approximation is useful when the inducing set is much smaller than the training set. Its quality depends on kernel structure, inducing locations, and data coverage. The asymmetry between training and test roles is an important modeling feature, not merely an implementation detail.

Three practical examples

Regression scaling: investigate whether a smaller inducing representation provides acceptable predictions when full covariance calculations are too expensive.
Approximation comparison: compare DTC with a subset-of-regressors approximation to understand why equal predictive means can accompany different predictive variances.
Teaching and diagnostics: inspect which conditional covariance terms an inducing approximation removes, rather than describing every sparse Gaussian process method as the same low-rank replacement.

Think of it this way

Think of summarizing a training record through a small set of reference measurements. DTC treats the training signal as fully specified by those references, while allowing a new location to contain extra unresolved variation. The analogy has limits: inducing values are latent mathematical quantities, not necessarily observed reference measurements, and the approximation can distort the relationships it summarizes.

Worked example

Consider an original scalar calculation with one inducing value, one training observation, and one test location. Let K_uu = 1, the training-to-inducing covariance be 0.8, and the test-to-inducing covariance be 0.5. Let the test prior variance be 1 and observation-noise variance be 0.36.
Then Q_ff = 0.8² = 0.64, Q_star,f = 0.5 × 0.8 = 0.4, and Q_star,star = 0.5² = 0.25. For observed training value y = 1 and zero prior mean, the DTC predictive mean is 0.4 × 1 / (0.64 + 0.36) = 0.4.
Its latent predictive variance is 1 − 0.4² / 1 = 0.84. The subset-of-regressors version uses 0.25 in place of the test prior variance 1, producing variance 0.25 − 0.16 = 0.09 with the same mean.
The difference, 0.75, is the test conditional residual variance that DTC retains. To predict a new noisy observation with noise variance 0.36, add that variance to 0.84, giving 1.20. These assumed covariance quantities illustrate the formulas; they do not establish the accuracy of either approximation for an actual dataset.

Illustrative case study

Fictional case: Hazel Modelling compares sparse Gaussian process methods for a large sensor dataset. Two implementations return similar means but very different uncertainty away from inducing locations. The analysts trace the difference to whether test conditional residual variance is retained.
They compare the approximations with a full Gaussian process on a manageable subset and evaluate held-out predictive scores. The team also changes inducing locations to assess sensitivity. It does not assume that DTC uncertainty is automatically calibrated simply because it is larger than the simpler approximation's uncertainty. This scenario illustrates a review process only.

Common misconceptions and mistakes

“Deterministic” does not mean the final prediction has no uncertainty. Inducing values remain uncertain after observing data, and the test conditional retains a residual variance.
Nor does the term require noise-free training observations. DTC modifies the latent training conditional while the observation model may still include noise.
DTC is not interchangeable with FITC, which retains a diagonal training conditional correction, or with variational inducing-point methods that use a different approximation principle.
Because training and test points are treated differently, DTC should not casually be described as one globally consistent replacement Gaussian process prior applying identically to every input role.

Business use

For businesses, this term matters when reviewing an implementation's speed-versus-accuracy trade-off and understanding the meaning of its uncertainty. The practical question is whether the approximation is adequate for the intended data and decisions.
Compare predictions, probabilistic scores, memory use, and runtime with appropriate alternatives. Document the inducing design and distinguish latent-function uncertainty from observation uncertainty. Where uncertainty drives decisions, check coverage on representative held-out data; the name of an approximation does not provide a calibration guarantee.

Check your understanding

A DTC latent prediction has variance 0.7, and independent observation noise has variance 0.2. What variance applies to a new noisy observation?

Answer: 0.7 + 0.2 = 0.9. Reporting 0.7 would describe the latent function uncertainty rather than the full observation uncertainty.

Related terms

Gaussian process
inducing variable
sparse approximation
conditional covariance
subset of regressors
FITC
observation noise
predictive variance

Learn next

Review Gaussian conditioning and the decomposition of covariance into an inducing-explained component and a residual. Then compare the training and test conditionals for subset of regressors, DTC, and FITC. Check both means and variances in a small example before assessing a large implementation.

Sources

Quiñonero-Candela and Rasmussen, A Unifying View of Sparse Approximate Gaussian Process Regression, section on DTC: https://www.jmlr.org/papers/volume6/quinonero-candela05a/quinonero-candela05a.pdf Independent source review: Original sparse-GP paper section 5, equations 18–20 and training/test distinction were read.

Previous: Deterministic Sampling Next: Detrended correspondence analysis