Is it just a larger Gaussian process?

No. It composes probabilistic mappings, which changes the model family and inference problem.

Must every layer have the same dimension?

No. Dimensions are modeling choices, and intermediate layers can provide different representations.

Are predictions exactly Bayesian?

Usually not computationally. Practical methods approximate the posterior or predictive integration, so their accuracy should be assessed rather than assumed.

AI Glossary / Probabilistic machine learning / Intermediate

Deep Gaussian process

Definition

A deep Gaussian process is a probabilistic model that composes multiple mappings governed by Gaussian processes. Outputs from one layer become inputs to another, creating intermediate representations whose uncertainty can be carried through the hierarchy. Despite its name, the complete composed function is not generally a Gaussian process.

What it means

A Gaussian process describes uncertainty about a function. A deep Gaussian process places such uncertainty over several connected transformations. Instead of deciding on one exact hidden representation and treating it as known, the model can consider different plausible intermediate values.
This distinction matters when several explanations fit the observations. A prediction should reflect uncertainty about both the hidden representation and the final mapping. In practice, computation usually requires approximations. Adding probabilistic layers does not automatically produce reliable uncertainty or better predictions; these remain properties to evaluate against data.

How it works

Specify the number of layers, their dimensions, covariance functions, and an observation model. Each layer maps values from the previous layer into a new representation. Some models start from observed inputs for prediction; others learn latent representations of the observations themselves.
Training seeks plausible functions and hidden values given the observations. Exact integration through the whole hierarchy is generally difficult. Variational methods approximate the relevant distributions and optimize a lower bound on the model evidence; sampling may help propagate uncertainty between layers.
For prediction, propagate uncertain intermediate values rather than only their means. Combine the resulting conditional output distributions. Check performance against a shallower Gaussian process and assess calibration, sensitivity to architecture, and the effect of the inference approximation.

Three practical examples

Sensor modeling: represent a nonlinear relationship between operating conditions and measurements while tracking uncertainty through learned intermediate features.
Scientific surrogate modeling: investigate whether a hierarchy captures a simulator's changing behavior better than a single smooth mapping, using held-out simulator runs for comparison.
Representation learning: explore probabilistic hidden descriptions of complex observations, checking whether the representations remain useful and stable across different fitting choices.

Think of it this way

Think of a forecast passing through several uncertain translations. The first translates raw observations into a hidden description; the next translates that description into a prediction. Keeping several plausible translations can preserve information lost by choosing one immediately. The analogy stops at the mechanism: these are mathematical distributions over mappings, not people making linguistic judgments or explaining causes.

Worked example

Consider an original, deliberately simplified uncertainty-propagation calculation. At a particular input, approximate the hidden layer using two equally weighted representative values. After passing them through the next layer, suppose the corresponding conditional output distributions have means 1 and 3, each with variance 0.25.
The mixture mean is (1 + 3) / 2 = 2. Average conditional variance is 0.25. Variance of the conditional means around 2 is ((1 − 2)² + (3 − 2)²) / 2 = 1.
The total predictive variance is therefore 0.25 + 1 = 1.25, with standard deviation approximately 1.118. Reporting only 0.25 would omit the uncertainty represented by the different hidden values.
These assumed conditional statistics illustrate the law of total variance; they are not a fitted deep Gaussian process or an exact two-point hidden posterior. Using only the mean hidden value can also change the predicted mean when the next mapping is nonlinear. The mixture need not be Gaussian, even if each conditional distribution is Gaussian.

Illustrative case study

Fictional case: Cedar Instruments investigates a deep Gaussian process for a laboratory sensor with different operating regimes. The team compares it with a single-layer model on runs excluded from fitting. It examines errors and whether uncertainty increases where observations are sparse.
One architecture gives attractive average error but intervals that are too narrow. Analysts inspect the approximation and the training data before using those intervals in decisions. They retain the simpler model as a reference and document the extra computation. This scenario illustrates a validation process, not a demonstrated commercial result.

Common misconceptions and mistakes

Do not assume that Gaussian layers make the composed output Gaussian. Nonlinear composition and integration can produce more complicated distributions.
A deterministic deep feature extractor followed by a Gaussian process is commonly called deep kernel learning; its treatment of hidden uncertainty differs from a deep Gaussian process.
Do not confuse the uncertainty of a latent function with the variability of future observations. Observation noise must be handled explicitly.
More layers can make fitting harder, and a variational approximation can misrepresent uncertainty. Neither the model's Bayesian formulation nor successful optimization proves that its intervals have the coverage needed for a particular decision.

Business use

Businesses may investigate these models when nonlinear structure and uncertainty both matter, especially where obtaining observations is costly. The practical question is whether the hierarchy improves decisions enough to justify its training, prediction, and maintenance costs.
Evaluate suitable predictive scores, interval behavior, robustness to changed operating conditions, and comparison with simpler alternatives. Record which sources of uncertainty the implementation represents. Use domain review to determine whether the data support the intended application; a flexible statistical model cannot supply missing evidence about a new operating regime.

Check your understanding

Two equally weighted hidden scenarios give output means 4 and 6, with conditional variance 2 in each. What is the total predictive variance?

Answer: The mean is 5. Between-scenario variance is ((4 − 5)² + (6 − 5)²) / 2 = 1. Adding average conditional variance gives 2 + 1 = 3. Keeping only 2 discards the represented hidden-scenario uncertainty.

Related terms

Gaussian process
Latent variable
Covariance function
Deep kernel learning
Calibration
variational inference
predictive distribution
law of total variance

Learn next

First review Gaussian process regression and the difference between a latent function and a noisy observation. Then study total variance and variational inference. Compare a deep Gaussian process with a deterministic feature network followed by a Gaussian process, paying particular attention to which hidden quantities are integrated out.

Sources

Damianou and Lawrence, Deep Gaussian Processes, original paper and model formulation: https://proceedings.mlr.press/v31/damianou13a.pdf Independent source review: Damianou–Lawrence introduction and hierarchy/variational formulation were read; observed-input and latent-input uses remain distinguished.

Previous: Deep Feed-Forward Network Next: Deep image prior