Does stationary mean unchanging observations?

No. It concerns the covariance's response to jointly shifting inputs.

Can different directions have different length scales?

Yes. That is compatible with stationarity, though generally not isotropy.

Is a stationary prior's posterior necessarily stationary?

No. Conditioning on an uneven set of observations typically introduces location-dependent covariance.

AI Glossary / Probabilistic machine learning / Intermediate

Stationary kernel

Definition

A stationary kernel is a covariance function whose value depends on the displacement between two inputs, rather than their absolute locations. In Gaussian process modeling, it expresses a prior assumption that the same separation has the same covariance wherever that separation occurs in the input space.

What it means

Imagine comparing temperatures measured two hours apart. A stationary time kernel assigns the same prior covariance to the pair at 08:00 and 10:00 as to the pair at 14:00 and 16:00. The observations themselves need not have identical values.

Stationarity describes the covariance rule, not necessarily the entire model. A changing mean function can coexist with a stationary covariance kernel. A stationary Gaussian process requires an appropriate constant mean as well. After conditioning on observations, predictive uncertainty generally varies by location because some locations have more nearby evidence than others.

How it works

Write the kernel as k(x, x') = c(x minus x'). Moving both inputs by the same vector leaves their difference unchanged, so the covariance remains unchanged. A valid covariance kernel must also produce positive semidefinite covariance matrices; translation invariance alone is insufficient.

A common example is the squared exponential kernel: covariance equals a positive variance parameter multiplied by exp of minus one half the squared, scaled separation. Its length scales control how quickly covariance changes with separation. Different length scales for different coordinates permit directional differences while retaining stationarity.

Isotropy is a stronger restriction: covariance depends only on ordinary distance, not direction. A stationary kernel can be anisotropic. Kernel parameters are estimated or assigned priors, and the resulting model is checked against observations. Stationarity is a modeling assumption whose usefulness can change across operating regions.

Three practical examples

1. Production monitoring: model residual sensor fluctuations after accounting for scheduled operating changes. Equal time lags receive the same prior covariance, while an explicit mean model captures predictable shifts.

2. Spatial interpolation: estimate a field over a region where the same displacement plausibly has comparable dependence. Directional length scales can represent longer correlation along a production line than across it.

3. Experiment planning: use a stationary covariance in a Gaussian process surrogate over normalized machine settings. Check whether one length scale per setting adequately describes both stable and rapidly changing operating zones.

Think of it this way

Think of a ruler that compares the separation of two marks. Sliding both marks together does not change the ruler's reading. A stationary kernel follows that same principle for covariance. It need not treat horizontal and vertical separation equally, and it does not claim that all measured heights are identical.

Worked example

Consider the original one-dimensional kernel k(x, x') = 9 exp[minus (x minus x') squared divided by 8]. Its prior variance is 9, and its length scale is 2 input units.

For inputs 1 and 3, the separation is 2. Their covariance is 9 exp(minus 0.5), approximately 5.4588. Shift both inputs by 10, obtaining 11 and 13. The separation remains 2, so the covariance is again 5.4588. The corresponding prior correlation is 5.4588 divided by 9, approximately 0.6065.

Now compare inputs 1 and 5. Their separation is 4, giving covariance 9 exp(minus 2), approximately 1.2180, and correlation 0.1353. These calculations demonstrate dependence on separation.

For contrast, the valid linear kernel k(x, x') = x times x' gives covariance 3 for the first pair but 143 for the shifted pair. It is not stationary. Finally, adding a mean function m(x) = x to the stationary covariance produces a Gaussian process with a changing mean; the whole process is then not stationary despite its stationary kernel.

Illustrative case study

Fictional case: Cedar Instruments models deviations from a planned temperature schedule. Its analysts first subtract a fitted schedule effect and apply a stationary kernel to the residuals. Their review asks whether residual behavior at the start and end of a heating cycle has comparable correlation at matching time lags.

If late-cycle residuals fluctuate much faster, the team compares separate regimes or a nonstationary covariance rather than merely collecting more observations under the same assumption. It keeps the same held-out cycles for comparison and examines uncertainty coverage as well as average prediction error. This fictional study specifies a decision process and makes no claim of measured improvement.

Common misconceptions and mistakes

A stationary kernel does not imply constant predictions or constant posterior uncertainty. It also does not imply isotropy, periodicity, or independence at every large distance; these depend on the selected function. Conversely, a smooth-looking dataset does not establish stationarity. Confusing input normalization with evidence for stationarity can hide genuine changes in dependence across regions.

Business use

The practical value is a compact, reusable dependence model. It can reduce the number of assumptions a team must estimate separately at different locations. The cost is potential mismatch where volatility, smoothness, or correlation length changes substantially. Teams should record the input coordinates, scaling, mean model, kernel family, and validation region so that a convenient default does not silently become a universal claim.

Check your understanding

Answer: 1.

If both inputs move by the same amount, what must happen to stationary covariance?

Answer: It must remain unchanged.

Answer: 2.

Does a stationary covariance plus a sloping mean define a stationary Gaussian process?

Answer: No; the mean also matters.

Answer: 3.

Why can a translation-invariant formula still fail as a kernel?

Answer: Its covariance matrices may fail positive semidefiniteness.

Related terms

Gaussian process
Covariance function
Nonstationary kernel
Spectral mixture kernel
isotropic kernel
length scale
positive semidefinite matrix

Learn next

Compare stationary and linear kernels on shifted input pairs. Then study how the mean function and conditioning affect a Gaussian process, followed by anisotropic length scales and nonstationary covariance design.

Gaussian process

Sources

Rasmussen and Williams, Gaussian Processes for Machine Learning, Chapter 4, defines stationarity and isotropy and explains valid covariance functions: https://gaussianprocess.org/gpml/chapters/RW4.pdf

The numerical comparisons and Cedar Instruments scenario are original teaching examples; the scenario is fictional. Independent review consulted Chapter 4.1 for translation invariance, isotropy and positive semidefiniteness.

Previous: Stationary distribution Next: Stationary process