Must a nonstationary kernel have changing variance?

No. Some change their dependence structure while retaining constant marginal variance.

Is a linear kernel stationary?

Generally no; its value depends on the inputs themselves and changes under translation.

Is every function of two inputs a valid covariance?

No. Symmetry alone is insufficient; positive semidefiniteness is required.

AI Glossary / Probabilistic machine learning / Advanced

Nonstationary kernel

Definition

A nonstationary kernel is a covariance function whose value can change when both inputs are translated by the same amount. Its behavior depends on where the inputs are, not solely on their separation vector. In Gaussian process models, this can represent location-dependent variation or dependence patterns.

What it means

A stationary covariance applies the same relationship at every location for a given displacement. A nonstationary covariance permits that relationship to differ across the input domain. For example, a function may vary more strongly in one region than another, or its local correlation length may change.
These are modeling assumptions, not automatic discoveries about the world. A nonstationary kernel still needs to be mathematically valid and supported by relevant data. It should also be distinguished from a changing mean or changing observation noise, which describe different parts of a statistical model.

How it works

A covariance kernel must produce a symmetric positive semidefinite matrix for every finite collection of inputs. Arbitrarily replacing a constant length scale with a location-dependent number can violate that requirement. Valid constructions are therefore essential.
One simple construction starts with a valid stationary kernel k0 and a deterministic amplitude function a(x), then defines k(x,x') = a(x) k0(x,x') a(x'). If K0 is a covariance matrix and D contains the amplitudes on its diagonal, the new matrix is D K0 D. For every vector v, its quadratic form equals (Dv) transpose K0 (Dv), which is nonnegative.
Other constructions use transformed inputs or specially derived varying-scale covariance formulas. Stationarity depends on the chosen coordinates: a kernel stationary in a transformed space may be nonstationary in the original inputs. Model fitting and checking must account for the additional flexibility.

Three practical examples

Sensor modeling: an analyst represents a latent signal whose typical amplitude increases with a known operating condition. A separate likelihood describes sensor measurement noise.
Spatial research: a team investigates whether nearby locations remain similarly correlated across different geographic regions. It compares a suitable nonstationary model with simpler stationary alternatives.
Model auditing: researchers translate an input pair while keeping its displacement fixed. A change in covariance demonstrates nonstationarity for that kernel, even if the two pairs have equal correlations after variance normalization.

Think of it this way

Imagine using a map whose ruler changes with location. The same separation can correspond to different modeled relationships in different regions. However, the model must specify that changing ruler consistently; arbitrary local adjustments can produce a collection of pairwise values that cannot belong to any valid covariance matrix.

Worked example

Consider an original one-dimensional example on nonnegative inputs. Begin with the unit-variance squared exponential kernel k0(x,x') = exp[-(x-x') squared / 2]. Choose amplitude a(x) = 1 + x. The new kernel is k(x,x') = (1+x)(1+x') exp[-(x-x') squared / 2].
The pair x = 0 and x' = 1 has separation 1 and covariance 2 exp(-1/2), approximately 1.2131. Translate both inputs by 2. The pair 2 and 3 still has separation 1, but covariance is now 12 exp(-1/2), approximately 7.2784. The changed covariance proves nonstationarity.
Marginal variances at inputs 0, 1, 2 and 3 are respectively 1, 4, 9 and 16. For the first pair, dividing covariance by the product of standard deviations gives 2 exp(-1/2) / (1 times 2) = exp(-1/2). For the second pair, it gives 12 exp(-1/2) / (3 times 4), the same value, approximately 0.6065. Thus this construction changes amplitude while preserving the base correlation pattern.
For inputs 0 and 1, the covariance matrix has diagonal entries 1 and 4 and off-diagonal entries 2 exp(-1/2). Its determinant is 4 minus 4 exp(-1), approximately 2.5285, which is positive. Together with the positive leading diagonal entry, this confirms positive definiteness for this particular matrix. The general D K0 D argument establishes validity for arbitrary finite input collections. These calculations describe a stipulated prior kernel, not a fitted dataset or an observation-noise model.

Illustrative case study

Fictional company Pine Monitoring models a machine's latent vibration response across operating speeds. Its analysts consider a kernel with increasing signal amplitude and compare it with a constant-amplitude alternative. They retain a separate measurement-noise term so that uncertainty about the underlying signal is not automatically treated as unreliable instrumentation.
The team evaluates predictions on held-out operating ranges and inspects whether the extra flexibility is supported by observations. It records the coordinate scaling because changing the speed units changes the interpretation of the amplitude function. This fictional scenario illustrates model assessment; it reports no measured improvement or deployed system.

Common misconceptions and mistakes

Nonstationary does not necessarily mean time-varying: the input can be spatial location or another feature. It also does not necessarily mean that correlation length changes. The worked example is nonstationary solely through its marginal variance.
Stationarity and isotropy are different. A stationary kernel can depend on the direction of the separation vector rather than only its magnitude. Conversely, adding a changing mean to a model does not change the stationarity of its covariance kernel. Conditioning a stationary prior on observations can produce location-dependent posterior covariance; that does not retrospectively make the original prior kernel nonstationary.

Business use

Nonstationary kernels can support models when one global covariance pattern is too restrictive. The benefit must be balanced against estimation difficulty and the risk of fitting noise or sparse-region artifacts. Teams should compare meaningful predictive outcomes, inspect uncertainty and document which changing behavior the kernel represents. More flexible covariance is not automatically more accurate forecasting.

Check your understanding

Why are the example's translated covariances different?

Answer: The amplitude factors change even though the separation does not.

Why are its correlations equal?

Answer: Positive amplitude factors cancel when covariance is divided by the marginal standard deviations.

What protects the construction's validity?

Answer: Multiplication on both sides by the diagonal amplitude matrix preserves positive semidefiniteness.

Related terms

Gaussian process
Covariance function
Stationary kernel
Isotropy
Heteroscedasticity
Input warping
Positive semidefinite matrix

Learn next

Compare changing signal amplitude, changing correlation length and changing measurement noise in separate models. Check each proposed kernel's validity before interpreting its fitted behavior.

Sources

Rasmussen and Williams, Gaussian Processes for Machine Learning, 2006, Chapter 4, covariance definitions and constructions: https://gaussianprocess.org/gpml/chapters/RW4.pdf . The amplitude-modulated calculation and fictional company case are original illustrations. Chapter 4 stationarity and covariance-construction discussion independently consulted; amplitude validity and example verified.

Previous: Nonstationarity Next: Nontrivial functional dependency