Is a neural process identical to a Gaussian process?

No; its learned decoder and inference procedure can define different distributions and approximations.

Must it retrain for each context set?

The original inference design can use a forward pass without task-specific parameter updates.

Should a latent draw be shared across targets?

Yes, when sampling one function under the original global-latent model.

AI Glossary / Probabilistic machine learning / Advanced

Neural process

Definition

A neural process is a learned probabilistic model that uses a set of observed input-output pairs to predict outputs at other inputs. In the original latent neural process, a shared latent variable represents uncertainty about the underlying function. Neural networks learn how to encode observations and produce predictions across a distribution of related tasks.

What it means

Suppose several machines have related but different response curves. For a new machine, only a few measurements may be available. A neural process aims to use experience from earlier curves to interpret those measurements and predict the rest of the new curve.
Measurements form the context set; prediction locations are target inputs. The model learns how to respond to context. Changing the context can change its predictions through a forward computation while its learned network parameters remain fixed.
This is approximate inference. Its uncertainty depends on training and model design. Producing a predictive distribution does not automatically make its intervals calibrated, especially for tasks unlike those seen during training.

How it works

An encoder maps each context input-output pair into a representation. In the original architecture, a permutation-invariant aggregator, such as a mean, summarizes these representations. A network maps that summary to an approximate distribution over a global latent variable z. A decoder takes a target input and z and produces parameters of an output distribution.
One draw of z is shared across the target inputs when generating one possible function. Conditional observation noise can then be added separately at each point. This shared latent structure connects the predictions, even when outputs are conditionally independent given z.
Training uses many related tasks and varying context sets. A variational objective balances explaining target observations with agreement between latent distributions inferred from different amounts of task information. At prediction time, only available context observations can inform the latent distribution. Target outputs used during training must not leak into deployment evaluation. Deterministic conditional neural processes and later extensions differ from this original latent construction.

Three practical examples

A simulation project studies a family of response curves under related operating conditions. A neural process can be trained to predict a new curve from a small context set. Evaluation should hold out entire conditions rather than only individual points from familiar curves.
An image-completion experiment treats pixel positions as inputs and observed pixel values as context outputs. A shared latent draw can represent one possible completion. Plausible completions remain model-generated hypotheses, not recovered evidence of the actual missing pixels.
A sensor study considers related calibration functions for multiple devices. Experience across devices may help interpret limited readings from a new one. Changes in sensor technology or operating range can break the learned assumptions and require fresh validation.

Think of it this way

Imagine having learned many related drawing styles. A few marks from a new drawing suggest several possible complete sketches. Choosing one overall sketch idea keeps its different regions mutually compatible. Choosing an unrelated idea for every point would preserve some local possibilities but lose the coherence of a single drawing.

Worked example

Use a deliberately simple decoder g(x,z) = z*x to illustrate the shared latent principle. Suppose the context encoder has produced q(z given context) = Normal(2, 0.25), where 0.25 is the variance. These values are stipulated, not fitted or an exact Bayesian posterior.
At target inputs x1 = 1 and x2 = 2, the latent function values are F1 = z and F2 = 2z. Their means are 2 and 4. Their variances are 0.25 and 4 × 0.25 = 1, and their covariance is 2 × 0.25 = 0.5. They move together because they use the same z.
Now add independent observation noise with variance 0.1 to each output. The observed-output variances become 0.35 and 1.1, while covariance remains 0.5. This separates uncertainty about the shared slope from point-specific observation noise.
A latent draw z = 1.5 produces the coherent function values (1.5, 3); z = 2.5 produces (2.5, 5). If instead independent latent slopes were drawn for the two target points, the marginal means and variances could remain the same, but their covariance would become zero. That would be a different joint predictive model, not another way to sample the same function.
This linear example permits analytic calculation. A nonlinear decoder can produce non-Gaussian predictions. Monte Carlo sampling may then be used to examine joint predictions and uncertainty.

Illustrative case study

Fictional company Aspen Calibration proposes learning across simulated device response curves. Its team reserves entire devices for testing and evaluates predictions after revealing different numbers of context measurements. It compares with suitable simpler regression and Gaussian-process baselines under matched budgets.
The team checks prediction error, interval coverage, and whether joint curve samples remain plausible across operating ranges. It also tests shifted conditions that were absent from training. This invented scenario describes an evaluation design; it does not claim reduced calibration cost, improved accuracy, or reliable extrapolation in production.

Common misconceptions and mistakes

A neural process is not simply a neural network with an uncertainty label added to its output. The context-based task formulation and, in a latent NP, the shared random variable are central.
Permutation invariance means that rearranging the same context pairs should not change the result; it does not mean their input coordinates are ignored. Another mistake is confusing observation noise with variation between possible functions. Finally, mean aggregation and approximate inference can lose useful detail. More context does not guarantee that a trained model's uncertainty will contract correctly in every situation.

Business use

Consider neural processes when many related prediction tasks are available and rapid adaptation from limited context could help. Define what constitutes a new task, how its context is collected, and which changes require retraining. Compare predictive quality and uncertainty calibration on held-out tasks before using distributions to support operational decisions.

Check your understanding

What creates covariance 0.5 in the example?

Answer: The shared uncertain slope.

Does independent observation noise increase that covariance?

Answer: No.

Why is a new latent draw at each target different?

Answer: It removes the shared slope dependence while potentially leaving the marginals unchanged.

Related terms

Latent variable
Meta-learning
Gaussian process
Conditional neural process
context set
variational inference
predictive uncertainty

Learn next

Compare deterministic conditional neural processes with latent neural processes, then study attention-based context aggregation and task-level calibration tests.

Sources

Garnelo and colleagues, Neural Processes (2018), Section 2 independently consulted: https://arxiv.org/pdf/1807.01622 . The shared-slope calculation and Aspen Calibration scenario are original illustrations.

Previous: Neural operators Next: Neural radiance field