AI Glossary / Mathematics and optimization / Intermediate
Stochastic process
Definition
A stochastic process is an indexed collection of random variables defined on a common probability space. It models uncertain quantities together, including how their values depend on one another across time, space or another index set.
What it means
One random variable might represent demand tomorrow. A stochastic process can represent demand on every future day and the relationships among those daily quantities. Those relationships matter because several consecutive high-demand days create different planning problems from isolated high days.
The index often represents time, but it need not. A random field can describe measurements at spatial locations, while random variables indexed by node pairs can describe edges in a random graph.
A realized sequence of observations is one sample path of the process. The process itself is the probabilistic model of possible paths, not merely the one line drawn on a chart. Observing a path does not reveal the whole underlying distribution.
How it works
To specify a process, identify the index set, the possible values at each index and their joint probability behavior. Discrete time might use integer steps; continuous time can include every instant in an interval. The values can themselves be discrete or continuous.
Marginal distributions describe individual indexed variables. Joint distributions describe combinations. Identical daily distributions do not imply independent days: a process may repeatedly produce similar values because of strong dependence.
Different process families impose different structures. A Markov model makes the present state sufficient for its specified next-state distribution. A Gaussian process requires every finite collection of values to have a joint Gaussian distribution. Neither property is part of the general definition.
Model checking should examine dependence and changes over time, not only average levels. A process can have independent increments while its levels remain dependent, as a random-walk example shows.
Three practical examples
- A sensor model represents measurement noise over time, including possible correlation between successive readings.
- A queueing simulation models uncertain arrivals and their accumulated effect on waiting workloads.
- A spatial prediction model represents uncertain temperatures across locations and uses relationships between nearby measurements.
Think of it this way
Think of a collection of possible films rather than one photograph. Each film is a possible path, and a probability model describes which films are more likely and how neighboring frames relate.
The analogy emphasizes joint behavior. A stochastic process can use a spatial index or other structure, so it need not literally unfold as a time sequence.
Worked example
Independent steps, dependent positions
Start a simple random walk at X0 = 0. At each step, independently choose an increment of +1 or −1 with equal probability, and set X(t+1) = Xt plus that increment.
After two steps, X2 can be −2, 0 or 2, with probabilities 1/4, 1/2 and 1/4. Its expectation is 0, while its variance is 4 × 1/4 + 0 × 1/2 + 4 × 1/4 = 2.
The first position X1 is +1 or −1. Although the increments are independent, X1 and X2 are not: X2 includes the first increment. Their covariance is 1, because X2 = X1 + the independent second increment and Var(X1) = 1.
Conditioning on X1 = 1 leaves X2 equal to 0 or 2 with equal probability. Its conditional expectation is therefore 1, rather than the unconditional expectation 0. Knowledge of the current position changes the forecast.
One possible path is 0, 1, 0, −1, 0. It illustrates the model but does not establish that a real business time series follows a random walk.
Illustrative case study
A fictional maintenance team models sensor readings as independent noise around a constant level. Its simulated data match the observed histogram, yet the simulation rarely produces the long stretches of elevated readings seen in practice.
The team examines dependence across time and considers a model that preserves persistent deviations. It compares simulated run lengths and autocorrelation with held-out observations, rather than checking the histogram alone.
It also investigates whether changes in operating conditions should be modeled explicitly. The lesson is that matching individual-value distributions is insufficient when decisions depend on sequences. The team treats the stochastic process as an assumption-driven model and tests the aspects that matter for its monitoring task.
Common misconceptions and mistakes
- A stochastic process is any plotted sequence. A sequence can be one realization; the process includes a probabilistic description.
- Random variables in a process must be independent. Dependence is often the main reason to model them jointly.
- Every process is stationary. Distributions and dependence patterns can change across the index.
- Independent increments imply independent levels. A cumulative process shares earlier increments across later levels.
- A good fit proves the proposed data-generating mechanism. Different models can explain similar observations.
Business use
Stochastic processes support forecasting, simulation and uncertainty analysis when relationships among observations matter. Select a model according to the decision: average workload, extreme bursts and time-to-event questions may require different aspects of joint behavior.
Document the time scale, state definition and assumptions about dependence. Check whether training and evaluation periods represent comparable conditions. Use simulated paths to inspect consequences, but validate them against relevant observed patterns. A realistic-looking chart is weaker evidence than tests tied to the actual operational question.
People also ask
Can a stochastic process have a deterministic component?
Yes. A trend or known input can be combined with random variation.
Is every time series a Markov process?
No. The Markov property is an additional modeling assumption, and it depends on the chosen state representation.
Can two processes have the same marginals but different behavior?
Yes. Different dependence structures can produce very different paths despite matching individual-time distributions.
Check your understanding
Why are X1 and X2 dependent in the random-walk example even though the two increments are independent?
Answer: X2 contains the first increment, which is also X1. The two positions therefore share a random contribution.
Related terms
- Random variable
- Markov Chain
- Gaussian process
- Autocorrelation
- sample path
- joint distribution
Learn next
Read conditional probability and stationarity. Compare independent daily coin outcomes with their cumulative sum. Describe which quantities are independent, which share past information and why that difference changes forecasting.
More in Mathematics and optimization
- Absolute error loss - Absolute error loss measures a numerical prediction's error as the absolute difference between its predicted and reference values: L(y,y-hat)=|y-y-hat|.
- Active-set method - An active-set method is an optimization approach that works with a changing subset of constraints believed to be tight at the solution.
- Adjoint state method - The adjoint state method computes how a chosen objective changes when the parameters of a mathematical model change.
- Σ-algebra
- Algebraic connectivity - Algebraic connectivity is the second-smallest eigenvalue of a graph's Laplacian matrix.
- Algorithm IMED - Algorithm IMED, short for Indexed Minimum Empirical Divergence, is a method for choosing actions in a stochastic multi-armed bandit problem.
Sources
Aalto Dictionary of Machine Learning, stochastic process, Markov chain and covariance function: https://aaltodictionaryofml.github.io/ADictML_Main.pdf. Definitions and indexing scope. The random-walk calculation, possible path and fictional sensor-model review are original teaching examples.
Previous: Stochastic policy Next: Stochastic program