AI Glossary / Mathematics and optimization / Advanced
Finite-dimensional distribution
Definition
A finite-dimensional distribution of a stochastic process is the joint probability distribution of its values at a finite selection of indices. For a time-indexed process X(t), selecting times t1,...,tn produces the random vector (X(t1),...,X(tn)). Its distribution describes which combinations of values can occur at those times and their probabilities. The indices can also represent spatial locations or other labels, rather than time alone.
What it means
A process may contain infinitely many random variables, but a particular question often concerns only a few of them. For example, a monitoring question might ask about equipment status at three scheduled checks. The relevant finite-dimensional distribution describes those three statuses jointly.
Knowing the probability of each status separately is not enough. The joint distribution also captures dependence: a machine that tends to remain idle can produce a different pattern of consecutive readings from independent readings with the same overall idle probability.
How it works
Choose a finite list of indices and specify the joint law of the corresponding values. For discrete variables, this can be a table of joint probabilities. For continuous variables, it may be described through a joint distribution function or, when one exists, a density.
Distributions for different index selections must be compatible. Removing a coordinate from a larger joint distribution must produce the stated distribution for the remaining coordinates. Reordering coordinates must reorder the distribution consistently.
Under suitable state-space assumptions, consistent families of these distributions define a process law on the product sigma-algebra. This statement concerns the whole family, not just a few checked marginals. Additional care is needed for sample-path properties such as continuity; specifying finite-dimensional laws does not automatically establish every desired path property.
Three practical examples
1. Equipment monitoring: A team models the joint status at opening, midday, and closing. The probability that all three readings are idle can differ from the product of the three individual idle probabilities because readings may be dependent.
2. Spatial modeling: A model predicts values at four selected locations. Their joint distribution can answer questions about simultaneous high values. Four separate prediction distributions alone do not describe how those values move together.
3. Synthetic data validation: A developer compares pairs and triples of generated observations with the target model. Matching individual histograms is useful, but it does not establish that the generator reproduces the intended dependence or the full process law.
Think of it this way
Think of choosing several frames from a randomly generated film. A finite-dimensional distribution describes the possible combinations appearing in those selected frames. Separate descriptions of each frame lose information about how the frames relate. One chosen collection also does not replace every possible collection from the film.
Worked example
Consider an original two-state Markov chain with states 0 and 1. Its initial probabilities are P(X0 = 0) = 2/3 and P(X0 = 1) = 1/3. From state 0, the next state is 0 with probability 3/4 and 1 with probability 1/4. From state 1, either next state has probability 1/2.
The initial distribution is stationary: the next probability of state 0 is (2/3)(3/4) + (1/3)(1/2) = 2/3.
The three-dimensional distribution of (X0,X1,X2) follows by multiplying the initial probability by the two transition probabilities. For example, P(0,0,0) = (2/3)(3/4)(3/4) = 3/8, while P(0,1,0) = (2/3)(1/4)(1/2) = 1/12.
In the order 000, 001, 010, 011, 100, 101, 110, 111, the eight probabilities are 3/8, 1/8, 1/12, 1/12, 1/8, 1/24, 1/12, and 1/12. They sum to one.
To obtain the distribution of (X0,X2), sum over X1. Thus P(X0 = 0, X2 = 0) = 3/8 + 1/12 = 11/24. The four endpoint-pair probabilities, ordered 00, 01, 10, 11, are 11/24, 5/24, 5/24, and 1/8. Summing the first two gives P(X0 = 0) = 2/3, as required.
Independent observations with the same one-time probabilities would instead give P(0,0,0) = (2/3)^3 = 8/27. That differs from 3/8, demonstrating why matching one-dimensional distributions does not establish matching three-dimensional distributions. The tables and marginal sums were checked with exact fractions.
Illustrative case study
In a fictional equipment-data project, a synthetic generator produces independent status values with probability 2/3 of state 0. Its one-time histograms match the specified monitoring model, so an initial review declares the generator accurate.
A reviewer checks the three-check event above. The intended Markov model assigns probability 3/8 to three zeros, while the independent generator assigns 8/27. The team replaces independent sampling with the specified transition mechanism and checks the pair and triple tables again.
The revised generator addresses this particular dependence error. The team does not claim that agreement on these few tables proves every aspect of a more complex real-world process. It records which finite-dimensional distributions were tested and the assumptions used to generate them.
Common misconceptions and mistakes
Finite-dimensional does not mean the process itself has only finitely many observations. A one-dimensional marginal is only one member of the family. Pairwise distributions do not generally determine all higher-order distributions without additional structure.
Do not present inconsistent tables as though they describe one process. Do not assume a joint density must exist for every finite-dimensional distribution. Also distinguish equality in distribution from two processes producing identical realized paths: the former is a statement about probability laws.
Business use
Finite-dimensional distributions provide a precise way to express and test multi-time or multi-location behavior in stochastic models. They help teams identify what a simulation or prediction service actually claims to reproduce. A validation report should name the indices, events, and dependence structure examined. This is particularly useful when individual-variable summaries look convincing but decisions depend on combinations, runs, or simultaneous outcomes.
People also ask
Can a finite-dimensional distribution involve nonconsecutive times?
Yes. The endpoint pair in the worked example does exactly that.
Is it always a multivariate normal distribution?
No. The example is discrete and non-Gaussian.
Do matching marginals imply independence?
No. Independence is an additional property of a joint distribution.
Check your understanding
What is the worked model's probability of 000?
Answer: 3/8.
How is P(X0 = 0, X2 = 0) obtained?
Answer: Add the probabilities of 000 and 010, giving 11/24.
Why does the independent generator fail the triple check?
Answer: It preserves individual probabilities but changes dependence.
Related terms
- Stochastic process
- Marginal distribution
- Consistency
- Markov Chain
- joint distribution
- Kolmogorov extension theorem
Learn next
Construct the worked model's distribution at four consecutive times, then marginalize it to recover the three-time table. Next examine a continuous-valued example and distinguish specifying selected finite-dimensional laws from establishing properties of entire sample paths.
More in Mathematics and optimization
- Absolute error loss - Absolute error loss measures a numerical prediction's error as the absolute difference between its predicted and reference values: L(y,y-hat)=|y-y-hat|.
- Active-set method - An active-set method is an optimization approach that works with a changing subset of constraints believed to be tight at the solution.
- Adjoint state method - The adjoint state method computes how a chosen objective changes when the parameters of a mathematical model change.
- Σ-algebra
- Algebraic connectivity - Algebraic connectivity is the second-smallest eigenvalue of a graph's Laplacian matrix.
- Algorithm IMED - Algorithm IMED, short for Indexed Minimum Empirical Divergence, is a method for choosing actions in a stochastic multi-armed bandit problem.
Sources
Cosma Shalizi, Building Infinite Processes from Finite-Dimensional Distributions, definitions, consistency, and extension caveats: https://www.stat.cmu.edu/~cshalizi/754/2006/notes/lecture-02.pdf
The Markov tables, exact marginalization, and fictional generator case are original illustrations.
Previous: Finger search tree Next: Finite horizon