Does exchangeable mean independent?

No. Independence may hold only after conditioning on the mixing variable.

Does the theorem require a beta prior?

No. The binary mixing distribution can be any probability distribution on [0,1].

Can it apply to a finite dataset?

A finite dataset can be modeled as part of an infinite exchangeable sequence, but that extendibility is an assumption, not a consequence of observing finitely many values.

AI Glossary / Statistics and probability / Advanced

De Finetti's theorem

Definition

De Finetti's theorem connects exchangeability with conditional independence. In its binary form, an infinite exchangeable sequence can be represented by first drawing an unknown probability P and then generating independent Bernoulli observations with that same probability, conditional on P. The observations need not be independent before conditioning. The theorem provides an important foundation for Bayesian probability models.

What it means

Exchangeability means that rearranging any finite collection of observations does not change its joint probability distribution. For binary outcomes, sequences with the same number of successes have equal probability, regardless of their order.

The theorem says that infinite exchangeability has a particular representation: a mixture over ordinary independent, identically distributed sequences. The mixture describes uncertainty about the common probability. It does not say that observations collected in real settings are automatically exchangeable.

The infinite qualification matters. A finite exchangeable collection may not admit this representation unless it can be extended consistently to an infinite exchangeable sequence. Broader versions cover other kinds of observations under appropriate mathematical conditions.

How it works

For binary observations x1 through xn, let s be their sum. Under the representation, the probability of this particular ordered sequence is the integral of p^s times (1−p)^(n−s) over a mixing distribution G on [0,1]. For a discrete mixture, replace the integral with a weighted sum.

Conditional on a selected p, the observations are independent Bernoulli trials. After averaging over uncertainty in p, they can be dependent: one success provides information about the common probability and therefore changes predictions for another observation.

The theorem guarantees a representation under its assumptions; it does not choose a convenient prior family or estimate its parameters for a business application. A beta prior is a useful modeling choice for binary observations, but it is not required by the theorem. Assessing the appropriateness of exchangeability remains a substantive task.

Three practical examples

A Bayesian quality model treats comparable pass/fail inspections as exchangeable conditional on an uncertain common pass probability.
A survey model uses an exchangeable binary representation for responses within a deliberately defined homogeneous group, while questioning whether different groups can be pooled.
An AI evaluation model treats comparable test outcomes as exchangeable within one fixed evaluation setting, but considers changes in task difficulty or model version separately.

Think of it this way

Think of selecting a coin with an unknown bias and then repeatedly tossing that same coin. Once its bias is specified, the tosses are independent. Before the bias is known, observing several heads changes your prediction for the next toss. The shared uncertainty links the observations even though the conditional mechanism uses independent tosses.

Worked example

Consider an original two-component model. Choose P = 0.2 or P = 0.8 with equal probability, once for the entire sequence. Then generate all binary observations independently conditional on that chosen P.

The marginal success probability is 0.5 × 0.2 + 0.5 × 0.8 = 0.5. The probability of two successes is 0.5 × 0.2² + 0.5 × 0.8² = 0.34. If the observations were marginally independent, it would instead be 0.5² = 0.25. Their covariance is therefore 0.34 − 0.25 = 0.09.

After observing one success, the posterior probability of P = 0.8 is (0.5 × 0.8)/0.5 = 0.8. The next success probability becomes 0.8 × 0.8 + 0.2 × 0.2 = 0.68, agreeing with 0.34/0.5.

The ordered sequences 110 and 101 each have probability 0.5 × 0.2² × 0.8 + 0.5 × 0.8² × 0.2 = 0.08. This illustrates exchangeability alongside marginal dependence. Redrawing P separately for every observation would define a different model and remove this shared source of dependence.

Illustrative case study

In this fictional case, Alder Evaluation assesses an AI classifier on a carefully defined task family. The analysts initially use an exchangeable binary model for correct and incorrect answers, expressing uncertainty about a shared accuracy parameter.

An audit finds that the test set mixes easy template questions and difficult reasoning problems. The team compares separate task groups and checks whether order, repeated source material, or changing model versions introduce additional structure. It revises the model where a single exchangeable group is implausible.

The case is invented to illustrate assumption checking. The theorem supplies a representation for a stated probability model; it does not certify that the evaluation design meets its assumptions.

Common misconceptions and mistakes

Exchangeability is weaker than unconditional independence, but stronger than merely having identical marginal distributions. Shuffling a spreadsheet does not remove dependence on time, difficulty, or collection conditions from its underlying distribution.

Finite symmetry alone is insufficient. Drawing two objects without replacement from a container holding one success and one failure gives an exchangeable pair, but the outcomes are negatively dependent. It cannot be represented as a mixture of independent Bernoulli pairs with one shared probability, whose covariance would be the nonnegative variance of that probability.

Do not interpret the mixing variable as necessarily a physically existing cause. The theorem concerns an equivalent probability representation. It also does not prove that a chosen prior or likelihood fits observed data.

Business use

The theorem helps teams explain why a shared uncertain parameter can be useful for repeated comparable outcomes. It clarifies the difference between independent observations given a parameter and predictions that remain dependent because that parameter is unknown.

For business use, define the population and grouping before pooling observations. Evaluate whether geography, product version, season, customer segment, or repeated subjects need explicit modeling. Document the prior and examine predictive behavior. A mathematical representation becomes useful operationally only when its assumptions and resulting uncertainty match the decision context.

Check your understanding

Check your understanding: Why is the two-success probability 0.34 rather than 0.25?

Answer: Both outcomes share the same uncertain P.

What changes after one success?

Answer: Beliefs about P and the predictive probability.

Why does order not matter in the example?

Answer: The joint probability depends on the count of successes.

Does rearranging records establish exchangeability?

Answer: No.

Related terms

Learn next

Compare a beta-Bernoulli model with sampling without replacement. Then investigate partial exchangeability and hierarchical models when observations form distinct groups.

Sources

Sources: Alexandre Bouchard-Côté, Exchangeability and de Finetti, https://www.stat.ubc.ca/~bouchard/courses/stat520-sp2021-22/T11-definetti.html. Barber, Candès, Ramdas and Tibshirani, De Finetti's Theorem and Related Results for Infinite Weighted Exchangeable Sequences, introduction and classical theorem discussion, https://www.stat.berkeley.edu/~ryantibs/papers/defin.pdf. The numerical mixture and fictional evaluation case are original.

Previous: De Casteljau's algorithm Next: De-identification