Does every context have the same distribution?

No. Related distributions can differ.

Must the weights vary?

No. A single-weights DDP holds weights common and varies atom processes.

Is the worked example an exact DDP realization?

No. It illustrates a finite shared-weight mixture; the full stochastic construction requires additional ingredients.

AI Glossary / Statistics and probability / Advanced

Dependent Dirichlet process

Definition

A dependent Dirichlet process is a Bayesian construction for a collection of related random probability distributions indexed by predictors, groups, time, or other context. In MacEachern's classical formulation, each distribution individually has a Dirichlet-process marginal law, while the distributions are dependent across index values. This allows information sharing without requiring every context to have exactly the same distribution. Related extensions sometimes relax the Dirichlet-process marginal requirement.

What it means

An ordinary Dirichlet process places a prior over one unknown probability distribution. A dependent construction instead models a family of distributions, such as a distribution of service times for each workload level.

The goal can be to let the whole distribution change, including its shape, spread, and multiple peaks, rather than modeling only a changing mean. Dependence links the distributions so that evidence in one context can inform another according to the specified prior.

The word dependent concerns the random distributions. It does not automatically impose temporal dependence between every pair of observations, identify causation, or specify how nearby contexts should be related. Those features come from the chosen construction and observation model.

How it works

A Dirichlet process can be represented by an infinite weighted collection of point masses. A dependent version lets the weights, point locations called atoms, or both vary with an index x. Suitable stochastic processes connect these quantities across x while preserving the required marginal distributions.

In a single-weights construction, weights are shared and atoms vary with context. In a single-atoms construction, atom locations are shared and weights vary. These designs imply different kinds of distributional change and information sharing.

For continuous outcomes, the random measure often serves as a mixing distribution over kernel parameters. Integrating a continuous kernel against that measure yields a density, even though the underlying Dirichlet-process draw is discrete. Posterior computation estimates the relevant random measures and parameters from data. Truncation, simulation, or other inference methods require accuracy checks, and smoothness across contexts depends on the selected processes rather than the name alone.

Three practical examples

A research team models how an entire service-time distribution changes with workload rather than assuming that only the average changes.
An analyst studies related groups with limited observations, using a dependent prior to share information while allowing different distribution shapes.
An AI forecasting project explores flexible predictive distributions across operating conditions, comparing the added complexity with simpler distributional models.

Think of it this way

Think of related maps for different settings. Shared structure allows learning about one to inform another. The model must specify what is shared and what can change.

Worked example

Consider an original finite illustration of shared weights with context-dependent component locations. It is a three-component approximation for teaching, not an exact draw from an infinite Dirichlet process or a fitted posterior.

Take stick fractions v1 = 0.6 and v2 = 0.625. The first weight is 0.6. The second is (1−0.6)×0.625 = 0.25. The remaining mass is 0.15, which this toy approximation assigns entirely to a third component. The weights sum to one.

Let component means at context x be 1+x, 4+2x, and 8+x, with normal observation kernels of variance one. At x = 0, the mixture mean is 0.6×1 + 0.25×4 + 0.15×8 = 2.8. At x = 1, it becomes 0.6×2 + 0.25×6 + 0.15×9 = 4.05. The components move differently even though their weights remain unchanged.

The mixture variance at x = 0 is 1 + [0.6×1² + 0.25×4² + 0.15×8² − 2.8²] = 7.36. At x = 1 it is 1 + [0.6×2² + 0.25×6² + 0.15×9² − 4.05²] = 8.1475. Thus a shared-weight construction can change both mean and spread through its atoms. A full DDP model adds the appropriate random-process priors and inference; these fixed illustrative numbers do not establish those properties by themselves.

Illustrative case study

In this fictional case, Aspen Scheduling studies completion times under different workload conditions. A simple regression captures the average but misses a changing mix of short and long jobs. Researchers investigate a dependent mixture model to describe the full predictive distribution.

They examine whether shared weights or context-varying weights better match the intended assumptions, test predictions on held-out conditions, and assess sensitivity to the prior and computational approximation. They retain a simpler model when additional flexibility does not improve relevant predictions.

This invented case demonstrates evaluation, not superiority.

Common misconceptions and mistakes

Nonparametric does not mean assumption-free. The base distributions, concentration parameters, dependence structure, kernels, and computation all influence the model. A highly flexible prior still needs enough relevant data.

A Dirichlet-process draw is discrete, but a mixture using it can have a continuous outcome density. Confusing the mixing measure with the resulting observation distribution obscures this distinction.

Do not assume every covariate-dependent stick-breaking model has Dirichlet-process marginals. That property needs construction-specific verification. Nor is a dependent Dirichlet process identical to every hierarchical Dirichlet-process model. The terms describe related families with different sharing mechanisms. Finally, extrapolation beyond observed contexts can be driven strongly by the assumed atom or weight processes and should not be mistaken for evidence from data.

Business use

Businesses may benefit when the distribution's shape matters for decisions, such as allocating capacity for both routine and unusually long work. The practical question is whether flexible information sharing improves predictive decisions enough to justify modeling and computation costs.

Define the index, observation unit, sharing assumptions, and target quantities before choosing a construction. Evaluate tail probabilities or quantiles directly if those drive the decision. Check posterior computation, approximation sensitivity, and predictive behavior in sparse contexts. Communicate uncertainty about both the distributions and the assumptions connecting them, rather than presenting a smooth family of curves as automatically established fact.

Check your understanding

Check your understanding: Why is the second weight 0.25?

Answer: It takes 0.625 of the remaining 0.4 stick.

What changes with x in the example?

Answer: Component means, and consequently the mixture distribution.

Why can the density be continuous?

Answer: Continuous kernels are mixed.

Does context dependence automatically imply causation?

Answer: No.

Related terms

Dirichlet process
Bayesian nonparametrics
stick-breaking construction
random probability measure
mixture model
density regression
hierarchical Dirichlet process

Learn next

Study Dirichlet-process mixtures, compare single-weights and single-atoms constructions, and examine how prior dependence and truncation affect predictions.

Dirichlet process

Sources

Sources: Quintana, Müller, Jara and MacEachern, The Dependent Dirichlet Process and Related Models, https://arxiv.org/pdf/2007.06129, especially the classical construction and single-weights discussion. Prior Design for Dependent Dirichlet Processes, https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0147402. The finite approximation and fictional scheduling case are original and explicitly distinguished from a full DDP fit.

Previous: Dependency tree Next: Dependent variable