Can a circuit model dependent variables?

Yes. Mixtures of factorized components can induce dependencies, as the worked example shows.

Are sum weights always probabilities?

In normalized mixture constructions they are, but broader circuit definitions may use unnormalized nonnegative weights.

Can missing values always be replaced by one at the leaves?

For a normalized leaf that is fully marginalized, its integral or sum is one. Partially observed multivariate leaves require the appropriate marginal calculation, not indiscriminate replacement.

AI Glossary / Probabilistic machine learning / Intermediate

Probabilistic circuit

Definition

A probabilistic circuit represents a probability distribution through a computational graph of simple functions, weighted sums, and products. With suitable structural properties, it supports particular probability queries efficiently and exactly relative to the represented model. Which queries are tractable depends on the circuit's structure and the capabilities of its input functions.

What it means

Many models describe rich probability distributions but make questions involving missing variables expensive to answer. A probabilistic circuit organizes the distribution's calculation so that selected inference operations can reuse a compact set of computations.
Its graph describes arithmetic computation rather than simply drawing a variable for every node and an edge for every dependency. Products combine factors, while sums mix alternatives. A mixture of locally factorized components can express dependencies in the resulting joint distribution, even when variables are independent within each component.

How it works

In a common normalized construction, leaves are tractable probability distributions, product nodes multiply functions on disjoint variable sets, and sum nodes combine functions with nonnegative weights that sum to one. More general circuit definitions allow unnormalized nonnegative functions, so normalization conditions should be stated.
Two important structural properties are decomposability, meaning the children of a product use disjoint variable scopes, and smoothness, meaning the children of a sum have the same scope. With tractable leaves, these properties support efficient marginalization by integrating or summing the relevant leaf variables and evaluating the graph upward.
Conditional probabilities can then be formed as ratios when the conditioning event has positive probability. Other tasks, such as finding a most probable assignment or calculating more complex information measures, may require additional structure. Efficient inference in the size of a circuit does not imply that a compact circuit can represent every distribution or be learned cheaply.

Three practical examples

Incomplete-record analysis: answer model-based probability queries when different records contain different subsets of observed attributes.
Probabilistic classification: compute a class distribution conditioned on the available features using the circuit's supported marginal and conditional queries.
Structured uncertainty research: compare tractable model families that permit exact selected queries rather than relying on a new sampling run for each query.

Think of it this way

Think of a reusable calculation diagram with junctions that either multiply compatible pieces or blend alternative explanations. Sharing subcalculations can save work. The analogy is limited: arbitrary wiring does not produce a valid probability model or guarantee efficient marginalization. The meaning of each input and the structural rules determine what the diagram can safely compute.

Worked example

Consider an original normalized circuit for binary variables X and Y. Its root mixes two product components with weights 0.6 and 0.4. In component A, P(X = 1) = 0.8 and P(Y = 1) = 0.7. In component B, the respective probabilities are 0.2 and 0.3. Within each component, X and Y are independent.
The circuit gives P(X = 1, Y = 1) = 0.6 × 0.8 × 0.7 + 0.4 × 0.2 × 0.3 = 0.336 + 0.024 = 0.36.
To marginalize Y, sum each component's Y distribution over both outcomes, obtaining 1. Therefore P(X = 1) = 0.6 × 0.8 + 0.4 × 0.2 = 0.56. Similarly P(Y = 1) = 0.54, so P(Y = 1 | X = 1) = 0.36/0.56 = 9/14, approximately 0.643.
The variables are dependent in the mixture: 0.56 × 0.54 = 0.3024 differs from their joint probability 0.36. This demonstrates why product components do not imply global independence. Both products have disjoint scopes, and both root children cover {X, Y}.

Illustrative case study

Fictional case: Meridian Records studies a dataset in which some noncritical attributes are absent. The analysts evaluate a probabilistic circuit because they need repeated conditional queries with different observed subsets.
They first verify the two-variable calculation above, then document which inference operations their larger circuit supports. They also examine whether missingness is informative; simply marginalizing an unobserved variable does not automatically model why it was missing. The team evaluates predictive behavior separately from computational tractability. This scenario is invented and makes no claim that a particular circuit improves real customer records or solves every missing-data problem.

Common misconceptions and mistakes

A probabilistic circuit is not a quantum circuit or a physical electronic circuit in this context. Its nodes implement a mathematical probability computation. Nor is every directed acyclic computational graph automatically a tractable probabilistic circuit for every query.
Exact inference means exact for the encoded distribution, subject to numerical implementation. It does not mean that the encoded distribution is the true data-generating process. Likewise, easy marginalization does not automatically make all optimization, learning, or maximum-probability queries easy.

Business use

The practical value lies in matching a model's inference capabilities to recurring business questions. If many different subsets of attributes must be conditioned on, reusable exact marginal calculations may be useful.
Evaluate circuit size, query latency, training cost, and predictive fit together. Specify which tasks are guaranteed tractable and which still require approximation. A small fast model with poor fit can answer its own probability questions precisely while providing little value for the actual application.

Check your understanding

In the worked circuit, does marginalizing Y require selecting its most likely value?

Answer: No. It sums over both values of Y. Each normalized Y leaf contributes one, giving P(X = 1) = 0.56.

Related terms

Marginalization
Conditional probability
Sum-product network
arithmetic circuit
tractable probabilistic model
mixture model
decomposability
smoothness
variable scope

Learn next

Build the two-component binary circuit and check all four joint probabilities sum to one. Compare summing out a variable with maximizing over it. Then study how smoothness and decomposability enable marginal queries and why stronger properties are needed for other inference tasks.

Sources

Choi, Vergari and Van den Broeck, Probabilistic Circuits: A Unifying Framework for Tractable Probabilistic Models, authors' technical report: https://yoojungchoi.github.io/files/ProbCirc20.pdf Dang, Vergari and Van den Broeck, Strudel: Learning Structured-Decomposable Probabilistic Circuits, original research record: https://proceedings.mlr.press/v138/dang20a.html Choi-Vergari-Van den Broeck Section 4.2 Definitions 15-16 and Proposition 17 were independently consulted.

Previous: Probabilistic causation Next: Probabilistic classification