Can marginals be recovered from a joint distribution?

Yes, by summing or integrating.

Can a density exceed one?

Yes; its integral must satisfy probability rules.

AI Glossary / Statistics and probability / Advanced

Joint probability distribution

Definition

A joint probability distribution describes how two or more random variables behave together. It assigns probabilities to combinations or regions of their possible values. Unlike a list of separate marginal distributions, it also specifies their dependence structure. For discrete variables it can be represented by a joint probability mass function; a continuous model may be represented by a joint density when such a density exists.

What it means

Knowing the separate distributions of response time and answer quality does not tell you whether the slow responses tend to be the good ones. A joint model represents both variables for the same observation and allows questions about their combinations.
The same idea extends to more variables, such as request type, response time, quality, and user outcome. The model must describe one coherent population or process. Combining unrelated datasets without a justified relationship does not automatically produce a meaningful joint distribution.

How it works

For discrete variables, specify a nonnegative probability for each possible combination and ensure the total is one. Sum cells belonging to an event to obtain its probability. Summing over one variable gives the marginal distribution of the others.
For a model with a joint density, integrate the density over the desired region. Density values themselves are not probabilities. Conditional distributions can be obtained by dividing a joint mass or density by the appropriate positive marginal mass or density.
Independence is a special case in which the joint distribution factors into the product of marginal distributions. It should not be imposed merely because separate summaries are available.

Three practical examples

An AI service team models response quality and whether the response meets a time target, so it can assess combined service performance.
A simulator generates demand and processing time together, retaining dependence that affects congestion.
A classifier uses a probabilistic model of labels and observed features to derive conditional predictions for new observations.

Think of it this way

Separate distributions describe the ingredients available in two baskets. A joint distribution also describes how the ingredients are paired. The baskets can contain the same quantities while the pairing patterns differ substantially.

Worked example

Consider an original binary model for AI responses. Let X equal one if a response meets a quality criterion and zero otherwise. Let Y equal one if it meets a time target and zero otherwise.
Assign probabilities 0.10 to (X = 0, Y = 0), 0.20 to (0, 1), 0.30 to (1, 0), and 0.40 to (1, 1). These four mutually exclusive outcomes cover every case and their probabilities sum to one.
The quality marginal is P(X = 1) = 0.30 plus 0.40 = 0.70. The time marginal is P(Y = 1) = 0.20 plus 0.40 = 0.60. The probability of satisfying both requirements is 0.40, which is a particular event probability obtained from the joint distribution.
The probability of satisfying at least one requirement is 0.20 plus 0.30 plus 0.40 = 0.90. Exactly one is satisfied with probability 0.20 plus 0.30 = 0.50. These operational questions differ.
Among responses meeting the quality criterion, the probability of meeting the time target is 0.40/0.70 = 4/7, approximately 0.5714. Among responses failing the quality criterion, it is 0.20/0.30 = 2/3, approximately 0.6667. Neither conditional proportion is the overall time-success probability of 0.60.
The expected value of X is 0.70, and that of Y is 0.60. Since their product equals one only when both indicators are one, E(XY) = 0.40. Their covariance is therefore 0.40 minus 0.70 times 0.60 = minus 0.02.
Now construct a second joint model using cells 0.12, 0.18, 0.28, and 0.42 in the same order. Its quality and time marginals remain 0.70 and 0.60, but the probability of meeting both requirements is now 0.42.
Every cell in this second table equals the corresponding product of marginal probabilities: for example, 0.30 times 0.40 = 0.12 and 0.70 times 0.60 = 0.42. The second model is independent. The first is not. Identical marginals therefore do not determine the joint model.
For a separate continuous illustration, let U and V lie between zero and one and have joint density f(u, v) = 2u on that square, with density zero outside. Integrating first over v and then u gives total probability one.
The probability that U is at most one half and V is at most one half is the integral of 2u over that lower-left square. The integral over u is one quarter, multiplied by the v interval length of one half, giving one eighth, or 0.125.
The marginal densities are fU(u) = 2u and fV(v) = 1 on their respective intervals, so this continuous joint density factors as their product. At u = 0.75, the joint density is 1.5. That value exceeding one is valid; probabilities come from integrals over regions, not from reading a density as a percentage.
These stipulated models illustrate calculations; real estimates require suitable data and validation.

Illustrative case study

Fictional company Cedar Responses initially monitors quality and speed separately. Its analyst adds a joint table to show how often individual responses meet both targets.
The team discovers that separate success rates alone cannot answer its combined-service question. It evaluates the observed pairing and checks whether the pattern changes with request type before adopting a model for planning. This fictional scenario illustrates a measurement improvement, not a demonstrated product-performance gain.

Common misconceptions and mistakes

Marginal distributions do not generally determine dependence.
A joint density is not a table of point probabilities.
A full joint distribution is more information than one joint event probability.
Zero covariance does not generally imply independence.
Not every joint distribution has an ordinary multivariate density; some concentrate probability on a lower-dimensional set.

Business use

Joint models are useful when decisions depend on combinations of outcomes, such as meeting several service requirements together. Validate combined events as well as individual variables. For many variables, consider structured probabilistic models rather than assuming that every combination can be estimated reliably from a limited dataset.

Check your understanding

What is the first model's probability of meeting both targets?

Answer: 0.40.

What is its covariance?

Answer: Minus 0.02.

Does the second model have different marginals?

Answer: No; its dependence structure differs.

Related terms

Learn next

Study marginalization and conditioning, then examine how graphical models represent larger joint distributions through conditional relationships.

Sources

Pishro-Nik, Introduction to Probability, Statistics, and Random Processes, Joint Probability Mass Function: https://www.probabilitycourse.com/chapter5/5_1_1_joint_pmf.php . Delft University of Technology, Joint, Marginal, and Conditional Distributions: https://mude.citg.tudelft.nl/fundamental-concepts/2025/fundamentals/joint_marginal_conditional.html . Models and company scenario are original illustrations.

Previous: Joint probability Next: Joint source and channel coding