How does it relate to the binomial distribution?

With two categories, one count determines the other, giving the binomial case.

Can one trial belong to several categories?

Not in the ordinary multinomial experiment. Multiple labels require a different representation or model.

Why is the covariance negative?

With a fixed total, more observations in one category leave fewer available for another under the model.

AI Glossary / Statistics and probability / Intermediate

Multinomial distribution

Definition

The multinomial distribution describes the counts of outcomes in a fixed number of independent trials when each trial falls into exactly one of several categories and the category probabilities stay constant. It generalizes the binomial distribution from two categories to K categories. The count vector always sums to the number of trials, and the category probabilities sum to one. A multinomial model is useful for categorical counts, but its assumptions must match the sampling process. Dependence, changing probabilities or structural constraints can require a different model.

What it means

Imagine repeating a categorical experiment and recording how many times each outcome occurs. The multinomial distribution assigns a probability to every possible count combination. It does not specify the order in which those outcomes occurred; many different sequences produce the same counts. Because the total is fixed, the category counts are not independent even when the underlying trials are independent. A larger count in one category leaves fewer trials available for the others. Each individual count has a binomial marginal distribution when all remaining categories are grouped together.

How it works

Specify the number of trials n and probabilities p1 through pK. For nonnegative integer counts x1 through xK summing to n, the probability is n! divided by the product of the xi! values, multiplied by the product of pi raised to xi. The factorial factor counts the sequences with those category totals. The expected count in category i is npi, its variance is npi(1-pi), and the covariance of distinct counts i and j is -npipj. Estimate probabilities or test hypotheses using methods appropriate to the sampling design and sample size, rather than assuming all observed categorical tables are automatically multinomial.

Three practical examples

Service categories: model the counts of independently sampled requests assigned to several exclusive types.
Survey responses: describe totals across mutually exclusive response options under an appropriate sampling model.
Simulation: generate random allocations of a fixed number of independent events among categories with specified probabilities. The model applies to counts rather than directly to continuous proportions, even though counts can be divided by the total to produce sample proportions.

Think of it this way

Think of placing a fixed number of tokens into labeled boxes, choosing one box independently for each token using the same probabilities. The final box totals follow a multinomial pattern. Once one box receives more tokens, fewer remain for the others, explaining why the totals are linked despite the independent placement decisions.

Worked example

Suppose five independent requests belong to categories A, B and C with probabilities 0.5, 0.3 and 0.2. The probability of exactly two A requests, two B requests and one C request is [5!/(2!2!1!)] times 0.5 squared times 0.3 squared times 0.2. The sequence-count factor is 30, giving probability 0.135. One particular ordering with those counts has probability 0.0045; multiplying by 30 includes every ordering with the same totals. The expected A count is 5 times 0.5 = 2.5, which need not be an integer because it is a long-run average. Its variance is 5 times 0.5 times 0.5 = 1.25, and the A-B covariance is -5 times 0.5 times 0.3 = -0.75. This original example assumes constant probabilities and independent requests.

Illustrative case study

In a fictional case, Birch Support uses a multinomial model to simulate the category mix of a fixed number of incoming tickets. Reviewers discover that tickets from the same outage arrive in clusters and share a category, violating the independent-trial assumption. The team compares the observed variability with the model's predictions and considers a model that accounts for clustering. It retains the multinomial calculation as a transparent baseline while avoiding unsupported claims that its uncertainty estimates describe every operating period.

Common misconceptions and mistakes

Independent trials do not imply independent category counts when their total is fixed. Expected counts can be fractional even though realized counts are integers. The multinomial coefficient is essential when calculating the probability of counts rather than one ordered sequence. Categories must be mutually exclusive and collectively cover the modeled outcomes. A model with varying probabilities across trials is not the ordinary multinomial distribution. Sampling without replacement from a small finite population generally calls for a multivariate hypergeometric model rather than treating the draws as independent.

Business use

Businesses should identify the sampling unit and confirm what fixes the total count. Check whether observations are clustered, probabilities change over time or categories are assigned inconsistently. Report estimated proportions with uncertainty that reflects the actual design. When comparing category mixes, distinguish changes in total volume from changes in relative probabilities. Sparse categories can make asymptotic tests unreliable, so consider suitable exact or simulation-based methods. The multinomial model provides a clear baseline for categorical variation, but its simplicity should not conceal dependence or incomplete category definitions.

Check your understanding

Five trials produce three category counts. Must each count have expectation equal to an integer?

Answer: No. Actual counts are integers, but expected values such as 2.5 are valid averages over repeated experiments.

Related terms

Binomial distribution
Probability mass function (pmf)
Count data
Covariance
Dirichlet distribution
Categorical distribution
probability mass function
multinomial coefficient
goodness-of-fit test

Learn next

Practice distinguishing one ordered sequence from a count vector and calculate both probabilities. Then examine multinomial estimation and goodness-of-fit testing, including how dependence changes the uncertainty calculation.

Goodness-of-fit

Sources

Penn State STAT 504, Introduction to Discrete Data, multinomial distribution: https://online.stat.psu.edu/stat504/Lesson01 Penn State STAT 504, Binomial and Multinomial Inference: https://online.stat.psu.edu/stat504/Lesson02

Previous: Multimodal variational autoencoder Next: Multinomial logistic regression