Is a probability simplex always a triangle?

No. Two categories give a line segment, three give a triangle, four give a tetrahedron, and larger numbers give higher-dimensional analogs.

Can a simplex component equal zero?

Yes. Boundary points include zero components.

Does softmax make probabilities accurate?

No. It enforces a valid categorical form, but predictive accuracy and calibration require evaluation.

AI Glossary / Statistics and probability / Intermediate

Probability simplex

Definition

The probability simplex is the set of all vectors whose components are nonnegative and sum to one. With k components, each vector can represent a probability distribution over k mutually exclusive, exhaustive categories. The simplex has dimension k − 1 because the sum constraint determines one component from the others. It is a geometric way to describe the space of valid finite probability distributions.

What it means

A classifier that predicts probabilities for three categories produces a point on a triangular simplex. Each vertex represents complete probability on one category. Points along an edge assign zero probability to the opposite category, and interior points assign positive probability to all three.

The simplex describes mathematical validity, not predictive quality. Two models can produce valid simplex vectors while differing greatly in accuracy or calibration. Category meanings and ordering must remain attached to the vector. Without them, a list of probabilities cannot tell a reader which outcomes the numbers describe.

How it works

For k categories, write a vector p = (p1, ..., pk). Require every component to be at least zero and require their sum to equal one. These constraints define the standard probability simplex, often written with a subscript or superscript indicating its dimension.

One way to generate a valid vector is to normalize nonnegative scores by their positive total. Another common construction is softmax, which converts finite real-valued scores into strictly positive probabilities that sum to one. Different transformations imply different modeling behavior; normalization alone does not justify interpreting arbitrary scores as calibrated probabilities.

Weighted averages of simplex vectors remain on the simplex when the weights are nonnegative and sum to one. This property supports mixtures of categorical distributions. Optimization algorithms may also enforce the constraints directly or project candidate vectors onto the simplex. Such projection is a particular mathematical operation, not simply a synonym for any normalization procedure.

Three practical examples

A text classifier outputs a simplex vector over billing, technical support, and account-management categories.
A mixture model uses simplex-valued weights to specify the proportions assigned to its components.
A decision system represents a randomized choice among several actions using probabilities that sum to one.

Think of it this way

Think of allocating one complete budget across labeled envelopes. Each envelope receives a nonnegative share, and all shares together exhaust the budget. The simplex is the collection of every allowed allocation, from putting everything in one envelope to spreading it among all of them.

Worked example

In an original three-category example, model A reports p = (0.20, 0.30, 0.50), while model B reports q = (0.50, 0.20, 0.30). Both vectors have nonnegative components summing to one.

Form a mixture giving model A weight 0.40 and model B weight 0.60. Its first component is 0.40 × 0.20 + 0.60 × 0.50 = 0.38. The second is 0.40 × 0.30 + 0.60 × 0.20 = 0.24. The third is 0.40 × 0.50 + 0.60 × 0.30 = 0.38.

The mixture (0.38, 0.24, 0.38) sums to one and remains on the same simplex. By contrast, (0.80, 0.40, −0.20) sums to one but is invalid because one component is negative.

The valid mixture is not automatically better than either model. Its predictive performance depends on the data and on whether combining their distributions is appropriate for the intended task.

Illustrative case study

Fictional case study: Cedar Routing combines category probabilities from two document models. During a review, the team discovers that the models list the same categories in different orders. Averaging the raw positions produces numerically valid but semantically incorrect probabilities.

The team aligns category labels before combining predictions, checks the simplex constraints, and evaluates the resulting distribution on held-out documents. This fictional example illustrates that geometric validity and correct interpretation are separate requirements. It does not claim that ensembling improves a particular deployment.

Common misconceptions and mistakes

A vector whose components sum to one is not necessarily a probability vector: negative values are disallowed. Conversely, a nonnegative vector is not on the simplex unless it is normalized to a total of one.

The k-component simplex has dimension k − 1, not k, because of its sum constraint. Also, a point on the boundary is allowed; zero-probability categories are mathematically valid even though finite-logit softmax outputs are strictly positive in exact arithmetic. Finally, separate probabilities for overlapping labels need not sum to one. Forcing a multilabel prediction onto a categorical simplex can change the meaning of the task.

Business use

The simplex helps teams inspect category predictions, mixture weights, and randomized policies. Its constraints provide simple data-quality checks and clarify why changing one probability requires compensating changes elsewhere.

For operational AI systems, validate both numeric constraints and the category schema. Allow reasonable numerical tolerances for floating-point sums, but investigate substantial deviations. Keep calibration, discrimination, and downstream decision quality as separate evaluations. If categories overlap or do not cover all possibilities, revise the representation rather than assuming that every set of confidence scores belongs on one simplex.

Check your understanding

Is (0.15, 0.25, 0.60) on the three-category probability simplex, and how many independent coordinates does that simplex have?

Answer: Yes. All components are nonnegative and sum to one. The simplex has two independent coordinates because the third equals one minus the first two.

Related terms

Probability mass function (pmf)
Softmax
Normalization
Calibration
Constrained optimization
Probability vector
probability mass function
categorical distribution
mixture model
convex combination

Learn next

Practice checking probability vectors and forming mixtures. Then explore the geometry of the three-category simplex and how softmax maps scores into its interior. Study constrained optimization and calibration to understand why a mathematically valid vector still needs empirical and contextual validation.

Sources

The Aalto Dictionary of Machine Learning defines the probability simplex and explains softmax as a mapping into it: https://aaltodictionaryofml.github.io/ADictML_Main.pdf

The mixture calculation, invalid-vector comparison, category-ordering scenario, and fictional case are original. The discussion concerns mutually exclusive categorical outcomes; overlapping multilabel events require a different interpretation.

Previous: Probability Ranking Principle Next: Probability space