Can it involve more than two groups?

Yes; the contrast coefficients define the comparison.

Is independence required by the definition?

No, but dependence must be represented correctly.

Does more data change the population probability?

Not by itself; more data can improve its estimation.

AI Glossary / Statistics and probability / Advanced

C+-probability

Definition

C+-probability is the probability that a specified contrast variable is strictly positive. A contrast variable combines random outcomes from two or more groups using coefficients that sum to zero. The quantity describes the direction of a comparison at the level of outcomes, rather than the statistical significance of an estimated mean contrast.

What it means

Suppose an analyst compares one operating condition with the average of two alternatives. The contrast can be V = A minus one-half B minus one-half C. C+-probability is then P(V greater than zero): the chance that the selected A outcome exceeds the average of the selected B and C outcomes under the specified joint distribution.
This terminology appears in Zhang's work on contrast variables and effect sizes. The coefficients, outcome direction, and relationship among the group draws are essential parts of the definition.

How it works

Specify the contrast and the distributions of the component outcomes. For independent draws, their joint distribution factors into the component distributions. For matched or otherwise dependent outcomes, preserve the relevant dependence instead of independently combining observations.
Calculate the probability mass or density for which the resulting contrast is above zero. Enumeration, integration, or simulation can perform this calculation when the distributions are specified. Estimating those distributions from finite data introduces uncertainty that should be reported separately.
Under a normal contrast distribution with positive standard deviation, the probability is the standard normal cumulative distribution evaluated at the contrast mean divided by its standard deviation. That normal conversion should not be applied automatically to a discrete or strongly nonnormal contrast.

Three practical examples

An experimental analyst compares one configuration with an average of two reference configurations. The coefficients express the intended comparison before the probability is evaluated.
A researcher investigates an interaction contrast across four conditions. A contrast variable can represent the difference between two differences, rather than forcing every question into a single pairwise comparison.
An AI evaluation study examines hypothetical score distributions for several systems. A positive contrast indicates the prespecified direction of advantage, while its practical value still depends on what the scores measure.

Think of it this way

Imagine drawing a card from each of several decks and combining the printed numbers with a fixed scoring rule. C+-probability asks how often the combined score is above zero. It does not ask how confident you are that the average score has been estimated correctly.

Worked example

Define three independent outcome distributions. A is equally likely to be 2 or 4. B is equally likely to be 1 or 3. C is equally likely to be 1 or 5. Use V = A minus (B plus C)/2. The coefficients 1, minus one-half, and minus one-half sum to zero.
There are eight equally likely triples, each with probability one-eighth. In the order (A, B, C), the triples and their contrasts are: (2, 1, 1) gives 1; (2, 1, 5) gives minus 1; (2, 3, 1) gives 0; and (2, 3, 5) gives minus 2.
The remaining four triples are (4, 1, 1), giving 3; (4, 1, 5), giving 1; (4, 3, 1), giving 2; and (4, 3, 5), giving 0.
Four contrasts are positive, two are zero, and two are negative. Therefore, C+-probability is 4/8 = 0.5. The probability of a nonnegative contrast is instead 6/8 = 0.75. If someone chooses a separate convention that awards half credit to ties, the result is (4 plus 1)/8 = 0.625. These are three different quantities; the strict positive definition selects the first.
The mean contrast is [1 minus 1 plus 0 minus 2 plus 3 plus 1 plus 2 plus 0]/8 = 0.5. Thus a positive mean contrast can coexist with a strict positive probability of exactly one-half. Distributional details and zero outcomes matter.
Reverse every coefficient to compare the opposite direction. The new positive probability is the old negative probability, 2/8 = 0.25. It is not one minus 0.5, because the zero outcomes retain probability 0.25. In general, P(V greater than zero) plus P(V less than zero) equals one minus P(V equals zero).
Multiplying all coefficients by a positive constant leaves every sign unchanged and preserves C+-probability. Adding the same constant to all three outcomes also leaves V unchanged because the coefficients sum to zero. These checks help distinguish the comparison's direction from arbitrary measurement origins and positive scaling.

Illustrative case study

Fictional company Birch Evaluation compares a proposed system with two reference systems using a prespecified contrast of quality scores. The analyst explains whether scores are drawn independently from separate evaluation populations or jointly from matched evaluation cases.
The team estimates the contrast's positive probability and reports its uncertainty. It also examines the size of unfavorable outcomes: frequently being slightly ahead does not necessarily compensate for occasionally being far behind. Decision-makers retain the original score distributions and the comparison rule alongside the probability. This fictional scenario demonstrates interpretation, not an established use of this terminology across commercial AI products.

Common misconceptions and mistakes

C+-probability is not a p-value. It describes an outcome comparison, not the probability of data at least as extreme under a null hypothesis.
It is not automatically a posterior probability that a population mean contrast is positive.
Strict positivity excludes zero outcomes. Half-weight ties belong to a different explicitly defined index.
The marginal distributions alone may be insufficient when outcomes are dependent. Changing the pairing can change the contrast distribution.
A higher positive probability does not reveal the magnitude of wins and losses or establish which decision has higher expected value.

Business use

Use the concept when the contrast itself matches the decision and an outcome-level directional probability is useful. State the coefficients, sampling interpretation, tie rule, and estimation method. Pair the result with magnitude and uncertainty summaries. Avoid presenting it as a universal performance score or as evidence that sampling uncertainty has disappeared.

Check your understanding

Why is the example probability 0.5?

Answer: Four of eight equally likely contrasts are strictly positive.

Why is the reversed probability 0.25?

Answer: Only two original contrasts are negative.

Does a positive average guarantee more than half the contrasts are positive?

Answer: No, as the example shows.

Related terms

Effect size
P-value
Contrast variable
probability of superiority
standardized mean of a contrast variable
joint distribution
dependence

Learn next

Study contrast coefficients, matched versus independent comparisons, and uncertainty intervals for probabilistic effect sizes. Compare strict-positive and half-tie conventions.

Sources

Zhang, Contrast Variable Potentially Providing a Consistent Interpretation to Effect Sizes, 2010: https://www.hilarispublisher.com/open-access/contrast-variable-potentially-providing-a-consistent-interpretation-to-effect-sizes-2155-6180.1000108.pdf . Original 2009 method abstract: https://pubmed.ncbi.nlm.nih.gov/20397965/ . Discrete distributions, calculations, and company scenario are original illustrations.

Previous: Byzantine device Next: C4.5 algorithm