Can it handle tied scores?

Yes, through half-weight comparisons or equivalent midranks.

No. Repeated measurements on the same people require a suitable paired or dependent-data analysis.

AI Glossary / Statistics and probability / Advanced

Brunner-Munzel test

Definition

The Brunner-Munzel test compares two independent populations through the probability that a randomly selected observation from one exceeds an observation from the other, counting ties as half. It tests whether this relative probability equals one-half while allowing the populations to have different spreads and distributional shapes.

What it means

Suppose two groups receive ordered quality scores. Rather than comparing average scores, the question is whether one group tends to receive larger scores in cross-group comparisons. The ordering matters; numerical gaps between adjacent score categories need not represent equal amounts of quality.
Write the target as theta = P(Y greater than X) plus one-half P(Y equals X). The null hypothesis is theta = 0.5. This is not a claim that both populations have identical distributions, equal variances, or equal means. A probability of superiority above one-half also does not imply that every part of one distribution dominates the other.

How it works

Compare observations across groups, assigning one for a larger Y value, zero for a smaller Y value, and one-half for a tie. Their average estimates theta. Rank calculations provide an efficient equivalent approach.
Uncertainty accounts separately for the variability contributed by each independent sample. It does not treat every cross-group comparison as an independent observation, because the same original observations appear repeatedly.
The standardized difference from one-half is assessed with a suitable reference distribution. Common implementations offer a t approximation with estimated degrees of freedom or a large-sample normal approximation. Very small samples may need a carefully implemented studentized permutation approach; ordinary label permutation is not automatically exact under the weaker probability null.

Three practical examples

An evaluator compares ordered usefulness ratings from two independently sampled user groups. Ties are expected because only a few rating categories are available.
A service analyst compares completion times under two configurations with different variability. The interpretation must acknowledge that smaller times, rather than larger values, are preferable.
An AI study compares independent groups of evaluation scores whose shapes differ. It reports the estimated cross-group probability alongside uncertainty, rather than assuming a difference in means is being tested.

Think of it this way

Imagine drawing one score from each of two bags. Give the second bag a point when its score is larger and half a point for a tie. The question concerns its long-run average points per draw, not whether the bags contain the same mix of scores.

Worked example

Take invented samples X = 1, 2, 4, 5 and Y = 2, 3, 4, 6. There are sixteen cross-group comparisons, but only eight original observations.
When X equals 1, all four Y values are larger, producing four points. When X equals 2, the tie contributes one-half and the other three comparisons contribute three, totaling 3.5. When X equals 4, the comparison points total 1.5. When X equals 5, they total one.
Altogether, nine comparisons have Y strictly larger, two are ties, and five have Y smaller. The estimated relative probability is (9 plus half of 2)/16 = 10/16 = 0.625.
For an illustrative standard-error calculation, average the comparison points separately for each X observation. These averages are 1, 0.875, 0.375, and 0.25. Their unbiased sample variance is 13/96. Average separately for each Y observation to obtain 0.375, 0.5, 0.625, and 1, with unbiased sample variance 7/96.
The estimated variance of the relative-probability estimator is (13/96)/4 plus (7/96)/4 = 5/96. Its standard error is approximately 0.22822. Standardizing 0.625 minus 0.5 gives approximately 0.54772.
For the t approximation, the Welch-style degrees of freedom are [(13/384 plus 7/384) squared] divided by [(13/384) squared/3 plus (7/384) squared/3]. This equals 600/109, approximately 5.505. These computations demonstrate the mechanics; four observations per group are too few to casually rely on asymptotic calibration.
Changing the orientation to X larger than Y changes the estimated probability to 0.375 and reverses the statistic's sign. The two-sided evidence is unchanged. Software conventions therefore matter when describing direction.
Finally, 0.625 is an estimate about two independently drawn observations. It does not say that 62.5 percent of individuals would personally improve if switched from one configuration to the other. That would require a different causal interpretation and study design.

Illustrative case study

Fictional company Willow Answers compares ordered usefulness ratings from independent groups using two support interfaces. The scores contain many ties, and one group has a wider spread. The analyst chooses a relative-probability comparison rather than describing the result as a test of average satisfaction.
The report names which group appears first in the calculation and explains whether higher scores are better. It also checks that multiple ratings by one person have not been counted as independent participants. The team considers the estimated effect, uncertainty, and actual rating distributions before changing the interface. This fictional scenario illustrates a decision process, not a reported commercial result.

Common misconceptions and mistakes

Nonparametric does not mean assumption-free. Independence, sampling quality, and adequate information remain important.
The null is not simply equal medians. Different distributional shapes can make median-based interpretations misleading.
Sixteen comparisons in the example do not create sixteen independent data points. Reusing observations creates dependence among those comparisons.
A p-value is not the probability that one group is better. The estimated relative probability and the test p-value answer different questions.
Complete separation can produce a degenerate estimated variance, so software warnings or undefined results need investigation rather than automatic interpretation.

Business use

Use this test when ordered outcomes and a probability-of-superiority question fit the decision. Specify the direction, independent sampling unit, handling of ties, and inference method. Report an effect estimate and uncertainty with the original distributions. Determine whether a statistically detectable ordering difference is large enough to matter operationally.

Check your understanding

Why is the example estimate 0.625?

Answer: Nine wins plus half of two ties gives ten points across sixteen comparisons.

What does the null specify?

Answer: A relative probability of one-half.

Why record orientation?

Answer: Reversing the groups reverses the direction of the comparison.

Related terms

Effect size
Probability of superiority
Mann–Whitney U test
midrank
nonparametric inference
independent samples
studentized permutation test

Learn next

Compare relative-probability inference with tests of means and medians. Study confidence intervals for probabilistic effects and methods for clustered or paired ratings.

Sources

SciPy, brunnermunzel: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.brunnermunzel.html . Statsmodels, rank_compare_2indep: https://www.statsmodels.org/stable/generated/statsmodels.stats.nonparametric.rank_compare_2indep.html . These references identify the Brunner-Munzel and studentized permutation literature. Numerical samples, calculations, and company scenario are original illustrations.

Previous: Broyden - Fletcher - Goldfarb - Shanno algorithm Next: Brushing