Is it the same as the Wilcoxon rank-sum test?

They provide equivalent rank-based testing information under corresponding conventions, although their reported raw statistics differ.

Is it suitable for before-and-after data from the same people?

Not in its ordinary independent-samples form.

Can it handle tied ratings?

Yes, with a calculation that appropriately accounts for ties and sample size.

AI Glossary / Statistics and probability / Intermediate

Mann-Whitney U test

Definition

The Mann-Whitney U test is a rank-based statistical test for comparing two independent samples. In its standard form, it tests the null hypothesis that the samples come from the same distribution. It is often used to investigate whether values in one population tend to be larger than values in another. Interpreting it specifically as a comparison of medians requires additional assumptions about the distributions, such as a common shape with a possible location shift.

What it means

Instead of comparing numerical averages directly, the test considers how the observations from the two groups are ordered. The U statistic can be understood through comparisons of every observation in one group with every observation in the other.

This makes it useful for outcomes whose ordering is meaningful even when numerical spacing is questionable. It also reduces sensitivity to the numerical magnitude of an extreme observation. Nevertheless, rank-based analysis still requires an appropriate sampling or experimental design and careful treatment of ties, dependence, and the intended hypothesis.

How it works

Pool the observations and assign ranks, using average ranks for tied values. For a first group of size n with rank sum R, one common convention defines U = R − n(n + 1)/2. If the second group has size m, the complementary statistic is nm − U.

Under this convention, U counts first-group values exceeding second-group values, with half a contribution for ties. Compare the statistic with its null distribution using an exact calculation, an appropriate permutation procedure, or a large-sample approximation with suitable adjustments.

Choose a one-sided or two-sided alternative before examining the results. Report which group defines U and how the p-value was computed, because software conventions and numerical approximations can differ. Independence applies to observations within and between groups under the usual formulation; paired observations require a different analysis.

Three practical examples

A randomized interface experiment compares task-completion times from different people assigned to two designs.
A service team compares ordered satisfaction ratings from independent samples of customers using two support channels.
An AI evaluation compares independently sampled human quality ratings under two experimental conditions, while ensuring repeated ratings from the same evaluator do not masquerade as independent observations.

Think of it this way

Think of two teams entering a race. Rather than averaging their finishing times, you compare how often a runner from the first team finishes ahead of a runner from the second. The ordering supplies the information. The analogy needs a direction convention: in a race smaller times are better, while U commonly counts numerically larger first-group values.

Worked example

Consider original samples A = 1, 2, 4 and B = 3, 5, 6, with larger values representing a higher measured outcome. The pooled ranks equal these values. Group A's rank sum is 1 + 2 + 4 = 7. Its statistic is U_A = 7 − 3 × 4/2 = 1. The complementary statistic is U_B = 9 − 1 = 8.

Direct checking gives the same answer: only A's value 4 exceeds a B value, namely 3. Thus one of nine cross-group comparisons favors A, and the empirical superiority proportion is 1/9, approximately 0.111.

There are 20 equally likely allocations of three of the six distinct ranks to A under the exchangeable null. Two allocations have U at most 1, and two have U at least 8. The symmetric two-sided exact p-value is therefore 4/20 = 0.20.

This small sample does not provide strong evidence against identical distributions at a 5% threshold. Its p-value does not establish equivalence, and the observed ordering proportion is not a precise population estimate.

Illustrative case study

Fictional case: AnswerNest compares independently recruited customers' ratings of two chatbot explanations. The team records the full rating distributions and notices many identical scores on its five-point scale.

It selects a procedure that handles ties appropriately and reports the sample sizes, direction of comparison, U statistic, and p-value alongside descriptive results. When it discovers that some customers rated both explanations, it separates that paired component for a suitable analysis. The case illustrates that choosing a nonparametric test does not remove the need to understand how the observations were generated.

Common misconceptions and mistakes

The test is not automatically a test of median equality. Differences in shape or spread complicate a location-only interpretation, and some unequal distributions are poorly detected by this statistic.

Nonparametric does not mean assumption-free. Treating repeated measurements as independent can invalidate inference. A conventional exact calculation intended for data without ties should not be used uncritically when ties are present. A small p-value does not measure the size or practical importance of a difference, and failure to reject does not demonstrate that the populations are identical.

Business use

Businesses can use the test when comparing independent groups on ordered outcomes and when the scientific question concerns relative ordering. It can complement distribution plots, medians, other descriptive summaries, and an effect estimate with appropriate uncertainty.

Before selecting it, decide whether the decision actually concerns a mean, a median, a probability of superiority, or another feature. These are different targets. If average monetary cost is the quantity that matters, replacing a mean comparison with a rank test may answer a different business question.

Check your understanding

Two groups have the same sample median but different spreads. Must their Mann-Whitney result be nonsignificant?

Answer: No. The statistic uses cross-group ordering, not only the two medians. Its interpretation and behavior depend on the full distributions and sample sizes.

Related terms

Rank
Nonparametric tests
Permutation test
Wilcoxon signed-rank test
Effect size
Wilcoxon rank-sum test
nonparametric test
probability of superiority
independent samples

Learn next

Learn the distinction between independent and paired samples, then study exact permutation distributions and tie handling. Practice identifying the estimand before choosing between rank-based methods and tests designed for population means.

Rank

Sources

SciPy, official mannwhitneyu reference, including hypotheses, rank-sum relationship, and calculation methods: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.mannwhitneyu.html

The six-observation example is original and was checked by enumerating all 20 rank allocations. AnswerNest is fictional.

Previous: Manipulation check Next: Manner adverb