AI Glossary / Causal inference and experimentation / Intermediate

Fowlkes-Mallows scores

Definition

Fowlkes-Mallows scores compare two partitions of the same observations by examining which pairs are grouped together. The Fowlkes-Mallows Index is the geometric mean of pairwise precision and pairwise recall. It rewards pairs placed together in both partitions and penalizes inappropriate merging or splitting. In clustering evaluation, one partition often serves as a reference. The score does not require cluster names to match, because it evaluates relationships between observations rather than the numerical labels assigned to clusters.

What it means

Clustering labels such as 0, 1, and 2 are arbitrary names. A correct grouping should not be penalized merely because another algorithm calls its first cluster “blue” instead of “0”. Pairwise comparison avoids this problem. For every pair of distinct observations, ask whether both belong together according to the reference and according to the predicted partition. The score summarizes agreement on positive pair relationships. It does not directly reward the many pairs kept separate in both partitions.

How it works

Count TP pairs that share a cluster in both partitions. Count FP pairs grouped together only in the predicted partition, and FN pairs grouped together only in the reference. Pairwise precision is TP/(TP + FP), and pairwise recall is TP/(TP + FN). Their geometric mean is the square root of their product, equivalently TP divided by the square root of (TP + FP)(TP + FN) when defined. State conventions for degenerate partitions with no positive pairs. The score is symmetric between partitions, although the precision and recall interpretations swap.

Three practical examples

• Reference-based clustering: compare discovered document groups with a reviewed topic partition without aligning cluster identifiers.
• Algorithm comparison: measure how similarly two clustering procedures group the same items, while recognizing that neither is necessarily ground truth.
• Hierarchical analysis: compare partitions obtained at a chosen cut level, recording that level because changing the number of groups changes the comparison.

Think of it this way

Think of comparing two seating plans by checking which pairs of guests share a table. Table names are irrelevant; the relationships matter. The analogy explains label invariance. Its limit is that large tables create many more pairs than small tables, so the score's implicit weighting differs from giving every table or every guest equal importance.

Worked example

Consider six items A, B, C, D, E, and F. The reference clusters are {A, B, C} and {D, E, F}. There are three within-cluster pairs in each reference group, for six positive reference pairs in total. The predicted clusters are {A, B, C, D} and {E, F}. The first predicted group contributes six pairs and the second contributes one, for seven predicted positive pairs.
Four pairs are together in both partitions: AB, AC, BC, and EF. Thus TP = 4. The predicted group wrongly joins D with A, B, and C, so FP = 3. The reference pairs DE and DF are split by the prediction, so FN = 2. Pairwise precision is 4/7 and recall is 4/6. Their product is 16/42, giving a Fowlkes-Mallows score of 4 divided by the square root of 42, approximately 0.6172. Renaming either partition's clusters does not change any pair relationship or the score. The value is not the percentage of individually correct item labels.

Illustrative case study

A fictional document-organizing team compares a new clustering system with an editorial grouping. Its Fowlkes-Mallows score improves, but most of the gain comes from a large, common topic. The team inspects smaller topics and discovers that several useful distinctions have been merged. It reports the pairwise score alongside cluster-level examples and sizes. The reference grouping is also reviewed because some documents legitimately span topics, while the evaluation assumes a single partition. The metric remains informative without being treated as a complete measure of editorial usefulness.

Common misconceptions and mistakes

Fowlkes-Mallows uses a geometric mean, unlike F1's harmonic mean. Its TP, FP, and FN refer to pairs of observations, not individual classification decisions. It is not generally adjusted for chance, so a random-partition baseline should not be assumed to equal zero in every setting. True-negative pairs do not appear in the formula. A high score can therefore hide particular small-cluster failures, and a reference partition may not represent the only meaningful way to group the data.

Business use

Use the score when pairwise co-membership is a useful definition of clustering agreement. Keep the evaluated observations identical across partitions and document the reference's meaning. Report cluster counts, size distribution, and any noise-label policy. If the application values rare groups or permits overlapping membership, check whether the partition-based metric fits that requirement. Complement the score with adjusted agreement measures, stability analysis, and concrete error inspection. Treat a comparison between two algorithms as similarity evidence unless one partition has an independently justified reference role.

People also ask

Does swapping cluster label numbers change the score?

No. Pair relationships remain identical.

Is a score of one always possible?

Identical nondegenerate partitions score one, but partitions with no positive pairs require a stated implementation convention.

Does the score tell which clustering is better without a reference?

It tells how similarly two partitions group items. Their practical quality requires additional evidence.

Check your understanding

Why is the pair AD a false positive in the worked example?

Answer: The prediction puts A and D together, but the reference places them in different clusters. False positive here describes a predicted co-membership relationship, not a wrong class name for one item.

Related terms

• Partition
• Hierarchical clustering
• Cluster agreement
• pairwise precision
• pairwise recall
• geometric mean
• adjusted Rand index
• clustering reference
• co-membership
• label invariance

Learn next

List all unordered pairs for a six-item dataset and mark whether each pair is together in two partitions. Calculate pairwise precision, recall, and their geometric mean. Then compare the result with adjusted Rand index and inspect how a large cluster influences pair counts, so the weighting implicit in the metric becomes visible.

• Rand index

More in Causal inference and experimentation

• F 1 - F1, sometimes written F 1 or F1 score, combines precision and recall through their harmonic mean.
• False negative rate - False negative rate is the proportion of actual positive cases that a binary classifier incorrectly labels negative.
• False positive rate - False positive rate is the proportion of actual negative cases that a binary classifier incorrectly labels positive.
• Gini impurity - Gini impurity measures how mixed the class labels are within a group of observations, commonly a node in a classification tree.
• Granger causality - Granger causality is a predictive relationship between time series: the past of one series helps predict another beyond the information already included in...
• Hamming - Hamming is a name attached to several distinct concepts, including Hamming distance, Hamming loss, Hamming codes, and the Hamming window.

Sources

Scikit-learn, fowlkes_mallows_score: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.fowlkes_mallows_score.html Scikit-learn, clustering evaluation guide: https://scikit-learn.org/stable/modules/clustering.html Zaki and Meira, original Data Mining and Machine Learning teaching material on clustering evaluation: https://www.cs.rpi.edu/~zaki/DMML/slides/pdf/ychap17.pdf

Previous: Fourth Normal Form Next: Fractal compression