AI Glossary / Causal inference and experimentation / Intermediate

Calinski-Harabasz Index

Definition

The Calinski-Harabasz Index is an internal clustering score that compares separation between cluster centers with variation inside clusters, adjusted for the number of clusters and observations. It is also called the variance ratio criterion. Larger values generally favor compact, well-separated groups under its centroid-based geometry. The index requires data and assigned cluster labels, but no external class labels. It does not prove that clusters represent meaningful real-world categories.

What it means

Clustering divides observations into groups without necessarily knowing the right answer in advance. The Calinski-Harabasz Index asks whether each group's points stay relatively close to their own center while group centers remain far from the overall center. This is a particular definition of a good partition. Curved, connected, or density-based structures may be meaningful even when they are not well summarized by compact clouds around separate centroids.

How it works

For n observations assigned to k clusters, compute each cluster mean and the overall mean. Within-cluster dispersion W is the sum of squared Euclidean distances from observations to their own cluster mean. Between-cluster dispersion B sums each cluster size multiplied by its center's squared distance from the overall mean. The score is [B/(k - 1)] divided by [W/(n - k)]. Ordinary use requires more than one cluster and fewer clusters than observations. Degenerate zero-dispersion cases need an explicit handling rule rather than an unexamined interpretation.

Three practical examples

• Segmentation exploration: compare several cluster counts on the same standardized customer-behavior features.
• Sensor analysis: assess whether groups of operating measurements form compact, separated regions in a chosen feature space.
• Pipeline comparison: examine how an alternative representation changes apparent clustering, while recognizing that scores across changed geometries are not directly interchangeable evidence.

Think of it this way

Think of seating people around several tables: a useful arrangement under this criterion keeps each table's group close together while spacing tables apart. The analogy helps explain the two dispersions. Its limit is that closeness depends on the chosen measurement, and people arranged in a meaningful long line or ring would not fit the table-shaped assumption well.

Worked example

Take four one-dimensional observations: 0, 1, 5, and 6. Partition them into clusters {0, 1} and {5, 6}. Their centers are 0.5 and 5.5, and the overall mean is 3. Each point is 0.5 from its cluster center, so W = 4 times 0.5 squared = 1. Between-cluster dispersion is B = 2 times (0.5 - 3) squared + 2 times (5.5 - 3) squared = 25.
With n = 4 and k = 2, the index is [25/(2 - 1)]/[1/(4 - 2)] = 25/0.5 = 50. Now use clusters {0, 5} and {1, 6}. Their centers are 2.5 and 3.5. W becomes 25 and B becomes 1, giving 1/12.5 = 0.08. The first partition is much better under this compactness-and-separation criterion. The value 50 is not 50% accuracy, and there is no universal cutoff at which it proves the groups are operationally useful.

Illustrative case study

A fictional equipment team clusters measurements from machines in several operating conditions. One feature is recorded in a much larger numerical scale than the others, so it dominates squared distances and produces a high-scoring partition mainly reflecting that feature. The team reviews units, chooses a justified scaling procedure, and repeats the analysis. It then checks stability and whether the groups correspond to interpretable operating patterns. The original score was mathematically valid for its input geometry, but it did not answer the team's intended question.

Common misconceptions and mistakes

A higher score does not establish the true number of clusters. The criterion favors particular shapes and can be affected by outliers and feature scaling. It is not a classification metric and needs no known correct labels. Scores should not be casually compared across unrelated datasets or representations. Increasing the number of clusters changes both dispersion and normalization; the score is not merely the inverse of within-cluster error. Clusters can be statistically separated yet too small, unstable, or impractical for a business workflow.

Business use

Use the index to support exploration among plausible partitions of the same prepared data. Document the feature representation, scaling, distance interpretation, cluster counts, and treatment of noise points. Inspect group sizes, representative examples, and stability across reasonable data perturbations. Compare with other diagnostics whose assumptions fit the data. If selecting a partition for downstream action, evaluate that action directly rather than using internal geometry as a substitute for usefulness, reliability, or a defensible category definition.

People also ask

Is the score bounded between zero and one?

No. It can take much larger values, so it should not be read as a probability.

Can it assess density-based clusters?

It can sometimes be computed from their labels, but its centroid-based preference may not match the clustering objective, and noise handling matters.

Does uniformly multiplying every feature by the same constant change it?

For a nondegenerate fixed partition, both dispersions scale by that constant squared, so their ratio stays unchanged.

Check your understanding

Why does the first partition in the worked example score higher?

Answer: Its points have small within-cluster squared distances and its centers are far from the overall mean. The alternative mixes distant points inside each group while placing the group centers near each other.

Related terms

• Centroid
• k-means
• Davies-Bouldin Index
• Cluster validation
• within-cluster dispersion
• between-cluster dispersion
• feature scaling
• silhouette score
• cluster stability
• internal evaluation

Learn next

Calculate the two dispersion terms for a tiny dataset and compare two partitions. Then vary one feature's scale to observe how the geometry changes. Study silhouette analysis and stability checks next, especially if your groups are irregularly shaped or your purpose requires more than compactness around a mean.

More in Causal inference and experimentation

• BLEU (Bilingual Evaluation Understudy) - BLEU, short for Bilingual Evaluation Understudy, is an automatic metric originally designed to compare machine translations with human reference translations.
• Box-Benkhen experiment - A Box - Behnken experiment uses a response-surface design to study how several controllable factors affect an outcome and to fit a quadratic model.
• Brier score - The Brier score measures the quality of probability predictions by averaging squared differences between predicted probabilities and observed outcomes.
• Causal estimand - A causal estimand is the precisely defined causal quantity that an analysis aims to learn.
• Causal graphs - Causal graphs are diagrams that represent assumptions about causal relationships among variables.
• Causal inference - Causal inference is the process of drawing conclusions about how outcomes would change under specified interventions.

Sources

Scikit-learn, calinski_harabasz_score: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.calinski_harabasz_score.html Scikit-learn, clustering guide and Calinski-Harabasz discussion: https://scikit-learn.org/stable/modules/clustering.html Scikit-learn, original implementation: https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/metrics/cluster/_unsupervised.py

Previous: Calibration layer Next: Camera path