AI Glossary / Causal inference and experimentation / Intermediate
Silhouette Coefficient
Definition
The Silhouette Coefficient measures how well an observation fits its assigned cluster compared with the nearest competing cluster. It contrasts average distances within and between clusters, producing a value between −1 and 1. An overall silhouette score usually averages these observation-level values.
What it means
Clustering creates groups without necessarily having known correct labels. Silhouette analysis provides an internal check using the chosen representation, distances and cluster assignments. It asks whether points are relatively close to their own group and farther from alternatives.
A value near one indicates strong relative separation. A value near zero suggests little distance-based distinction between the assigned group and a competitor. A negative value indicates that another cluster is closer on average.
These are geometric findings, not probabilities that an assignment is correct. A well-separated grouping can still be unhelpful for a business question if the features or distance measure emphasize the wrong characteristics.
How it works
For observation i, calculate a(i), its average distance to the other observations in its assigned cluster. Exclude the point itself from this average. Next calculate its average distance to each other cluster and let b(i) be the smallest of those averages.
The coefficient is s(i) = [b(i) − a(i)] / max[a(i), b(i)]. The competing cluster is selected by average distance, not by the single nearest outside point or necessarily the nearest centroid.
A silhouette plot displays individual coefficients grouped by cluster, exposing weak or negative regions hidden by a mean. The usual calculation requires at least two clusters and fewer clusters than observations. Software conventions handle special cases such as singleton clusters, so check implementation details when interpreting unusual partitions.
Three practical examples
- An analyst compares several candidate cluster counts while keeping the same features, observations and distance measure.
- A support team examines low-silhouette tickets to understand where proposed issue groups overlap.
- A researcher compares clustering results across repeated samples and uses silhouette plots alongside stability checks and domain interpretation.
Think of it this way
Think of assigning people to tables by how similar their interests are. For one person, compare their average difference from people at their table with their average difference from each other table. The closest alternative table provides the comparison.
The analogy highlights group-average fit. It does not imply that real people should be grouped by a numerical score or that interests have one natural distance. The quality of the measurement depends on what similarity was chosen to represent.
Worked example
Calculating one point's silhouette
In an invented distance table, point P has distances 2 and 4 to the two other members of its assigned cluster. Its within-cluster average is a = (2 + 4)/2 = 3.
Its distances to a second cluster's members are 5, 6 and 7, giving average 6. Its distances to a third cluster's members are 4 and 12, giving average 8. Therefore b = min(6,8) = 6.
The silhouette is (6 − 3)/6 = 0.5. Notice that the third cluster contains an outside point only distance 4 away, yet the second cluster is the nearest competitor by average distance.
The result describes P, not the entire clustering. If another point has a = 7 and b = 4, its coefficient is −3/7, approximately −0.429, suggesting its assignment deserves inspection under this distance definition.
Illustrative case study
Fictional Fern Services clusters support requests using document embeddings. A two-cluster solution has the highest average silhouette, but one cluster combines several operationally different issues that happen to share vocabulary.
The team inspects per-cluster plots and representative requests. It compares the score with the intended use: routing issues to different specialist teams. It also checks whether boilerplate text is dominating the representation.
Rather than declare two to be the objectively correct number of topics, the analysts treat the silhouette result as evidence about geometric separation. They revise preprocessing and assess whether the resulting groups are stable and useful for routing. The case shows why internal validity and practical usefulness need separate attention.
Common misconceptions and mistakes
A silhouette of 0.8 is not 80% classification accuracy. No external correct labels are required for the calculation.
The score tends to favor compact, well-separated groups and can be less suitable for curved or density-connected clusters. A low value does not automatically invalidate a grouping based on another meaningful notion of structure.
Do not compare scores across changed feature scales or distance definitions as though the ruler stayed fixed. A large, well-separated cluster can also dominate the overall mean, concealing a small cluster with poor assignments. Inspect the distribution and cluster sizes.
Business use
Silhouette analysis is useful when exploring customer, document or product groupings, provided the representation reflects the intended question. Document the distance measure, preprocessing, cluster labels and any sampled subset used to estimate the score.
Pair it with domain review, stability across reasonable data changes and assessment of the action the clusters will support. For large datasets, full pairwise distance work can be expensive; sampling can help but introduces additional variability and can omit rare groups. Report that approximation rather than presenting it as an exact full-data result.
People also ask
Does the calculation need correct class labels?
No. It uses the proposed cluster assignments and distances, making it an internal clustering measure.
Is b the distance to the nearest point?
No. It is the smallest average distance to all members of another cluster.
Should I always choose the cluster count with the highest mean silhouette?
No. That is one selection signal. The desired structure, stability, small-group behavior and practical use also matter.
Check your understanding
Point Q has average distance 2 to its own cluster and average distances 3 and 9 to the other clusters. What is its silhouette?
Answer: The nearest competing average is b = 3, so s = (3 − 2)/max(2,3) = 1/3, approximately 0.333. Averaging 3 and 9 together would use the wrong comparison.
Related terms
- Clustering
- internal validation
- distance measure
- cluster cohesion
- cluster separation
- silhouette plot
- Davies–Bouldin index
- clustering stability
Learn next
Learn pairwise distances and the difference between a cluster centroid and its members. Hand-calculate a few silhouettes, then inspect a plot containing both positive and negative values. Compare the same data under different feature scales and explain why the resulting scores answer different geometric questions.
More in Causal inference and experimentation
- ROUGE-S - ROUGE-S is a ROUGE evaluation measure based on matching skip-bigrams between a candidate text and a reference.
- Sequential exchangeability - Sequential exchangeability is a causal assumption that, at each treatment decision, treatment is independent of the relevant future potential outcomes...
- Side-by-side evaluation - Side-by-side evaluation compares two systems by judging their outputs for the same input.
- Similarity measure - A similarity measure is a rule that assigns a numerical score to how alike two objects are in a chosen representation.
- Squared hinge loss - Squared hinge loss is a classification loss that squares the positive shortfall from a target margin.
- Structural causal model - A structural causal model, or SCM, represents how variables are generated by causal mechanisms.
Sources
Sources: Rousseeuw, Silhouettes: a graphical aid to the interpretation and validation of cluster analysis: https://doi.org/10.1016/0377-0427(87)90125-7 Scikit-learn, Silhouette Coefficient: https://scikit-learn.org/stable/modules/clustering.html#silhouette-coefficient Scikit-learn, silhouette_score: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.silhouette_score.html Scikit-learn, silhouette analysis example: https://scikit-learn.org/stable/auto_examples/cluster/plot_kmeans_silhouette_analysis.html
Previous: Silence compression Next: Simalto