The Silhouette Trap: A Metric That Prefers a Shape

Clustering has no labels, so quality gets measured by internal metrics — numbers computed from the data and the assignment alone. The most used is the silhouette: for each point, how much closer it is to its own cluster than to the nearest other one, averaged over the dataset.

s(i) = (b − a) / max(a, b)
       a = mean distance to points in its own cluster
       b = mean distance to points in the nearest other cluster

It is a perfectly good number, and it encodes a preference: small distances inside a cluster, large distances to the nearest other one. Nothing in the formula demands round clusters — but under Euclidean distance that preference lands on compact, well-separated shapes. Where the structure you want is not that shape, the metric does not report failure. It reports a high score for the wrong answer.

The clean demonstration is two interleaving crescents. The correct grouping is the two moons. K-means cannot represent them, so it slices the plane into two compact halves that cut across both. That partition scores a higher silhouette than the correct one — 0.49 against 0.33 — while matching the actual crescent structure far worse: ARI 0.27 against 1.00. Maximize the metric you can compute and you pick the partition you did not want.

The tempting repair is to say the bias cancels if you hold the algorithm fixed. It does not. Here is K-means on four Gaussian blobs arranged as two close pairs — a dataset whose answer is four groups — with K swept from 2 to 8:

K        2      3      4      5      6      7      8
silhouette  0.739  0.611  0.517  0.468  0.394  0.373  0.341
ARI         0.499  0.681  0.855  0.782  0.674  0.583  0.509

Silhouette is maximal at K = 2, where each pair of true clusters has been merged into one rounder, better-separated blob. ARI is maximal at K = 4, the right answer. Change K and you change the geometry of the candidates, so the preference does not cancel — it points at fewer, rounder clusters.

Nor is the problem that silhouette cannot compare algorithms: it is computed from distances and assignments, so it will happily rank K-means against HDBSCAN on the same data and metric. The problem is what it rewards, whichever produced the partition. The best-scoring partition is not the most meaningful one by default — in either comparison.

What to do instead of trusting one number. Stack three checks, cheapest first:

1. A metric, for parameter selection within one algorithm.
2. Stability, read in one direction only: re-run on bootstrap samples or different seeds and measure how much the assignment moves. An assignment that dissolves under resampling is a warning. One that holds is not a verdict — the wrong K-means partition above reproduces at ARI 0.99 across seeds and 0.95 across bootstrap resamples, while agreeing with the truth at 0.27. An algorithm can be consistently wrong.
3. Looking at it: plot the assignment where the geometry allows, and read representative members of each cluster where it does not. Note that neither is free — for the moons, reading individual points tells you nothing, and for high-dimensional embeddings a two-dimensional projection can invent separation that is not there. The question to carry is whether the groups serve the task, not whether they look separated.

And know what the alternatives assume too. Density-based methods make the opposite bet — arbitrary shapes, but a density threshold and a noise class — so silhouette is a weak diagnostic for them at best, and it is usually computed on the non-noise points only — which quietly drops the hardest examples from the evaluation and flatters the score, so such a number cannot be compared with one computed over everything. For topic models the comparable number is coherence between a topic's top words, which has its own bias: it rewards topics built from words that co-occur, whether or not the topic is useful.

The question, in the end, is not whether the metric is good. It is whether what it rewards is the structure you were trying to find. The broader version, with the metrics table and the practical recipe per algorithm, is in Clustering in Practice; the algorithms themselves and what each one can represent are in Clustering Methods.