Data Science Hierarchical Clustering
Hierarchical clustering builds a tree of nested groupings by repeatedly merging the closest clusters, letting you choose the number of clusters after seeing the full structure.
Grouping Data Without Labels
Clustering is an unsupervised technique: there is no target column telling the algorithm the right answer. Instead, the algorithm groups examples together based purely on how similar their features are, with the goal that points inside a cluster are more alike than points in different clusters.
How Agglomerative Clustering Builds a Hierarchy
The most common form of hierarchical clustering is agglomerative (bottom-up): it starts by treating every single data point as its own cluster. At each step, it finds the two closest clusters and merges them into one. This repeats, cluster count shrinking by one each time, until every point has been merged into a single all-encompassing cluster.
- Single linkage: distance between two clusters is the distance between their closest pair of points; tends to produce long, chained clusters.
- Complete linkage: distance is the distance between their farthest pair of points; tends to produce compact, evenly sized clusters.
- Average linkage: distance is the average distance across all pairs of points between the two clusters.
- Ward linkage: merges the pair of clusters that increases total within-cluster variance the least; often the default choice.
The record of every merge, and the distance at which it happened, can be drawn as a dendrogram: a tree diagram where the leaves are individual points, branches show merges, and the height of each branch represents how far apart (dissimilar) the two merging clusters were. Reading a dendrogram from the bottom up shows you the entire merge history at a glance.
Example
from sklearn.cluster import AgglomerativeClustering
import numpy as np
X = np.array([
[1, 2], [1.5, 1.8], [1.2, 2.3],
[8, 8], [8.5, 7.8], [9, 8.2],
])
model = AgglomerativeClustering(n_clusters=2, linkage='ward')
labels = model.fit_predict(X)
print('Cluster labels:', labels)The trade-off is computational cost: agglomerative clustering typically requires computing and updating a full pairwise distance matrix, which scales roughly quadratically with the number of points. That makes it comfortable for datasets of a few thousand rows but impractical for millions, where an algorithm like k-means scales much better.
- Pros: no need to pre-specify k, produces an interpretable dendrogram, works with any distance metric and several linkage strategies.
- Cons: computationally expensive on large datasets, and once two points are merged the merge cannot be undone.
