ML Hierarchical Clustering

What Is Hierarchical Clustering?

Hierarchical clustering groups data points into nested clusters rather than a single flat partition. Instead of committing to a fixed number of clusters up front the way K-Means does, it builds an entire tree of merges — from every point in its own tiny cluster all the way up to one giant cluster containing everything — and lets you decide afterward how many groups makes sense by cutting that tree at a chosen height.

The most common approach, agglomerative clustering, works bottom-up: it starts with every sample as its own single-point cluster, then repeatedly finds the two closest clusters and merges them into one. This merge-the-closest-pair step repeats until only one cluster remains, and the full sequence of merges is what gets drawn as a dendrogram.

• Single linkage merges clusters based on the distance between their closest pair of points — tends to produce long, chained clusters.
• Complete linkage merges based on the distance between their farthest pair of points — tends to produce compact, evenly sized clusters.
• Average linkage merges based on the average distance between all pairs across the two clusters — a middle ground between single and complete.
• Ward linkage merges the pair that increases total within-cluster variance the least — usually the best default for roughly spherical clusters.

Agglomerative Clustering in scikit-learn

from sklearn.datasets import make_blobs
from sklearn.cluster import AgglomerativeClustering

X, _ = make_blobs(n_samples=150, centers=3, cluster_std=0.8, random_state=1)

model = AgglomerativeClustering(n_clusters=3, linkage="ward")
labels = model.fit_predict(X)

print("Cluster labels for first 10 points:", labels[:10])
print("Points per cluster:", [sum(labels == i) for i in range(3)])

Reading a Dendrogram

A dendrogram plots every merge made during agglomerative clustering: the x-axis lists the individual samples (or clusters), and the y-axis shows the distance at which two clusters were joined. Drawing a horizontal line across the dendrogram at a chosen height and counting how many vertical lines it crosses tells you how many clusters you'd get by cutting the hierarchy at that point — a low cut gives many small clusters, a high cut gives fewer, larger ones.

from scipy.cluster.hierarchy import linkage, dendrogram
import matplotlib.pyplot as plt

merge_matrix = linkage(X, method="ward")

dendrogram(merge_matrix, truncate_mode="lastp", p=12)
plt.xlabel("Cluster size")
plt.ylabel("Merge distance")
plt.title("Dendrogram (last 12 merges)")
plt.show()

from scipy.cluster.hierarchy import fcluster

# Cut the tree at a fixed distance instead of a fixed number of clusters
flat_labels = fcluster(merge_matrix, t=6, criterion="distance")
print("Number of clusters at distance 6:", len(set(flat_labels)))

# The scikit-learn equivalent: let distance_threshold pick the cluster count
auto_model = AgglomerativeClustering(n_clusters=None, distance_threshold=6, linkage="ward")
auto_labels = auto_model.fit_predict(X)
print("Number of clusters found:", auto_model.n_clusters_)