ML K-means

Clustering: Learning Without Labels

Unlike logistic regression or decision trees, k-means is an unsupervised algorithm, it is given only feature values, with no target column to predict. Its goal is to discover groups of similar observations on its own, which is useful for tasks like customer segmentation, grouping similar images, or finding natural categories in sensor data.

How the K-means Algorithm Works

import numpy as np
from sklearn.cluster import KMeans

X = np.array([
    [1, 2], [1.5, 1.8], [1, 0.6],
    [9, 11], [8, 9], [9.5, 10],
    [4, 4.5], [3.5, 5], [4.5, 4]
])

kmeans = KMeans(n_clusters=3, random_state=42, n_init=10)
kmeans.fit(X)

print('Cluster assignments:', kmeans.labels_)
print('Cluster centers:')
print(kmeans.cluster_centers_)

Choosing k With the Elbow Method

K-means requires you to decide k, the number of clusters, in advance, it will not figure this out for you. The elbow method helps: fit k-means for a range of k values, record the inertia, the sum of squared distances from each point to its assigned cluster center, for each, and plot inertia against k. Inertia always decreases as k grows, but it drops sharply at first and then flattens out; the elbow where the curve bends is a good candidate for k, because adding more clusters past that point buys little improvement for a lot of added complexity.

inertias = []
k_values = range(1, 8)

for k in k_values:
    model = KMeans(n_clusters=k, random_state=42, n_init=10)
    model.fit(X)
    inertias.append(model.inertia_)

for k, inertia in zip(k_values, inertias):
    print(f'k={k}: inertia={inertia:.2f}')

# import matplotlib.pyplot as plt
# plt.plot(k_values, inertias, marker='o')
# plt.xlabel('Number of clusters (k)')
# plt.ylabel('Inertia')
# plt.show()

• Centroid: the mean position of all points currently assigned to a cluster; k-means alternates between assigning points and recomputing centroids.
• Inertia: the sum of squared distances from each point to its cluster's centroid, lower means a tighter cluster.
• n_init: how many times k-means restarts with different random centroid initializations, keeping the best result, this avoids getting stuck in a poor local minimum.
• random_state: fixes the random initialization so results are reproducible across runs.
• Silhouette score is an alternative to the elbow method for choosing k, measuring how similar a point is to its own cluster versus the next closest one.

import pandas as pd

final_model = KMeans(n_clusters=3, random_state=42, n_init=10)
labels = final_model.fit_predict(X)

df = pd.DataFrame(X, columns=['feature_1', 'feature_2'])
df['cluster'] = labels

print(df)
print('Average feature values per cluster:')
print(df.groupby('cluster').mean())