ML Scale
Learn why raw feature values on different scales can mislead a model, and how StandardScaler fixes that.
Why Feature Scaling Matters
Real-world datasets often mix features measured on wildly different scales — a person's age (0-100) next to their annual salary (20,000-200,000). Many machine learning algorithms compute distances or gradients directly on these raw numbers, so a feature with a larger numeric range can dominate the calculation even if it isn't actually more important.
- Distance-based models (KNN, K-Means, SVM) treat every unit the same, so large-range features silently outweigh small-range ones.
- Gradient descent based models (logistic regression, neural networks) converge faster and more reliably when inputs share a similar scale.
- Regularized models (Ridge, Lasso) penalize coefficients by size, which only makes sense if features are on comparable scales.
- Tree-based models (Decision Trees, Random Forests, Gradient Boosting) split on per-feature thresholds, so they are unaffected by scaling.
The StandardScaler Formula
StandardScaler applies z-score standardization to each feature independently: z = (x - mean) / std. After transforming, every feature has a mean of 0 and a standard deviation of 1, so no single feature's raw magnitude can dominate just because of its units.
Example
import numpy as np
from sklearn.preprocessing import StandardScaler
# Two features on very different scales
X = np.array([
[25, 50000],
[32, 64000],
[47, 120000],
[51, 98000],
[23, 41000]
])
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
print("Mean before:", X.mean(axis=0))
print("Mean after: ", X_scaled.mean(axis=0).round(2))
print("Std after: ", X_scaled.std(axis=0).round(2))StandardScaler vs MinMaxScaler
Example
from sklearn.preprocessing import MinMaxScaler
mm_scaler = MinMaxScaler()
X_minmax = mm_scaler.fit_transform(X)
print("Min after:", X_minmax.min(axis=0))
print("Max after:", X_minmax.max(axis=0))Example
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import accuracy_score
X_data, y_data = make_classification(n_samples=300, n_features=2, n_redundant=0, random_state=1)
X_data[:, 1] *= 1000 # blow up the scale of the second feature
X_train, X_test, y_train, y_test = train_test_split(X_data, y_data, test_size=0.3, random_state=1)
# Without scaling
knn_raw = KNeighborsClassifier().fit(X_train, y_train)
print("Unscaled accuracy:", accuracy_score(y_test, knn_raw.predict(X_test)))
# With scaling
scaler = StandardScaler().fit(X_train)
knn_scaled = KNeighborsClassifier().fit(scaler.transform(X_train), y_train)
print("Scaled accuracy: ", accuracy_score(y_test, knn_scaled.predict(scaler.transform(X_test))))Exercise: ML Scale
Why is feature scaling often needed before training certain ML models?
