ML Standard Deviation

Measuring Spread in Data

Knowing the mean of a dataset only tells half the story. Two datasets can share the exact same mean while looking completely different - one tightly clustered, the other widely scattered. Standard deviation and variance quantify that spread, which matters in machine learning because many algorithms assume features are on a similar scale.

Variance

Variance is the average of the squared differences between each value and the mean. Squaring the differences ensures negative and positive deviations do not cancel each other out, but it also means variance is expressed in squared units (for example, squared dollars), which makes it hard to interpret directly.

import numpy as np
import statistics

speeds = [86, 87, 88, 86, 87, 85, 86]

print('Variance (numpy, population):', np.var(speeds))
print('Variance (statistics, sample):', statistics.variance(speeds))
print('Population variance (statistics):', statistics.pvariance(speeds))

Standard Deviation

Standard deviation is the square root of the variance, which brings the measure back into the original units of the data. A small standard deviation means values cluster tightly around the mean; a large standard deviation means values are spread widely. In a roughly normal (bell-curve) distribution, about 68% of values fall within one standard deviation of the mean.

import numpy as np
import statistics

class_a = [80, 82, 78, 81, 79]    # tightly clustered scores
class_b = [60, 100, 70, 90, 80]  # same mean, much more spread

print('Class A mean:', np.mean(class_a), 'std:', np.std(class_a))
print('Class B mean:', np.mean(class_b), 'std:', np.std(class_b))
print('Class A std (statistics.stdev):', statistics.stdev(class_a))

Population vs Sample

NumPy's np.var() and np.std() compute the population version by default (dividing by n), while Python's statistics.variance() and statistics.stdev() compute the sample version by default (dividing by n-1), which corrects for bias when your data is a sample from a larger population. Pass ddof=1 to NumPy's functions to match the sample calculation.

import numpy as np

sample = [4, 8, 6, 5, 3, 7]

population_style = np.std(sample)        # divides by n
sample_style = np.std(sample, ddof=1)    # divides by n - 1

print('Population-style std (ddof=0):', population_style)
print('Sample-style std (ddof=1):', sample_style)

• Standard deviation shares the same units as the original data; variance does not
• A standard deviation near zero means the data barely varies
• Always check whether you have a full population or a sample before choosing ddof
• Standard deviation is the backbone of feature scaling techniques like z-score normalization