ML Data Distribution
Real machine learning projects need much larger datasets than a handful of numbers - this chapter covers generating and visualizing large random datasets.
#Jobseeker
Find matching jobs & Auto apply with AI
Why Data Distribution Matters
Machine learning models learn general patterns, which requires seeing many examples, not just a handful. Understanding how your data is distributed - whether it is spread evenly, clustered around a center, or skewed to one side - helps you choose the right model and catch problems, like insufficient data or bad sampling, before training even begins.
Generating a Large Random Dataset
NumPy's random module can generate thousands of data points instantly, which is useful for experimenting with statistics and distributions before you have a real-world dataset. numpy.random.uniform() draws values evenly across a range, so every value in that range is equally likely to appear.
Example
import numpy as np
np.random.seed(42)
data = np.random.uniform(0.0, 10.0, 10000)
print('Number of points:', len(data))
print('Mean:', np.mean(data))
print('Std deviation:', np.std(data))
print('Min:', data.min(), 'Max:', data.max())The Normal (Gaussian) Distribution
Many real-world measurements, such as heights, measurement errors, and test scores, roughly follow a normal distribution: a symmetric bell-shaped curve where values cluster around the mean and become rarer the farther they are from it. numpy.random.normal() generates data following this pattern, controlled by a mean (loc) and standard deviation (scale).
Example
import numpy as np
np.random.seed(42)
heights_cm = np.random.normal(loc=170, scale=8, size=10000)
print('Mean height:', np.mean(heights_cm))
print('Std deviation:', np.std(heights_cm))
within_one_std = np.mean(np.abs(heights_cm - np.mean(heights_cm)) <= np.std(heights_cm)) * 100
print('Approx percent of values within one std:', within_one_std)Visualizing Distributions with Histograms
A histogram divides a dataset into equal-width bins and counts how many values fall into each bin, turning thousands of raw numbers into a shape you can interpret at a glance. Matplotlib's plt.hist() is the standard way to draw one, and choosing the right number of bins matters: too few bins hide detail, too many make the shape noisy.
Example
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(42)
exam_scores = np.random.normal(loc=75, scale=10, size=10000)
plt.hist(exam_scores, bins=50)
plt.title('Distribution of Exam Scores')
plt.xlabel('Score')
plt.ylabel('Frequency')
plt.show()- Set a random seed (np.random.seed(42)) so your random dataset is reproducible
- Larger sample sizes make histograms look smoother and closer to the true distribution
- Skewed distributions with a long tail on one side often need the median rather than mean
- Always plot your data before training a model - many issues are visible in a histogram first
Exercise: ML Data Distribution
Why do ML tutorials often use large random datasets to demonstrate data distribution?
