ML Percentile

What Is a Percentile?

A percentile indicates the relative standing of a value within a dataset. The 75th percentile, for example, is the value below which 75% of the observations fall. Percentiles are widely used to describe test scores, growth charts, salaries, and response times, because they give context that a single average cannot.

Computing Percentiles with NumPy

NumPy's percentile() function takes a dataset and a percentage from 0 to 100 and returns the corresponding value. Internally it sorts the data and interpolates between the two closest ranked values when the exact percentile does not land exactly on a data point.

import numpy as np

ages = [5, 31, 43, 48, 50, 41, 7, 11, 15, 39, 80, 82, 32, 2, 8, 6, 25, 36, 27, 61]

p25 = np.percentile(ages, 25)
p50 = np.percentile(ages, 50)
p75 = np.percentile(ages, 75)

print('25th percentile:', p25)
print('50th percentile (median):', p50)
print('75th percentile:', p75)

Percentiles and Quartiles

The 25th, 50th, and 75th percentiles have special names: the first quartile (Q1), the median (Q2), and the third quartile (Q3). The gap between Q1 and Q3 is called the interquartile range (IQR), and it is commonly used to detect outliers - any value more than 1.5 times the IQR below Q1 or above Q3 is often flagged as an outlier.

import numpy as np

data = [12, 14, 13, 15, 100, 14, 13, 16, 12, 15]

q1 = np.percentile(data, 25)
q3 = np.percentile(data, 75)
iqr = q3 - q1

lower_bound = q1 - 1.5 * iqr
upper_bound = q3 + 1.5 * iqr

outliers = [x for x in data if x < lower_bound or x > upper_bound]
print('IQR:', iqr)
print('Outliers detected:', outliers)

Percentile Rank vs Percentile Value

It is easy to confuse two related ideas. A percentile value (what np.percentile returns) answers 'what number marks this percentage of the data?' A percentile rank answers the reverse question: 'what percentage of the data falls below this specific number?' You can compute a percentile rank directly by comparing a value against the sorted array.

import numpy as np

response_times_ms = [120, 135, 98, 250, 110, 105, 400, 130, 115, 108]

value = 130
rank = (np.sum(np.array(response_times_ms) < value) / len(response_times_ms)) * 100

print('Percentile rank of 130ms:', rank)

• Standardized test scores are usually reported as percentiles, not raw scores
• Website performance is monitored using the 95th or 99th percentile response time, not the average
• Growth charts for children use percentiles to compare a child's height or weight to peers
• Outlier detection with the IQR rule is a standard data-cleaning step before model training