ML Linear Regression

What Is Linear Regression?

Linear regression finds the straight line that best fits a set of (x, y) points, minimizing the overall distance between the line and every point. That line is defined by two numbers: the slope (how steep it is) and the intercept (where it crosses the y-axis). Once you have those two numbers, you can predict a y value for any new x.

Using scipy.stats.linregress()

SciPy's linregress() function takes two arrays of equal length and returns five values: slope, intercept, the correlation coefficient r, a p-value, and the standard error. The slope and intercept are exactly what you need to build a predictive line; r tells you how well that line actually fits the data.

from scipy import stats

x = [5, 7, 8, 7, 2, 17, 2, 9, 4, 11, 12, 9, 6]
y = [99, 86, 87, 88, 111, 86, 103, 87, 94, 78, 77, 85, 86]

slope, intercept, r, p, std_err = stats.linregress(x, y)

print("Slope:", slope)
print("Intercept:", intercept)
print("r (correlation):", r)

Drawing the Regression Line

To draw the fitted line, apply the slope and intercept to every x value using the equation y = slope * x + intercept, then plot that alongside the original scatter points. This visually confirms whether the line tracks the data well.

import matplotlib.pyplot as plt
from scipy import stats

x = [5, 7, 8, 7, 2, 17, 2, 9, 4, 11, 12, 9, 6]
y = [99, 86, 87, 88, 111, 86, 103, 87, 94, 78, 77, 85, 86]

slope, intercept, r, p, std_err = stats.linregress(x, y)

def predict(x_value):
    return slope * x_value + intercept

fitted_line = list(map(predict, x))

plt.scatter(x, y)
plt.plot(x, fitted_line, color="red")
plt.title("Linear Regression Fit")
plt.show()

• slope: the change in y for every one-unit increase in x.
• intercept: the predicted y value when x is 0.
• r (correlation coefficient): how tightly the points hug the line, from -1 to 1.
• p-value: whether the observed relationship is likely to be due to chance.
• Once slope and intercept are known, prediction is just plugging in a new x.

from scipy import stats

# Predict exam score from hours studied
hours = [1, 2, 3, 4, 5, 6, 7, 8]
scores = [50, 55, 63, 68, 73, 79, 84, 90]

slope, intercept, r, p, std_err = stats.linregress(hours, scores)

def predict_score(hours_studied):
    return slope * hours_studied + intercept

print("Predicted score for 9.5 hours:", predict_score(9.5))
print("r-squared:", r ** 2)