ML Multiple Regression
Multiple regression extends linear regression to two or more input features at once, letting scikit-learn's LinearRegression model predict an outcome from several variables together.
From One Feature to Many
Simple linear regression predicts y from a single x. Real problems are rarely that simple - a car's CO2 emissions depend on both engine size and weight; a house's price depends on square footage, location, and age. Multiple regression fits one equation that combines several input features, each with its own coefficient, to predict a single output.
Multiple Regression with scikit-learn
scikit-learn's LinearRegression class handles multiple features naturally. You pass it a 2D array where each column is a feature (and each row is one observation), plus a 1D array of target values, and call .fit(). The result exposes .coef_ (one coefficient per feature) and .intercept_.
Example
import pandas as pd
from sklearn import linear_model
data = {
"Weight": [790, 1160, 929, 865, 1140, 929, 1109, 1365, 1112, 1150],
"Volume": [1000, 1200, 1000, 900, 1500, 1000, 1400, 1500, 1500, 1600],
"CO2": [99, 95, 95, 90, 105, 105, 90, 92, 98, 99]
}
df = pd.DataFrame(data)
X = df[["Weight", "Volume"]]
y = df["CO2"]
model = linear_model.LinearRegression()
model.fit(X, y)
print("Coefficients:", model.coef_)
print("Intercept:", model.intercept_)Making Predictions from Multiple Features
Once the model is fit, predicting a new outcome means passing a new row (or rows) with the same features, in the same order, to .predict(). scikit-learn returns the predicted target for each row.
Example
import pandas as pd
from sklearn import linear_model
data = {
"Weight": [790, 1160, 929, 865, 1140, 929, 1109, 1365, 1112, 1150],
"Volume": [1000, 1200, 1000, 900, 1500, 1000, 1400, 1500, 1500, 1600],
"CO2": [99, 95, 95, 90, 105, 105, 90, 92, 98, 99]
}
df = pd.DataFrame(data)
X = df[["Weight", "Volume"]]
y = df["CO2"]
model = linear_model.LinearRegression()
model.fit(X, y)
# Predict CO2 for a car weighing 2300kg with a 1300cm3 engine
predicted_co2 = model.predict([[2300, 1300]])
print("Predicted CO2:", predicted_co2)- Features must be passed as a 2D array/DataFrame (rows = observations, columns = features), even for a single prediction.
- model.coef_ returns one coefficient per input feature, in the same order the features were given.
- model.intercept_ is the predicted value when every feature is 0.
- Adding more features can improve fit, but irrelevant features add noise instead of signal.
- Features on very different scales (e.g. weight in kg vs. income in dollars) can be standardized first for more stable, comparable coefficients.
Example
import pandas as pd
from sklearn import linear_model
data = {
"SquareFeet": [850, 900, 1200, 1500, 1800, 2100, 2400, 2800],
"Bedrooms": [1, 2, 2, 3, 3, 4, 4, 5],
"Price": [120000, 135000, 175000, 210000, 240000, 275000, 300000, 340000]
}
df = pd.DataFrame(data)
X = df[["SquareFeet", "Bedrooms"]]
y = df["Price"]
model = linear_model.LinearRegression()
model.fit(X, y)
new_house = [[2000, 3]]
print("Predicted price:", model.predict(new_house))
print("Coefficients:", model.coef_)Exercise: ML Multiple Regression
What distinguishes multiple regression from simple linear regression?
