Data Science Polynomial Regression

Polynomial regression fits a curved line through data by adding powers of x, and numpy.polyfit solves for the coefficients that best match a rise-and-fall trend a straight line can't capture.

When a Straight Line Isn't Enough

Some relationships aren't straight: a value might climb, peak, and then fall back down. Forcing a single straight line through data shaped like that produces a poor fit no matter how you tilt it - exactly the kind of shape a scatter plot check catches before you model anything.

Fitting a Curve With numpy.polyfit

numpy.polyfit(x, y, degree) finds the coefficients of a degree-N polynomial that best fits the data by least squares, returned from the highest power down to the constant term. A degree of 2 fits a parabola: a*x**2 + b*x + c.

Example

import numpy as np

minutes = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
clicks_per_minute = np.array([43, 77, 107, 133, 155, 173, 187, 197, 203, 205])

coefficients = np.polyfit(minutes, clicks_per_minute, 2)
print(coefficients)    # array([-2., 40.,  5.])

The fitted coefficients say the click rate follows -2*minutes**2 + 40*minutes + 5. The negative leading coefficient means the curve opens downward: clicks accelerate right after the campaign email goes out, then decelerate as interest fades.

Example

import numpy as np

model = np.poly1d(coefficients)

print(model(5))     # 155.0 - matches the observed value at minute 5
print(model(10))    # 205.0 - the peak, matching the observed value at minute 10
print(model(15))    # 155.0 - the model's guess past the observed window

Choosing the Right Degree

  • degree 1 is just a straight line - it's ordinary linear regression written as a polynomial
  • degree 2 adds a single bend, enough for one peak or one dip
  • degree 3 and higher can bend more than once, chasing shapes that may just be noise
  • a higher degree always fits the training points at least as well, which is not the same as fitting reality better
DegreeShape it capturesMain risk
1A straight trendUnderfits any real curve
2A single peak or dipGood match for simple rise-then-fall data
3+Multiple bendsOverfitting - modeling noise instead of the pattern
Note: model(15) sits outside minutes 1 through 10, the range the data actually covers. The parabola happens to mirror minute 5's value there, but nothing guarantees clicks really behave that way 15 minutes out - treat any prediction outside your observed range with caution.

Polynomial regression still describes a relationship between just one input and one output - it only changes the shape of the line. The next page tackles a different extension: predicting an outcome from two or more separate input variables at once.

Keep Going

Data Science Interview QuestionsPractice real questions asked for Data ScienceData Scientist SalarySee what Data Scientist roles pay in the marketAuto Apply for JobsApply to open roles related to Data Science and tweak your resume