Data Science Logistic Regression

What Logistic Regression Actually Predicts

Despite the word 'regression' in its name, logistic regression is used for classification, not for predicting numbers on an open scale. Given a set of input features, it outputs a probability between 0 and 1 that a row belongs to a particular class, such as 'will churn' vs 'will not churn', or 'spam' vs 'not spam'. A threshold, usually 0.5, converts that probability into a final label.

The Sigmoid Function

Linear regression produces a raw weighted sum of the inputs, z = b0 + b1*x1 + b2*x2 + ..., and that value can range from negative infinity to positive infinity. Logistic regression takes that same linear combination and squashes it through the sigmoid function, sigmoid(z) = 1 / (1 + e^-z). The sigmoid curve is S-shaped: very negative z values map close to 0, very positive z values map close to 1, and z = 0 maps to exactly 0.5. This is what turns an unbounded linear score into a valid probability.

• The model learns coefficients (weights) for each feature during training, just like linear regression.
• Those coefficients are combined into a linear score z, exactly as in linear regression.
• The sigmoid function converts z into a probability between 0 and 1.
• A decision threshold (commonly 0.5) turns the probability into a predicted class label.

import numpy as np
from sklearn.linear_model import LogisticRegression

# Hours studied for a set of students
X = np.array([[0.5], [1.0], [1.5], [2.0], [3.0], [4.0], [5.0], [6.0]])
# 1 = passed the exam, 0 = failed
y = np.array([0, 0, 0, 0, 1, 1, 1, 1])

model = LogisticRegression()
model.fit(X, y)

# Predicted probability of passing for a student who studied 3.5 hours
prob = model.predict_proba([[3.5]])
print(prob)          # [[P(fail), P(pass)]]
print(model.predict([[3.5]]))  # final class label, 0 or 1

Interpreting the Coefficients

In linear regression, a coefficient tells you how much the predicted value changes per unit of the feature. In logistic regression, coefficients describe how much the log-odds of the positive class change per unit of the feature. A positive coefficient means increasing that feature pushes the probability of the positive class up; a negative coefficient pushes it down. The magnitude also matters: coefficients close to zero contribute little to the prediction.

import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

data = pd.DataFrame({
    'age': [22, 25, 47, 52, 46, 56, 23, 60],
    'income_k': [25, 30, 55, 60, 50, 62, 28, 80],
    'bought_product': [0, 0, 1, 1, 1, 1, 0, 1]
})

X = data[['age', 'income_k']]
y = data['bought_product']

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=0)

model = LogisticRegression()
model.fit(X_train, y_train)

print('Coefficients:', model.coef_)
print('Intercept:', model.intercept_)
print('Test accuracy:', model.score(X_test, y_test))

Logistic regression is a strong default choice for binary classification: it trains quickly, its coefficients are interpretable, and it works well as a baseline before trying more complex models. For more than two classes, scikit-learn extends it automatically using a multinomial (softmax) formulation.