ML Logistic Regression

From Regression to Classification

Linear regression predicts a continuous number, such as a price or a temperature. Many real problems instead ask a yes-or-no question: will this email be marked as spam, will this tumor be malignant, will this customer churn? Logistic regression answers these binary classification questions by predicting the probability that an observation belongs to the positive class (labeled 1) rather than the negative class (labeled 0).

The Sigmoid Function

Logistic regression starts the same way linear regression does, by computing a weighted sum of the input features plus an intercept: z = b0 + b1*x1 + b2*x2 + .... That value z can range from negative infinity to positive infinity, so it is passed through the sigmoid (logistic) function, sigmoid(z) = 1 / (1 + e^-z), which squashes it into the range (0, 1). Large positive z values push the output close to 1, large negative z values push it close to 0, and z = 0 gives exactly 0.5, forming the characteristic S-shaped curve.

import numpy as np

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

z_values = np.array([-6, -2, -0.5, 0, 0.5, 2, 6])
probabilities = sigmoid(z_values)

for z, p in zip(z_values, probabilities):
    print(f'z = {z:>5} -> sigmoid(z) = {p:.4f}')

Fitting LogisticRegression in scikit-learn

scikit-learn hides the sigmoid math behind a familiar fit and predict interface. Give LogisticRegression a 2D array of features X and a 1D array of binary labels y, call fit(X, y), and the model finds the coefficients that best separate the two classes. Calling predict(X_new) then returns the predicted class labels, 0 or 1, for new observations.

import numpy as np
from sklearn.linear_model import LogisticRegression

# Tumor diameter in cm
X = np.array([1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5]).reshape(-1, 1)
# 0 = benign, 1 = malignant
y = np.array([0, 0, 0, 0, 0, 1, 1, 1, 1, 1])

model = LogisticRegression()
model.fit(X, y)

new_tumors = np.array([2.2, 3.8, 4.9]).reshape(-1, 1)
predictions = model.predict(new_tumors)
print('Predicted classes:', predictions)

• Feature values are combined into a linear score z, then passed through the sigmoid function.
• The model outputs a probability between 0 and 1, not a raw class label.
• A decision threshold, 0.5 by default, converts the probability into a class prediction.
• Coefficients represent the change in log-odds for a one-unit increase in a feature.
• Logistic regression handles binary classification and extends to multiple classes with a one-vs-rest or multinomial strategy.

probabilities = model.predict_proba(new_tumors)
print('P(benign), P(malignant):')
print(probabilities)

print('Coefficient (log-odds per cm):', model.coef_[0][0])
print('Intercept:', model.intercept_[0])

odds_ratio = np.exp(model.coef_[0][0])
print(f'Each extra cm multiplies the odds of malignancy by {odds_ratio:.2f}')