ML Categorical Data

Why Categories Need Encoding

A column like car_color with values Red, Blue, and Green looks perfectly reasonable to a human, but scikit-learn's estimators expect numeric arrays. Simply mapping Red to 0, Blue to 1, and Green to 2 is dangerous because it implies an order and a distance between categories that does not exist, the model might learn that green is greater than red, which is meaningless for an unordered category.

One-Hot Encoding

One-hot encoding solves this by creating one new binary column per category. Each row gets a 1 in the column matching its category and 0 in all the others. Because no category is represented by a bigger or smaller number than another, the model can no longer invent a false ordering between them.

import pandas as pd

df = pd.DataFrame({
    'car_color': ['Red', 'Blue', 'Green', 'Blue', 'Red'],
    'price': [22000, 25000, 21000, 24000, 23000]
})

encoded = pd.get_dummies(df, columns=['car_color'])
print(encoded)

get_dummies vs OneHotEncoder

pandas.get_dummies() is the fastest way to one-hot encode a DataFrame during exploration, but it has a drawback, it only creates columns for the categories it has already seen. sklearn.preprocessing.OneHotEncoder is the safer choice inside a production pipeline because it can be fit on training data and then applied consistently to new data, even when a category is missing or new categories should raise an error instead of being silently dropped.

import numpy as np
from sklearn.preprocessing import OneHotEncoder

colors = np.array([['Red'], ['Blue'], ['Green'], ['Blue']])

encoder = OneHotEncoder(sparse_output=False, handle_unknown='ignore')
encoded_colors = encoder.fit_transform(colors)

print(encoder.get_feature_names_out())
print(encoded_colors)

• Use drop_first=True with get_dummies, or drop='first' with OneHotEncoder, to avoid the dummy variable trap where encoded columns are linearly dependent.
• Set handle_unknown='ignore' on OneHotEncoder so categories missing from training data do not crash predictions later.
• get_dummies works directly on a DataFrame and returns column names automatically; OneHotEncoder needs get_feature_names_out() to recover them.
• Wrap OneHotEncoder in a ColumnTransformer to encode categorical columns while leaving or scaling numeric columns separately.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression

numeric_features = ['mileage']
categorical_features = ['car_color']

preprocessor = ColumnTransformer([
    ('num', StandardScaler(), numeric_features),
    ('cat', OneHotEncoder(handle_unknown='ignore'), categorical_features)
])

model = Pipeline([
    ('preprocess', preprocessor),
    ('regressor', LinearRegression())
])

# model.fit(X_train, y_train) applies scaling and encoding automatically
print(preprocessor)