Categorical Encoding

Why Encode Categoricals?

ML algorithms operate on numbers. Categorical features — "country", "gender", "product_type" — must be converted to numeric representations. The wrong encoding is one of the most common sources of model performance loss.

Label Encoding (Ordinal Encoding)

Assigns an integer to each category: cat → 0, dog → 1, fish → 2. Only appropriate for ordinal variables where the ordering is meaningful.

Python

import pandas as pd
from sklearn.preprocessing import LabelEncoder, OrdinalEncoder

df = pd.DataFrame({
    "education": ["Bachelor", "Master", "PhD", "Bachelor", "Master"],
    "city":      ["NYC", "LA", "NYC", "Chicago", "LA"],
    "salary":    [60000, 80000, 110000, 65000, 85000],
})

# OrdinalEncoder with explicit order (CORRECT for ordinal)
enc = OrdinalEncoder(categories=[["Bachelor", "Master", "PhD"]])
df["education_enc"] = enc.fit_transform(df[["education"]])

# LabelEncoder (no order — BAD for "city", OK for target variable)
le = LabelEncoder()
df["city_label"] = le.fit_transform(df["city"])   # NYC=2, LA=1, Chicago=0
# BAD: now the model thinks Chicago < LA < NYC numerically!

⚠️

Never label-encode nominal features for linear models or neural networks. The arbitrary integer ordering (NYC=2, LA=1) introduces false numerical relationships. Use one-hot encoding instead. For tree-based models it's less harmful but still incorrect.

One-Hot Encoding

Creates a binary column for each category. No false ordinal relationship. Best for nominal variables with low-to-medium cardinality (<50 unique values).

Python

import pandas as pd
from sklearn.preprocessing import OneHotEncoder

# Pandas approach (quick, readable)
df_ohe_pd = pd.get_dummies(df, columns=["city"], drop_first=True, dtype=int)
# drop_first=True: drop one column to avoid perfect multicollinearity

# Sklearn approach (for pipelines)
enc = OneHotEncoder(handle_unknown="ignore", sparse_output=False, drop="first")
city_encoded = enc.fit_transform(df[["city"]])
city_df = pd.DataFrame(city_encoded, columns=enc.get_feature_names_out())
print(city_df.head())

Target (Mean) Encoding

Replace each category with the mean target value for that category. Handles high-cardinality features well. Risk: target leakage — must be done inside cross-validation folds.

Python

import pandas as pd
import numpy as np
from category_encoders import TargetEncoder    # pip install category_encoders
from sklearn.model_selection import cross_val_score
from sklearn.linear_model import Ridge
from sklearn.pipeline import Pipeline

df_large = pd.DataFrame({
    "country": np.random.choice(["US","UK","DE","FR","JP","IN","BR","CA"], 500),
    "revenue": np.random.exponential(1000, 500),
})
df_large["target"] = df_large["revenue"] * 0.001 + np.random.randn(500)

# TargetEncoder handles cardinality and smoothing
pipe = Pipeline([
    ("enc",   TargetEncoder(cols=["country"])),
    ("model", Ridge()),
])
scores = cross_val_score(pipe, df_large[["country"]], df_large["target"], cv=5)
print(f"CV R2: {scores.mean():.4f}")

# Manual target encoding with k-fold to prevent leakage
def kfold_target_encode(df, col, target, n_folds=5, smoothing=5):
    from sklearn.model_selection import KFold
    result = df[col].copy().astype(float)
    global_mean = df[target].mean()
    kf = KFold(n_splits=n_folds, shuffle=True, random_state=42)
    for train_idx, val_idx in kf.split(df):
        train_fold = df.iloc[train_idx]
        stats = train_fold.groupby(col)[target].agg(["mean","count"])
        smoothed = (stats["mean"] * stats["count"] + global_mean * smoothing) / (stats["count"] + smoothing)
        result.iloc[val_idx] = df.iloc[val_idx][col].map(smoothed).fillna(global_mean)
    return result

Binary & Hash Encoding

For very high cardinality (thousands of categories — e.g., ZIP codes, user IDs):

Binary EncodingEncode category as integer, then convert to binary bits. $K$ categories → $\log_2 K$ columns. Much more compact than one-hot.

HashingHashingEncoder maps categories to a fixed number of buckets via hashing. Handles unseen categories at inference time. Risk: collisions.

EmbeddingLearn a dense vector per category (neural network style). Best for very high cardinality in deep learning. Used in entity embeddings for tabular data.

Encoding Decision Tree

Ordinal (ordered)Use OrdinalEncoder with explicit order.

Nominal, low cardinality (<50)One-hot encoding. Use drop="first" for linear models.

Nominal, high cardinality (50–1000)Target encoding with K-fold. BinaryEncoder as alternative.

Very high cardinality (>1000)Hashing, target encoding, or learned embeddings.

Tree-based modelsLabelEncoder works (trees don't assume ordinal). Or use OHE — trees handle both.

Summary

Ordinal encoding: use only for naturally ordered categories (size: S/M/L, rating: 1–5).
One-hot encoding: nominal, low-medium cardinality. Use drop="first" for linear models.
Target encoding: high-cardinality nominals. Must be done inside CV folds to prevent leakage.
For very high cardinality: binary/hash encoding or learned embeddings.