Python Data Analysis Deep Dive — Practical Machine Learning — Mastering the scikit-learn Workflow

Lecture 5 Overview

scikit-learn is the de facto standard library for machine learning in Python. Its consistent API (fit, transform, predict) lets you work with dozens of algorithms the same way. This lecture walks through a real project start to finish so you fully internalize the scikit-learn workflow.

---

The Full scikit-learn Workflow

1. Prepare data
2. split into train/test
3. preprocessing pipeline
4. model selection
5. cross-validation
6. hyperparameter tuning
7. final evaluation

---

1. Splitting the Data

from sklearn.model_selection import train_test_split

X = df.drop('target', axis=1)
y = df['target']

# A standard split (20% test)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y  # stratify is recommended for classification
)

---

2. Pipelines

Bundling preprocessing and the model into a single pipeline prevents data leakage and keeps your code clean.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import RandomForestClassifier

# Define preprocessing per column
numeric_features = ['age', 'income', 'score']
categorical_features = ['dept', 'region']

preprocessor = ColumnTransformer([
    ('num', StandardScaler(), numeric_features),
    ('cat', OneHotEncoder(drop='first'), categorical_features)
])

# Assemble the pipeline
pipeline = Pipeline([
    ('preprocessor', preprocessor),
    ('classifier', RandomForestClassifier(n_estimators=100, random_state=42))
])

# Train and predict
pipeline.fit(X_train, y_train)
y_pred = pipeline.predict(X_test)

---

3. A Guide to Choosing a Model

Problem type	Data size	Recommended models
Classification	Small–medium	LogisticRegression, RandomForest
Classification	Large	XGBoost, LightGBM
Regression	Small–medium	Ridge, RandomForestRegressor
Regression	Large	XGBoost, LightGBM
Clustering	Any size	KMeans, DBSCAN

# Comparing several models (a common pattern in real work)
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.svm import SVC
from sklearn.model_selection import cross_val_score

models = {
    'Logistic': LogisticRegression(max_iter=1000),
    'RandomForest': RandomForestClassifier(n_estimators=100, random_state=42),
    'GBM': GradientBoostingClassifier(random_state=42),
}

for name, model in models.items():
    scores = cross_val_score(model, X_train, y_train, cv=5, scoring='f1_macro')
    print(f"{name}: {scores.mean():.4f} ± {scores.std():.4f}")

---

4. Evaluation Metrics

Classification Metrics

from sklearn.metrics import (
    accuracy_score, precision_score, recall_score, f1_score,
    confusion_matrix, classification_report, roc_auc_score
)

print(classification_report(y_test, y_pred))

# ROC-AUC
from sklearn.metrics import RocCurveDisplay
RocCurveDisplay.from_estimator(pipeline, X_test, y_test)

Metric	Description	When it matters
Accuracy	Overall correctness	When classes are balanced
Precision	Of predicted positives, how many are actually positive	When false positives are costly (spam filters)
Recall	Of actual positives, how many were caught	When false negatives are costly (cancer diagnosis)
F1-Score	Harmonic mean of precision and recall	Imbalanced data
ROC-AUC	Overall classification performance	Threshold-independent evaluation

Regression Metrics

from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score

rmse = np.sqrt(mean_squared_error(y_test, y_pred))
mae = mean_absolute_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

---

5. Hyperparameter Tuning

from sklearn.model_selection import GridSearchCV, RandomizedSearchCV

param_grid = {
    'classifier__n_estimators': [100, 200, 300],
    'classifier__max_depth': [None, 5, 10],
    'classifier__min_samples_split': [2, 5, 10]
}

grid_search = GridSearchCV(
    pipeline, param_grid, cv=5,
    scoring='f1_macro', n_jobs=-1
)
grid_search.fit(X_train, y_train)
print(f"Best parameters: {grid_search.best_params_}")
print(f"Best CV score: {grid_search.best_score_:.4f}")

---

Frequently Asked Questions

Q: How do I detect overfitting? A: If training scores are high but validation/test scores are low, that’s overfitting. Fix it with more data, regularization (L1/L2), dropout, or a simpler model.

Q: How do I integrate XGBoost with scikit-learn? A: XGBoost supports the scikit-learn API, so you can drop XGBClassifier directly into a Pipeline. Just import it with from xgboost import XGBClassifier.