Tag See all →

#scikit-learn

2 stories

MLOps Bug Report

scikit-learn InconsistentVersionWarning: unpickling a model from another version

Ryan Pham · Sep 2, 2026 · 6 min read

A routine dependency update in our scoring service upgraded scikit-learn, and the logs filled with InconsistentVersionWarning when loading models pickled with an older version. One model's predictions shifted slightly. We pinned versions with each model, added load-time checks and moved to portable formats.

Banner: training-serving skew from different preprocessing in training and production

AI & Machine Learning Bug Report

Model with 0.91 AUC offline performs near random online: training-serving skew

Ryan Pham · Nov 27, 2024 · 5 min read

Our fraud model scored 0.91 AUC on holdout data and close to random in production. The serving service re-implemented preprocessing in another language: the scaler was refit on each batch, one-hot categories came in a different order and text was not lowercased. Shipping the pipeline with the model fixed it.