Step by Step
D
Data drift — the input distribution changes
Data drift occurs when the statistical distribution of input data changes over time compared to what the model was originally trained on, even if the underlying relationship between features and labels stays the same.
Example: a fraud detection model trained on pre-pandemic spending patterns may see significant data drift once spending habits shift dramatically.
C
Concept drift — the relationship between features and labels changes
Concept drift occurs when the actual relationship between input features and the target label changes over time, meaning even the same input might now warrant a different prediction than it used to.
Example: a spam filter's concept of what constitutes "spam" may shift over time as spammers change their tactics, meaning the same email content patterns might now (or no longer) indicate spam.
M
Model monitoring — tracking drift and degradation over time
Ongoing production systems must actively track data drift, concept drift, and general model performance degradation, rather than assuming a deployed model will remain accurate indefinitely.
Example: setting up automated alerts that flag when a production model's real-world accuracy drops below an acceptable threshold.
O
(Overall infrastructure) — the 95% most people don't see
Version control for code (Git), data (DVC), and models (MLflow), CI/CD pipelines for automatic retraining and testing, and feature stores for centralized, reusable feature computation are all part of the substantial infrastructure surrounding the actual model code.
Example: an ML system might have a relatively small amount of core model training code, surrounded by a much larger amount of pipeline, monitoring, versioning, and infrastructure code.
Applied Walkthrough
1
A team deploys a model that performed excellently during testing, assuming it will remain accurate indefinitely without further attention.
2
Six months later, real-world spending patterns have shifted significantly, and the model's performance has quietly degraded without anyone noticing, since there was no active monitoring in place.
3
Ask: what specific type of drift does this scenario describe? Data drift, since the input distribution of spending patterns has changed over time.
4
Proper MLOps practice would have included active model monitoring from the start, tracking data drift and performance metrics continuously, triggering an alert or automatic retraining well before performance degraded this significantly.
Exam Application
Exams test whether you can distinguish data drift (input distribution changes) from concept drift (the feature-label relationship itself changes), and whether you understand the broader MLOps principle that model code is only a small fraction (around 5%) of a real production ML system, with the rest being pipelines, monitoring, versioning, and infrastructure.
⚠ Common Trap
The most common trap is confusing data drift with concept drift. Data drift means the INPUTS look different now; concept drift means the RELATIONSHIP between inputs and the correct output has itself changed — a model can experience one without the other, and they require different diagnostic approaches.
✓ Quick Self-Check
1. What is data drift?
A change in the statistical distribution of input data over time, compared to what the model was originally trained on.
Tap to reveal / hide
2. What is concept drift?
A change in the actual relationship between input features and the target label over time.
Tap to reveal / hide
3. What roughly what percentage of a real ML system's code is the model training code itself?
Roughly 5% — the remaining 95% is pipelines, monitoring, and infrastructure.
Tap to reveal / hide
4. What tools are commonly used for versioning code, data, and models respectively in MLOps?
Git for code, DVC for data, and MLflow for models.
Tap to reveal / hide
5. What is the purpose of a feature store in MLOps?
Centralized, reusable feature computation across different models and pipelines.
Tap to reveal / hide