Step by Step
O
Overfitting — memorized the textbook
Training accuracy is very high while validation accuracy is much lower — the model has memorized the specific training examples, including their noise, rather than learning the general underlying pattern.
Example: a student who memorized every practice exam question verbatim scores perfectly on those exact questions but struggles badly with new, slightly different questions on the real test.
U
Underfitting — didn't study at all
Both training AND validation accuracy are low — the model hasn't learned the underlying pattern well enough even on the data it directly trained on.
Example: a student who barely studied at all performs poorly on both the practice exams and the real exam, because they never learned the material in the first place.
📈
The diagnostic tool — learning curves
Plotting training accuracy and validation accuracy against training progress reveals the diagnosis directly: a large gap between the two curves signals overfitting; both curves being low and flat signals underfitting.
Example: a learning curve showing training accuracy near 99% while validation accuracy plateaus around 65% is a textbook overfitting signature.
Applied Walkthrough
1
A model's learning curve shows training accuracy climbing to 97% while validation accuracy plateaus at 68%, leaving a large persistent gap.
2
Ask: does this pattern match overfitting or underfitting? The large training/validation gap is the classic overfitting signature.
3
Contrast: if instead both training accuracy AND validation accuracy plateaued at a low 55%, with no meaningful gap between them, that would signal underfitting instead.
4
Correctly diagnosing which one is happening determines the fix: overfitting calls for simplifying the model, adding data, or regularization; underfitting calls for a more complex model or better features.
Exam Application
Exams almost always present training and validation accuracy numbers (or a learning curve description) and ask you to diagnose overfitting vs. underfitting from the gap between them. Memorize the pattern: big gap (training much higher) = overfitting; both low and close together = underfitting.
⚠ Common Trap
The most common trap is applying the wrong fix to the diagnosis — trying to simplify an underfitting model (making it worse) or trying to add more complexity to an overfitting model (making it worse). Always diagnose correctly first using the training/validation gap before choosing a fix.
✓ Quick Self-Check
1. What does a large gap between training accuracy and validation accuracy indicate?
Overfitting — the model memorized training data details rather than learning generalizable patterns.
Tap to reveal / hide
2. What does it mean if both training AND validation accuracy are low?
Underfitting — the model hasn't learned the underlying pattern well enough, even on data it directly trained on.
Tap to reveal / hide
3. What diagnostic tool is commonly used to distinguish overfitting from underfitting?
Learning curves, plotting training and validation accuracy against training progress.
Tap to reveal / hide
4. What is the correct fix for overfitting?
Simplify the model, add more training data, or apply regularization.
Tap to reveal / hide
5. What is the correct fix for underfitting?
Increase model complexity, add more relevant features, or reduce regularization.
Tap to reveal / hide