B
Bias — error from wrong assumptions (underfitting)
The model is too simple to capture the real pattern in the data. It makes strong, often wrong, assumptions and misses real structure.
Example: fitting a straight line to data that actually curves — the line will systematically miss the true pattern no matter how much data you add.
V
Variance — error from sensitivity to training noise (overfitting)
The model is too complex and memorizes the specific quirks and noise of the training data rather than the general pattern, so it fails badly on new, unseen data.
Example: a model that perfectly predicts every point in the training set (including its random noise) but performs terribly on a fresh test set.
⚖
The Tradeoff — finding the sweet spot
You cannot minimize both at the same time without limit — reducing bias tends to increase variance and vice versa. The goal is the right complexity for the amount and nature of the data.
Example: adding features and complexity to a too-simple model reduces bias but risks pushing it into high variance — you have to find the balance point for your specific dataset.
Applied Walkthrough
1
A model performs poorly on both the training data AND new test data — accuracy is low everywhere.
2
Ask: is the model matching the training data well at all? No — it's failing even on data it already saw.
3
That points to high bias / underfitting: the model is too simple to capture the pattern, even in training.
4
Contrast: if the model scored 99% on training data but only 60% on test data, that gap signals high variance / overfitting instead — it memorized training noise rather than learning the general pattern.
Exam Application
Exam questions typically give you two accuracy numbers — training accuracy and test accuracy — and ask you to diagnose the problem. A small gap with both numbers low means underfitting (high bias). A large gap where training accuracy is much higher than test accuracy means overfitting (high variance). Memorize this pattern-matching approach; it is tested constantly.
⚠ Common Trap
The most common trap is mixing up which fix applies to which problem. Underfitting fix: make the model MORE complex (more features, more layers, less regularization). Overfitting fix: make the model LESS complex or add more data (regularization, simpler model, more training examples, dropout). Applying the overfitting fix (simplifying) to an underfitting problem makes the underfitting worse, and vice versa.
✓ Quick Self-Check
1. What does high bias in a model mean?
The model is too simple and makes strong wrong assumptions, causing it to underfit and miss real patterns in the data.
Tap to reveal / hide
2. What does high variance in a model mean?
The model is too complex and is overly sensitive to training data noise, causing it to overfit and generalize poorly.
Tap to reveal / hide
3. If training accuracy is 55% and test accuracy is 53%, is this underfitting or overfitting?
Underfitting (high bias) — both numbers are low and close together, meaning the model fails to capture the pattern even in training.
Tap to reveal / hide
4. If training accuracy is 98% and test accuracy is 61%, is this underfitting or overfitting?
Overfitting (high variance) — the large gap shows the model memorized training data but fails to generalize.
Tap to reveal / hide
5. Name one correct fix for an overfitting model.
Add regularization, simplify the model, or add more training data — any of these reduce variance.
Tap to reveal / hide