B
Bias — how far your average shot lands from the bullseye
Bias represents systematic error — the model's average prediction is consistently off from the true answer in the same direction, due to the model being too simple to capture the real pattern (underfitting).
Example: a model that always predicts house prices about $50,000 too low, no matter what training data it sees, has high bias — a systematic, wrong "aim."
V
Variance — how spread out your shots are
Variance represents inconsistency — predictions swing wildly depending on small changes in the training data, because the model is too complex and overly sensitive to training noise (overfitting).
Example: a model that gives wildly different predictions each time it's retrained on a slightly different sample of the same underlying data has high variance.
⚖
The goal — low bias AND low variance
The ideal model has both low bias (accurate on average, good "aim") and low variance (consistent, tight "grouping") — but increasing model complexity to reduce bias tends to increase variance, and vice versa, creating an unavoidable tradeoff.
Example: a very simple linear model has low variance but potentially high bias on curved data; a very complex deep model has low bias but potentially high variance if not regularized properly.
Applied Walkthrough
1
A model consistently underestimates house prices by a similar amount, regardless of which specific training sample it was trained on.
2
Using the bullseye metaphor: are the shots consistently off-target (bias) or wildly scattered (variance)? Consistently off-target — this is high bias.
3
Contrast: if the same model instead gave wildly different price predictions each time it was retrained on a slightly different data sample, that would instead signal high variance.
4
Diagnosing which one you're facing determines the fix: high bias calls for a more complex model; high variance calls for a simpler model, more data, or regularization.
Exam Application
This is one of the most heavily tested ML theory concepts across every course. Exams frequently present a described symptom (consistent systematic error vs. wildly inconsistent predictions) and ask you to diagnose bias vs. variance, then recommend the correct fix — increasing complexity for high bias, decreasing complexity (or adding data/regularization) for high variance.
⚠ Common Trap
The most common trap is trying to fix both bias and variance with the same lever in the same direction. Reducing bias (by increasing model complexity) tends to increase variance, and reducing variance (by decreasing complexity) tends to increase bias — you cannot push both down simultaneously just by adjusting complexity in one direction; finding the sweet spot requires balancing the two.
✓ Quick Self-Check
1. In the bullseye metaphor, what does bias represent?
How far off-target your average shot lands — systematic error from a model that's too simple (underfitting).
Tap to reveal / hide
2. In the bullseye metaphor, what does variance represent?
How spread out or inconsistent your shots are — from a model that's too complex and sensitive to training noise (overfitting).
Tap to reveal / hide
3. What happens to variance when you increase model complexity to reduce bias?
Variance tends to increase — this is the unavoidable tradeoff between the two.
Tap to reveal / hide
4. What is the ideal combination of bias and variance a model should aim for?
Low bias AND low variance simultaneously, though achieving both perfectly is rarely possible in practice.
Tap to reveal / hide
5. A model gives wildly different predictions each time it's retrained on a slightly different sample. Is this a bias or variance problem?
A variance problem — the inconsistency across different training samples is the signature of high variance.
Tap to reveal / hide