Step by Step
R
ROC curve — plotting TPR against FPR at every threshold
The ROC (Receiver Operating Characteristic) curve plots the True Positive Rate (recall) against the False Positive Rate at every possible classification threshold, showing the full tradeoff across the entire range of thresholds at once.
Example: a ROC curve visually traces how TPR and FPR change together as you slide the classification threshold from very strict to very lenient.
A
AUC — area under that curve, a single summary number
AUC (Area Under the Curve) condenses the entire ROC curve into a single number representing the probability that the model ranks a randomly chosen positive example higher than a randomly chosen negative example.
Example: an AUC of 0.85 means there's an 85% chance the model would rank a randomly chosen positive example higher (more likely positive) than a randomly chosen negative example.
⚠
When AUC/ROC can mislead — severe class imbalance
AUC and ROC curves are threshold-independent and evaluate ranking ability generally, but they can be overly optimistic when the positive class is very rare — a PR (precision-recall) curve is preferred in that specific situation.
Example: in a fraud detection dataset where only 0.1% of transactions are actual fraud, ROC/AUC might look deceptively good even if the model performs poorly on the rare positive class specifically.
Applied Walkthrough
1
Two classifiers are being compared, and a team wants a single number summarizing overall ranking quality without committing to any specific decision threshold.
2
AUC is calculated for both classifiers, giving a threshold-independent comparison: AUC=0.5 would mean no better than random, AUC=1.0 would mean perfect, and AUC=0.8+ is generally considered good.
3
However, the team realizes their dataset has a severely rare positive class (fraud cases are only 0.1% of transactions).
4
In this specific situation, they should switch to a PR curve instead of relying on ROC/AUC, since ROC/AUC can be overly optimistic with severe class imbalance, potentially masking poor performance on the rare positive class.
Exam Application
Exams test whether you understand what AUC actually represents (probability of correctly ranking a random positive above a random negative) and specifically when to avoid it in favor of a PR curve — namely, when the positive class is very rare, since ROC/AUC can be misleadingly optimistic in that scenario.
⚠ Common Trap
The most common trap is assuming a high AUC always means strong real-world performance, regardless of class balance. With a very rare positive class, AUC/ROC curves can look deceptively strong even when the model is actually performing poorly on the class that matters most — a PR curve is the more honest choice in that situation.
✓ Quick Self-Check
1. What two rates does the ROC curve plot against each other?
True Positive Rate (recall) against False Positive Rate, at every possible classification threshold.
Tap to reveal / hide
2. What does AUC represent in plain terms?
The probability that the model ranks a randomly chosen positive example higher than a randomly chosen negative example.
Tap to reveal / hide
3. What AUC value indicates no better than random guessing?
0.5.
Tap to reveal / hide
4. What AUC value is generally considered good?
0.8 or higher.
Tap to reveal / hide
5. When should you use a PR curve instead of ROC/AUC?
When the positive class is very rare, since ROC/AUC can be overly optimistic with severe class imbalance.
Tap to reveal / hide