Step by Step
C
Classification metrics — Top-1 and Top-5 accuracy
For image classification, Top-1 accuracy measures whether the model's single top prediction was correct, while Top-5 accuracy measures whether the correct answer appeared anywhere among the model's top 5 predictions.
Example: a model achieving 76% Top-1 accuracy but 93% Top-5 accuracy on ImageNet, meaning the correct answer is often within its top 5 guesses even when its single top guess is wrong.
A
AP and mAP — the object detection standard
Average Precision (AP) is the area under the precision-recall curve for one class; mean Average Precision (mAP) averages this AP across all classes, forming the standard metric for object detection.
Example: computing AP separately for "car," "person," and "dog" classes, then averaging all three AP values together to get the overall mAP score.
P
Pascal VOC vs. COCO — different difficulty standards
Pascal VOC's mAP uses a single IoU threshold of 0.5, while COCO's mAP averages across a range of IoU thresholds from 0.5 to 0.95, making COCO's evaluation standard considerably harder and more demanding of precise localization.
Example: a detector achieving a strong mAP score under the easier Pascal VOC standard (IoU=0.5 only) but a noticeably lower mAP score under the harder COCO standard (averaged across IoU 0.5 to 0.95).
m
mIoU and FPS — segmentation and speed metrics
For segmentation, mIoU (mean Intersection over Union) averages IoU across all semantic classes. For real-time applications, FPS (frames per second) matters, with 30+ FPS generally required for real-time performance.
Example: a segmentation model achieving a strong mIoU score for accuracy, while also needing to sustain 30+ FPS to be usable in a real-time application like autonomous driving.
Applied Walkthrough
1
Two object detectors report seemingly similar mAP scores, but one was evaluated using the Pascal VOC standard and the other using the COCO standard.
2
Ask: can these two mAP scores be directly compared as equivalent? No — Pascal VOC's mAP uses only a single, easier IoU threshold of 0.5, while COCO's mAP averages across the much harder range of 0.5 to 0.95.
3
A detector's Pascal VOC mAP score will typically look higher than its COCO mAP score, purely due to this difference in evaluation difficulty, not necessarily reflecting genuinely better performance.
4
This is exactly why it's essential to know which evaluation standard (Pascal VOC or COCO) a reported mAP score is using before making any direct comparisons between different detectors.
Exam Application
Exams test whether you can distinguish classification metrics (Top-1/Top-5 accuracy) from detection metrics (AP/mAP) from segmentation metrics (mIoU), and specifically whether you understand why COCO's mAP standard (averaged across IoU 0.5-0.95) is considerably harder than Pascal VOC's (IoU=0.5 only) — a frequently tested distinction.
⚠ Common Trap
The most common trap is directly comparing mAP scores from different evaluation standards (Pascal VOC vs. COCO) as if they were equivalent. Because COCO's standard is considerably harder (averaging across a range of IoU thresholds rather than using just the single easiest 0.5 threshold), a lower COCO mAP score doesn't necessarily indicate a worse detector than a higher Pascal VOC mAP score.
✓ Quick Self-Check
1. What does Top-5 accuracy measure, as opposed to Top-1 accuracy?
Whether the correct answer appeared anywhere among the model's top 5 predictions, rather than requiring it to be the single top prediction.
Tap to reveal / hide
2. What does AP (Average Precision) measure for a single class?
The area under the precision-recall curve for that class.
Tap to reveal / hide
3. What is the key difference between Pascal VOC's mAP and COCO's mAP?
Pascal VOC uses a single IoU threshold of 0.5; COCO averages across a range of IoU thresholds from 0.5 to 0.95, making it considerably harder.
Tap to reveal / hide
4. What does mIoU measure, and for what task?
Mean Intersection over Union across all semantic classes, used for segmentation.
Tap to reveal / hide
5. What FPS threshold is generally required for real-time performance?
30+ FPS.
Tap to reveal / hide