Step by Step
B
BLEU — precision-oriented, for translation
BLEU measures how many n-grams in the MACHINE OUTPUT appear in the reference translation — a precision-oriented measure, since it asks "of what the model generated, how much was correct/matched?" It is the standard metric for machine translation, ranging from 0 to 1, with higher being better.
Example: if the machine's output contains 10 n-grams and 8 of them appear in the reference, that reflects the precision-oriented calculation BLEU performs.
R
ROUGE — recall-oriented, for summarization
ROUGE measures how many n-grams in the REFERENCE appear in the generated summary — a recall-oriented measure, since it asks "of what the reference contains, how much did the generated summary capture?" It is the standard metric for text summarization.
Example: if the reference summary contains 10 n-grams and the generated summary captures 7 of them, that reflects the recall-oriented calculation ROUGE performs.
V
Variants — ROUGE-1, ROUGE-2, ROUGE-L
ROUGE-1 measures unigram (single word) overlap, ROUGE-2 measures bigram (two-word sequence) overlap, and ROUGE-L measures the longest common subsequence between generated and reference text.
Example: ROUGE-L specifically capturing the longest common subsequence shared between a generated summary and its reference, even if that sequence isn't perfectly contiguous.
Applied Walkthrough
1
A translation system generates an output translation, and a summarization system generates an output summary, and both need to be evaluated against their respective reference texts.
2
For the translation task, BLEU is used, precisely checking how many of the n-grams the machine actually produced also appear in the reference — a precision-oriented check.
3
For the summarization task, ROUGE is used instead, checking how many of the n-grams present in the reference summary were successfully captured by the generated summary — a recall-oriented check.
4
This precision-vs-recall distinction is the precise, technical reason BLEU and ROUGE are used for different tasks, beyond simply being associated with "translation" and "summarization" as separate labels.
Exam Application
Exams test whether you understand the precise technical distinction between BLEU (precision-oriented: how much of the OUTPUT matches the reference) and ROUGE (recall-oriented: how much of the REFERENCE is captured by the output) — a more precise framing than simply "BLEU for translation, ROUGE for summarization."
⚠ Common Trap
The most common trap is treating BLEU and ROUGE as simply two interchangeable n-gram overlap metrics distinguished only by which task they're conventionally used for. The deeper, more precise distinction is that BLEU is precision-oriented (from the output's perspective) while ROUGE is recall-oriented (from the reference's perspective) — this precision/recall framing is what's specifically and repeatedly tested.
✓ Quick Self-Check
1. Is BLEU precision-oriented or recall-oriented?
Precision-oriented — it measures how many n-grams in the machine output appear in the reference.
Tap to reveal / hide
2. Is ROUGE precision-oriented or recall-oriented?
Recall-oriented — it measures how many n-grams in the reference appear in the generated output.
Tap to reveal / hide
3. What does ROUGE-1 measure, specifically?
Unigram (single word) overlap.
Tap to reveal / hide
4. What does ROUGE-L measure, specifically?
The longest common subsequence between the generated and reference text.
Tap to reveal / hide
5. What is the more precise technical distinction between BLEU and ROUGE, beyond just their conventional task associations?
BLEU is precision-oriented (output's perspective); ROUGE is recall-oriented (reference's perspective).
Tap to reveal / hide