Step by Step
Q
Query — what am I looking for?
For each output position, the Query represents what that position is currently trying to find or focus on among the other positions.
Example: while processing the word "it" in a sentence, the Query represents the network's attempt to figure out what "it" refers to.
K
Key — what do I contain?
Each position also produces a Key, representing a summary of what information that position holds, which gets compared against Queries to determine relevance.
Example: the word "dog" earlier in the sentence produces a Key representing its content, which can be compared against the Query from "it" to check relevance.
V
Value — what do I actually provide?
Once relevance scores are computed (Query dot Key, scaled and passed through softmax), those scores weight each position's Value, and the weighted sum of Values becomes the output — the actual "context" passed forward.
Example: if "dog" scores highly relevant to "it," then "dog"'s Value contributes heavily to the final context representation computed for "it."
H
Multi-Head — running h parallel spotlights
Rather than computing attention just once, multi-head attention runs h independent attention calculations in parallel, each potentially capturing a different type of relationship (grammatical, semantic, positional) between tokens.
Example: one attention head might focus on subject-verb relationships while another focuses on pronoun-antecedent relationships, all computed in parallel and then combined.
Applied Walkthrough
1
A Transformer model processes the sentence "The dog chased the ball because it was fast," and needs to figure out what "it" refers to.
2
The Query for "it" is compared against the Keys of every other word in the sentence, computing relevance scores via a dot product.
3
After scaling and softmax, "dog" and "ball" likely receive different weights based on which one the context suggests "it" refers to, and the corresponding Values are combined in that weighted proportion.
4
Running this process across multiple attention heads in parallel allows the model to simultaneously capture several different relationship types (like grammatical role and semantic similarity) rather than relying on just one single attention calculation.
Exam Application
Exams test whether you can define Query, Key, and Value and describe the attention calculation (Q·K, scaled, softmax, weighted sum of V), as well as whether you understand why multiple attention heads are used (capturing different relationship types in parallel) and the distinction between self-attention (tokens attend to each other) and causal/decoder attention (only attends left, masking future tokens for autoregressive generation).
⚠ Common Trap
The most common trap is confusing self-attention (used in encoders, where every token can attend to every other token, including ones that come later) with causal attention (used in decoders, where each position can only attend to earlier positions, with future tokens masked out to preserve the autoregressive generation property).
✓ Quick Self-Check
1. What does the Query represent in attention?
What the current position is looking for or trying to find among other positions.
Tap to reveal / hide
2. What does the Key represent in attention?
A summary of what information a given position holds, compared against Queries to determine relevance.
Tap to reveal / hide
3. How is the final attention output computed once relevance scores are known?
As a weighted sum of the Values, weighted by the softmax-normalized relevance scores.
Tap to reveal / hide
4. Why does multi-head attention run multiple attention calculations in parallel?
To capture different types of relationships between tokens simultaneously, rather than relying on just one single attention pattern.
Tap to reveal / hide
5. What is the key difference between self-attention and causal (decoder) attention?
Self-attention allows tokens to attend to all other tokens; causal attention only allows attending to earlier positions, masking future tokens to preserve autoregressive generation.
Tap to reveal / hide