Step by Step
L
Low temperature (0-0.3) — deterministic and focused
Low temperature produces deterministic, repetitive, focused output — well suited for factual answers, code generation, and structured data extraction.
Example: setting temperature to 0.1 for a code generation task, where consistent, predictable, correctly-formatted output matters more than creative variation.
H
High temperature (0.7-1.5) — creative and varied
High temperature produces creative, varied, surprising output — well suited for brainstorming, poetry, and fiction writing.
Example: setting temperature to 1.0 for a creative brainstorming task, wanting varied, surprising ideas rather than the single most predictable response.
0
Temperature=0 vs. Temperature=2 — the extremes
Temperature=0 almost always picks the single highest-probability token, producing maximally deterministic output; temperature=2 dramatically increases randomness, sometimes to the point of producing incoherent output.
Example: a temperature=2 setting occasionally producing text that becomes noticeably less coherent, due to the dramatically increased randomness at this extreme setting.
P
Top-p (nucleus sampling) — an alternative approach
Top-p (nucleus sampling) offers an alternative way to control randomness: sampling only from the smallest set of tokens whose cumulative probability exceeds a threshold p, dynamically adapting to how confident or uncertain the model is at each step.
Example: setting top-p=0.9 so the model only samples from the smallest group of tokens whose combined probability reaches 90%, rather than considering the full unrestricted probability distribution.
Applied Walkthrough
1
A developer is building two different applications: a code-generation tool requiring consistent, correct output, and a creative brainstorming tool wanting varied, surprising ideas.
2
For the code-generation tool, they set a low temperature (around 0.1-0.2), favoring deterministic, focused, predictable output appropriate for correct code generation.
3
For the brainstorming tool, they set a much higher temperature (around 1.0), favoring varied, creative, surprising output appropriate for generating diverse ideas.
4
As an alternative or complement to temperature, they could also use top-p (nucleus sampling), which dynamically adjusts how many tokens are considered based on cumulative probability, rather than uniformly scaling randomness the way temperature does.
Exam Application
Exams test whether you can match the appropriate temperature setting to a described use case (low for factual/code tasks, high for creative tasks) and whether you understand top-p as an alternative, dynamically-adaptive approach to controlling randomness, distinct from temperature's more uniform scaling.
⚠ Common Trap
The most common trap is assuming a higher temperature always produces "better" or more "natural" output. Temperature should be matched specifically to the use case: low temperature for tasks requiring consistency and correctness (code, factual answers), high temperature for tasks wanting creative variation — neither setting is universally superior.
✓ Quick Self-Check
1. What kind of output does low temperature (0-0.3) produce, and what tasks is it suited for?
Deterministic, repetitive, focused output — suited for factual answers, code, and structured data.
Tap to reveal / hide
2. What kind of output does high temperature (0.7-1.5) produce, and what tasks is it suited for?
Creative, varied, surprising output — suited for brainstorming, poetry, and fiction.
Tap to reveal / hide
3. What does temperature=0 almost always do?
Picks the single highest-probability token, producing maximally deterministic output.
Tap to reveal / hide
4. What can happen at temperature=2?
Randomness increases dramatically, sometimes to the point of producing incoherent output.
Tap to reveal / hide
5. What does top-p (nucleus sampling) do differently from temperature?
Samples only from the smallest set of tokens whose cumulative probability exceeds a threshold, dynamically adapting to the model's confidence at each step, rather than uniformly scaling randomness.
Tap to reveal / hide