Temperature is not a creativity dial
Temperature, top-k and top-p are three different operations on one probability distribution. Treat them as vibes and you'll tune the wrong one. Watch them act on a model small enough to see.
Most explanations of sampling settings stop at metaphor: temperature is "creativity", top-p is "focus". The metaphors aren't wrong so much as useless when something goes wrong, because they don't tell you which knob is responsible.
For the LLM course on this site I trained a deliberately tiny language model in the browser — word-level trigrams over Alice's Adventures in Wonderland — precisely because you can see every probability it produces. It is nothing like a frontier model in capability. In the part that matters here, it is identical: a list of scores becomes a probability distribution, and something has to choose from it.
Greedy decoding loops, and now you can see why
Set temperature to zero and give it "The Queen". It produces "The Queen! The Queen! The Queen!" and never stops. Greedy decoding always takes the single most likely next token, and once the most likely path leads back to where it started, nothing can break the cycle. Large models loop for the same structural reason, just over longer spans.
Temperature reshapes; it doesn't add ideas
Temperature divides the scores before they become probabilities. Below 1, the favourite gets more dominant; above 1, the distribution flattens. That is all it does. It cannot make a token likely that the model considers implausible — it only changes how often you gamble on the tail.
That distinction matters in practice. If outputs are bland, raising temperature increases variance, not quality. If outputs are occasionally unhinged, the problem is usually that the tail is being sampled at all — which is a job for truncation, not for a slightly lower temperature.
Truncation is a separate decision
top-k keeps only the k most likely candidates. top-p keeps the smallest set whose probabilities add up to p, so it adapts: one candidate when the model is sure, dozens when it isn't. Both cut the tail before sampling, which is why high temperature plus a sensible top-p is often better behaved than moderate temperature alone.
Change one at a time. They interact, and two people adjusting two knobs at once is how teams end up with settings nobody can explain.
Seeds are for engineers, not users
Sampling uses a random number generator; fix its seed and the same prompt and settings reproduce the same output. That is invaluable for debugging, regression tests and fair prompt comparisons — and irrelevant to users, who rarely want identical answers twice. Note that hosted APIs typically promise only best-effort reproducibility.
Name the operation, not the feeling. "Flatten the distribution" is debuggable; "be more creative" is not.
If you want to build the intuition properly, the course lets you move each dial and watch the bars move with it. Ten minutes of that is worth more than any metaphor, including mine.