Skip to content
ReviewRamble

The weekly brief

AI, reviewed, in your inbox.

  • New reviews of tools, models, MCP servers and repos, with the verdict up front
  • Re-checked prices and ratings when something changes
  • New analysis pieces as they are published

Models, agents, MCP servers and the tools actually worth your time.

One email a week. Unsubscribe with the link in any brief. How we use your email

Temperature (sampling)

Temperature is a sampling setting that controls how random a language model's word choices are. Low values make it favour the most likely tokens; higher values spread choices across less likely ones.

Also called: sampling temperature

By Divya Sharma · Checked Nov 15, 2026 · How we review

At each step a large language model produces a probability for every possible next token. Temperature rescales those probabilities before one is picked. Lowering it makes the distribution sharper, so likely tokens become even more likely. Raising it flattens the distribution, so unlikely tokens get picked more often.

How it works

Temperature divides the model's raw scores before the softmax step that turns them into probabilities. Hugging Face's guide to text generation describes lowering the temperature as making the distribution sharper, increasing the likelihood of high-probability words and decreasing that of low-probability ones. Near zero, sampling behaves almost like always taking the top token.

What to set

- Low temperature suits extraction, classification, code and anything with one right answer. - Higher temperature suits brainstorming, varied wording and creative writing.

Ranges and defaults differ by provider, so check the API reference for the model you call.

How it differs from top-p

Top-p sampling cuts off the unlikely tail of the distribution and samples from what is left. Temperature reshapes the whole distribution. A common rule of thumb is to adjust one and leave the other at its default.

Limits

Low temperature does not make a model correct. It makes the model more consistent, which includes being consistently wrong. It also does not promise identical output on every call. If you need reliable formats, use structured output; if you need facts, use grounding.

Related terms

Large language model
A large language model (LLM) is a neural network with billions of parameters, trained on large amounts of text to predict the next token, which lets it write, answer questions, summarise and follow instructions.
Top-p sampling
Top-p sampling, or nucleus sampling, picks each next token only from the smallest set of candidates whose combined probability reaches a threshold p, cutting off the long tail of unlikely tokens.
Token (LLM)
A token (LLM) is the basic unit of text a language model reads and writes, often a word, part of a word, a punctuation mark or a byte. Prices, speed and context limits are all counted in tokens.
Structured output
Structured output is an API feature that makes a language model's reply follow a format you define, usually a JSON schema, so that code can parse it reliably instead of scraping free text.
Hallucination (AI)
A hallucination is output from an AI model that sounds fluent and confident but is false, unsupported by its sources or made up, such as an invented citation, quote, statistic or API function.

Sources