A8gent
GlossaryAgent concepts

What is temperature?

Also called: sampling temperature

Temperature is the sampling setting that controls how much randomness a model uses when picking each next token - low values make output predictable, high values make it varied.

Written by A8gent
Contents
  1. At a glance
  2. What temperature does
  3. What to set it to
  4. What temperature does not do
  5. What you should know
  6. What temperature does not change
  7. FAQ
  8. Related terms

At a glance

Category
Agent concept - model plumbing
What it controls
Randomness in next-token sampling, not reasoning quality
Extraction and classification
0, or the lowest the provider allows
Customer-facing drafting
Low to moderate - enough variety to avoid identical repeats
Does not fix
Hallucination, or factual accuracy of any kind
Caveat
Some reasoning models ignore temperature entirely - check the model docs

What temperature does

At every step a model produces a probability distribution over possible next tokens. Temperature reshapes that distribution before one is sampled. At low values the distribution sharpens and the most likely token nearly always wins, producing consistent, conventional output. At high values it flattens, so unlikely tokens get a real chance and the output becomes varied and eventually incoherent.

The common mental model - "creativity dial" - is close enough for choosing a value and misleading about what it means. Temperature does not make the model more imaginative; it makes it less certain, and uncertainty applied to facts is what people experience as the model going off. It is a variance control, not a quality control.

What to set it to

For anything with a right answer, use 0 or the lowest your provider allows: classification, field extraction, routing, structured output, anything a downstream step will parse. You want the same input to produce the same output, both because it is correct more often and because it makes your evals meaningful and your bugs reproducible.

For customer-facing drafting - a follow-up message, a job summary - a low-to-moderate value avoids output that reads identically every time while keeping it on-topic. For genuine ideation, where you want twenty different angles, higher values earn their keep. Providers differ on the permitted range and on whether temperature interacts with top-p, so change one at a time and test rather than adjusting both by feel.

What temperature does not do

It does not prevent hallucination. Temperature 0 makes the model deterministic, not accurate: it will reproduce the same confident invention every time. Grounding, citation checks and validation are the controls that address accuracy, and presenting temperature 0 to a client as a safety measure is a claim you cannot support.

It also does not guarantee reproducibility on its own. Even at 0, provider-side batching, model updates and floating-point non-determinism mean outputs can vary between calls or between days. If you need identical results, pin the model version where the provider allows it, use any seed parameter offered, and still validate output rather than assuming stability. And note that some newer reasoning models ignore temperature entirely - check the model's own documentation before tuning a parameter it does not read.

What you should know

Temperature is one of the few knobs that is genuinely safe to tune, and one of the most over-tuned. In practice most production automations run at 0 or near it and get their quality from the system prompt, from retrieval, and from validation - not from sampling settings. If you find yourself adjusting temperature to fix a behaviour, the problem is almost always in the prompt or the input.

What temperature does not change

Temperature is free. It has no effect on what a call costs, because billing is by token count and not by sampling behaviour. What does change the cost is how much the model writes, and on every current lineup output tokens are several times the price of input tokens - gpt-5.6-sol is $5.00 input against $30.00 output per million as of August 2026, a six-to-one ratio.

So if a model is producing rambling answers, the fix that saves money is an instruction about length or a structured output schema, not a temperature change. Reach for temperature to control variability and reach for the prompt and schema to control spend. Note also that some current model families do not accept sampling parameters at all on their reasoning modes, so check the specific model's API reference before assuming the parameter is available.

Temperature FAQ

What does temperature do in an LLM?

It reshapes the probability distribution over next tokens before one is sampled. Low temperature sharpens it so the likeliest token nearly always wins, producing consistent output. High temperature flattens it, so less likely tokens get picked and output becomes varied.

What temperature should I use for business automations?

Zero, or the lowest value allowed, for anything with a correct answer - classification, extraction, routing, structured output. Use a low but non-zero value for customer-facing drafting so consecutive messages do not read word for word identically. Reserve high values for brainstorming.

Does temperature 0 stop hallucinations?

No. It makes the output deterministic, not true - the model will confidently reproduce the same invention every time. Accuracy comes from grounding the answer in retrieved sources, requiring verifiable citations and validating against your own systems.

Will temperature 0 give me the same output every time?

Usually, but not guaranteed. Provider-side batching, silent model updates and floating-point non-determinism all introduce variation. Pin the model version where you can, use a seed parameter if one is offered, and validate output rather than assuming byte-identical results.

Should I adjust temperature or top-p?

Pick one and leave the other at its default. They both control sampling breadth and adjusting them together makes the effect hard to reason about. Temperature is the more widely supported and better understood of the two, so it is the sensible default choice.

My output is still inconsistent at temperature 0 - what now?

The variance is coming from the input, not the sampler. Check whether the prompt is receiving different retrieved context, different record fields or different formatting between runs. Log the fully rendered prompt for two runs that diverged and compare them - the difference is almost always visible.

Primary source

OpenAI - text generation guide

Checked August 2026. Anything priced or rate-limited moves, so verify against the source before you budget.

Want this built rather than explained?

Knowing what temperature means is one thing; having it running in your business is another. We scope one workflow at a time, quote a fixed price in writing after a free consult, build it, and hand it over with documentation you own.