Does temperature 1 mean completely random output?

No. It normally retains the model's unscaled probability distribution, which may already strongly favor particular tokens.

Does lower temperature eliminate hallucinations?

No. It changes selection probabilities, not whether the model's favored claims are supported by evidence.

Can temperature be zero?

Some interfaces accept zero as a special decoding setting. The standard scaling formula requires positive T, so consult the implementation's documented behavior.

AI Glossary / Generative AI and language / Beginner

Temperature

Definition

Temperature is a setting that changes how concentrated or spread out a model's next-token probability distribution is before sampling. In the usual positive-temperature formulation, lower values favor already likely tokens more strongly, while higher values give less likely tokens more opportunity. Temperature controls a selection process; it is not a direct measure of creativity, knowledge, confidence, or factual accuracy.

What it means

A language model generally produces scores for possible next tokens. Those scores are converted into probabilities, and a decoding procedure chooses what comes next. Temperature changes the relative emphasis placed on differences between the scores.

At temperature 1, the usual scaling leaves the scores unchanged. A positive value below 1 sharpens the distribution, while a value above 1 flattens it. This describes the effect before any additional filtering rules are applied.

The setting does not add facts or retrain the model. A more predictable answer may still be wrong, and a more varied answer is not necessarily more useful. Its value depends on the task and the rest of the generation setup.

How it works

For ordinary temperature scaling, divide each next-token score, or logit, by a positive temperature T, then apply softmax to obtain probabilities. Equivalently, probabilities from the unscaled softmax can be raised to the power 1/T and normalized.

For positive T, scaling alone preserves the ordering of unequal scores. It changes how likely a lower-ranked token is to be selected, rather than making that token the highest-ranked choice.

Other settings, such as top-k or top-p filtering, may further change the available candidates. Greedy decoding directly selects a highest-scoring token and ordinarily does not need temperature sampling. Some interfaces use zero as a special instruction for greedy behavior; zero cannot simply be substituted into the division formula, and supported settings depend on the implementation.

Three practical examples

1. A brainstorming application compares several temperatures to see whether generated headline options become more varied while still meeting the brief.

2. An extraction tool tests a concentrated decoding setup but separately checks that required fields are supported by the input document.

3. An evaluation team records temperature along with the model, prompt, and other decoding settings so that comparisons do not silently mix different generation conditions.

Think of it this way

Think of a weighted draw in which some tickets already have much greater weight than others. Lowering temperature exaggerates those weight differences; raising it makes the draw more even.

The analogy does not mean that every answer is chosen from a fixed list of complete responses. Language generation usually repeats the process token by token, with the choices at one step changing the context for the next. Temperature also does not tell the draw which choices are factually correct.

Worked example

Suppose a toy model has just two possible next tokens, with probabilities 0.8 and 0.2 at temperature 1. At temperature 0.5, raise each probability to the power 2: 0.64 and 0.04. Their sum is 0.68, so the new probabilities are approximately 0.9412 and 0.0588.

At temperature 2, take square roots instead. The unnormalized weights are approximately 0.8944 and 0.4472, giving probabilities two-thirds and one-third after normalization.

The first token remains the most likely in all three cases. The second token becomes less likely at 0.5 and more likely at 2. These calculations assume temperature scaling alone, with no candidate filtering or other score adjustment. They illustrate probability changes, not the quality of any generated sentence.

Illustrative case study

Fictional case: Bright Page uses an assistant to draft workshop titles. At a low temperature, several drafts are nearly identical. The team compares a higher-temperature setting using the same collection of workshop descriptions.

Reviewers count useful distinct titles and reject titles that promise topics the workshops do not cover. They keep the factual review criterion unchanged rather than assuming novelty is automatically an improvement.

The fictional trial illustrates selecting a setting through task-specific evaluation. It does not claim that increasing temperature produces better writing in general or that one value works for every model.

Common misconceptions and mistakes

Do not call temperature a truthfulness dial. Concentrating a mistaken distribution can make a wrong answer more repeatable.

Do not assume higher temperature always produces greater meaningful creativity. It can also produce irrelevant or incoherent continuations, depending on the model and task.

Avoid expecting a temperature value to have identical practical effects across unrelated models. Their underlying distributions differ. Likewise, do not infer complete reproducibility from a low or zero setting alone; the whole serving and decoding setup matters. Record relevant versions and settings when repeatability is important.

Business use

Temperature is a useful experimental control for products that generate alternative wording or other variable outputs. Teams should select it using representative requests and criteria such as usefulness, consistency, and factual support.

Keep the chosen setting attached to evaluation records and deployment configurations. If a model update changes output behavior, retest the combination instead of assuming the old temperature has preserved the same balance. The appropriate choice is an observed property of the application, not a universal number for business writing.

Check your understanding

Two next tokens have probabilities 0.8 and 0.2. Can positive temperature scaling alone reverse which is more likely?

Answer: No. It changes the size of the probability gap while preserving their order. Additional processing or a changed context can alter the distribution separately.

Related terms

Learn next

Learn temperature sampling next to follow the repeated draw-and-update process. Then compare top-k and top-p sampling, which restrict candidate tokens in different ways and can interact with temperature scaling.

Sources

Google Machine Learning Glossary, “temperature”: https://developers.google.com/machine-learning/glossary

Hugging Face generation utilities and implementation: https://huggingface.co/docs/transformers/main/en/internal/generation_utils and https://github.com/huggingface/transformers/blob/main/src/transformers/generation/logits_process.py

Primary documentation and the score-division implementation were read. The two-token calculation, evaluation guidance, and Bright Page case are original teaching material.

Previous: Telephone survey Next: Temperature sampling