AI Glossary / Prompting and human-AI interaction / Intermediate

Frequency penalty

Definition

A frequency penalty is a text-generation setting that lowers the chance of a language model choosing a word or token again in proportion to how many times it has already appeared in the output so far. It is a sampling control, not a rule about meaning or quality. Exact formulas, value ranges and defaults differ between providers, so check each tool's documentation.

What it means

When a language model writes, it picks one small piece of text (a token) at a time. Left alone, it can fall into loops, repeat the same phrase, or lean on one favorite word. A frequency penalty is a dial that pushes back against this. The more often a token has already shown up, the harder the model is nudged away from choosing it again.

For a business user, the practical effect is variety. A high setting can make product descriptions, summaries or brainstormed lists read less repetitive. A setting of zero leaves the model's natural habits untouched. Most business tasks that need exact wording, such as quoting a contract clause or listing part numbers, work better with the penalty off, because repeating a word is often correct.

How it works

At each step the model scores every possible next token. Those scores are turned into probabilities, and one token is sampled. A frequency penalty adjusts the scores before sampling. For each token, the system counts how many times it has already appeared in the generated text, multiplies that count by the penalty value, and subtracts the result from that token's score. A token used five times is pushed down five times as hard as one used once.

This is different from a presence penalty. A presence penalty applies one flat reduction once a token has appeared at all, no matter how many times. The frequency penalty grows with each repeat, so it targets heavy repetition specifically. Some providers apply the count only to generated text, and others may include the prompt, so behavior can vary.

The penalty works on tokens, not ideas. It can discourage a repeated word while the model still repeats the same point in different words. Very high values have side effects: the model may avoid common words it needs, such as names, units or the word "the", and produce awkward or strange phrasing. Negative values, where allowed, do the opposite and encourage repetition.

Three practical examples

• Retail product copy: A shop owner asks a model for 20 short descriptions of similar mugs. The first batch keeps starting with "Perfect for" and repeating "cozy". A small positive frequency penalty spreads the vocabulary so the descriptions read less like copies of each other.
• Finance reporting: An analyst generates a monthly commentary on expense variances. The penalty is left at zero because terms like "variance", "budget" and the account names must repeat exactly. Raising it would risk the model swapping in vague synonyms for precise labels.
• Support desk: A team drafts reply templates and notices the model loops, saying "I understand your frustration" three times in one message. A modest frequency penalty reduces the looping, and a human agent still reviews each draft before it is sent.

Think of it this way

Think of a speaker who is gently tapped on the shoulder each time they reuse a favorite word, with the tap getting firmer on every repeat. They start reaching for other words. The analogy stops working because the speaker understands which repeats are harmless, while the model applies the tap to any token by count alone, including words that should repeat.

Worked example

Suppose a model's raw score for the token "great" is 4.0 and the frequency penalty is set to 0.5. Here are the adjusted scores as the word keeps appearing.

Before "great" has appeared: count is 0, so the reduction is 0 x 0.5 = 0. The score stays 4.0.

After 1 use: reduction is 1 x 0.5 = 0.5. The score becomes 4.0 - 0.5 = 3.5.

After 4 uses: reduction is 4 x 0.5 = 2.0. The score becomes 4.0 - 2.0 = 2.0.

Now compare with a competing token, "excellent", with a raw score of 3.0 that has appeared 0 times. At the start, "great" (4.0) beats "excellent" (3.0). After 4 uses of "great", its adjusted score is 2.0, so "excellent" (3.0) now scores higher and becomes more likely. The penalty has shifted the model toward variety without forbidding any word. Doubling the setting to 1.0 would make the same switch happen sooner: after 2 uses, 4.0 - (2 x 1.0) = 2.0.

Illustrative case study

Fictional case: Harbor Lane Outfitters, a made-up online camping retailer, used a language model to write 300 product blurbs. Reviewers noticed many blurbs opened the same way and repeated "rugged" two or three times each. The content lead raised the frequency penalty from 0 to 0.4, and the repetition dropped. Then a second problem appeared: a few blurbs avoided the brand's own product names and used clumsy substitutes, because the high penalty was discouraging a name that legitimately appeared several times. The lead lowered the value to 0.2, moved the product names into a fixed template field outside the model's free text, and added a human spot check of every tenth blurb. Variety stayed acceptable and the names stayed exact.

Common misconceptions and mistakes

A higher frequency penalty always makes writing better. Past a modest level it tends to make text strained, because the model avoids words it actually needs.

The penalty stops the model from repeating ideas. It only acts on individual tokens, so the same point can still return in different words.

Frequency penalty and presence penalty are the same thing. The frequency penalty grows with each repeat, while the presence penalty is a single flat reduction once a token has appeared.

The same setting behaves identically in every tool. Scales, defaults and what counts as earlier text differ by provider, so values do not transfer cleanly.

Business use

Start with the default, usually zero, and change it only when you see a real repetition problem. Adjust in small steps, such as 0.1 to 0.3 on a typical zero-to-two scale, and compare outputs side by side on the same prompts. Keep it at or near zero for tasks needing exact terms, quotes, codes or figures. Often a clearer prompt, such as asking for varied openings or a different structure, fixes repetition with less risk than a penalty.

Measure results on a fixed sample: count repeated openings or phrases, rate readability, and check that required terms still appear correctly. The cost is mostly review time, since the setting itself is free. The main risk is silent quality loss, such as wrong synonyms for technical terms, so keep a human check on anything customer-facing or regulated, and record the setting you chose so results are reproducible.

People also ask

What is a good frequency penalty value?

There is no universal value. Many people start at zero and try small positive amounts only if repetition is a problem. The scale and sensible range depend on the provider.

What is the difference between frequency penalty and presence penalty?

A frequency penalty reduces a token's chance more each time it repeats. A presence penalty applies one fixed reduction once the token has appeared at all.

Should I use a frequency penalty for factual or numeric work?

Usually not. Exact terms and figures often need to repeat, and a penalty can push the model toward wrong or vague substitutes.

Check your understanding

In the worked example, what is the score of "great" after 2 uses at a penalty of 0.5?

Answer: The reduction is 2 x 0.5 = 1.0, so the score is 4.0 - 1.0 = 3.0.

Why did Harbor Lane Outfitters lower the penalty from 0.4 to 0.2?

Answer: The higher value made the model avoid its legitimately repeated product names and use clumsy substitutes.

Does a frequency penalty stop a model from repeating the same idea in new words?

Answer: No. It acts on individual tokens by count, not on meaning.

When is it sensible to leave the frequency penalty at zero?

Answer: When the task needs exact wording, such as quoting text, listing codes, or reporting figures with fixed labels.

Related terms

• Presence penalty
• Temperature
• Top-p sampling
• Token
• Sampling
• Prompt engineering
• Hallucination
• Context window

Learn next

Try the same prompt at three penalty values and compare the outputs against a short checklist of repetition and accuracy. Then read about presence penalty and temperature, since the three settings interact and are often tuned together.

More in Prompting and human-AI interaction

• Fallback response - A fallback response is the pre-planned reply or action an AI system gives when it cannot answer well, such as when it is unsure, the request is out of...
• Few-shot learning - Few-shot learning is the challenge of adapting to a task or recognizing new classes from only a small number of labeled examples.
• Few-shot prompting - Few-shot prompting means including a small number of example inputs and desired outputs in a prompt before asking a model to handle a new input.
• Generalized Few-shot Learning - Generalized few-shot learning is a classification setting in which a model must recognize both previously learned base classes and new classes introduced...
• Human-AI interaction - Human-AI interaction is the study and design of how people communicate, collaborate and make decisions with AI systems.
• Human-computer interaction - Human-computer interaction, or HCI, is the study and design of how people interact with computing systems.

Sources

OpenAI, Prompt engineering guide: https://platform.openai.com/docs/guides/prompt-engineering DAIR.AI, Prompt Engineering Guide: https://www.promptingguide.ai/ Jurafsky and Martin, Speech and Language Processing: https://web.stanford.edu/~jurafsky/slp3/ The sources were consulted for how language models sample text and how generation settings are described in prompting guidance; the examples, analogy, worked example and case are original educational illustrations. Checked 5 Nov 2026.

Previous: Frequency distribution Next: Frequency principle/spectral bias