AI Glossary / Generative AI and language / Intermediate

Golden response

Definition

A golden response is a carefully selected reference answer used to represent an expected or high-quality response to a particular input. In AI evaluation, it provides a comparison point for judging model output. It may be written by an expert, curated from verified material, or reviewed after being drafted by another system. Calling a response golden describes its role in the evaluation; it does not make the content infallible or establish that only one wording is acceptable.

What it means

A golden response makes an expectation concrete. Instead of saying “the assistant should answer well”, evaluators can show the information, reasoning, tone, or structure a good answer should contain. This is especially useful for repeatable tests and discussions among reviewers. However, a single answer can underrepresent the range of valid responses. For an open-ended request, the reference should often be paired with required facts, permitted alternatives, and a rubric describing what matters most.

How it works

Choose a representative input and identify the evidence or task rules needed to answer it. Draft and review the reference, checking correctness, completeness, scope, and any time-sensitive details. Record its sources and version. Decide how model outputs will be compared: exact matching may suit a fixed identifier, while semantic or rubric-based assessment may suit a paragraph. Test the evaluator on valid paraphrases and plausible wrong answers. Keep the reference and scoring rules separate so reviewers can identify whether a disagreement comes from the answer or the metric.

Three practical examples

• Support evaluation: create a reviewed answer explaining a return procedure, including the conditions that must not be omitted.
• Extraction testing: use an exact expected order identifier or date where the output has a tightly defined representation.
• Summarization review: provide a reference summary plus required facts and prohibited distortions, allowing different wording that preserves the same meaning.

Think of it this way

Think of a worked solution used by a teacher when marking an exercise. It shows what a successful answer can look like and helps make judgments consistent. The analogy also reveals the limitation: a student can reach a correct result through different valid wording or reasoning, while the teacher's own solution may need correction if it contains an error.

Worked example

A fictional policy states that unopened items can be returned within 30 days and that the customer must provide proof of purchase. The golden response includes both requirements. An evaluation set contains four candidate answers. Candidate A repeats the reference exactly. Candidate B says the same thing in different words. Candidate C mentions the 30-day limit but omits proof of purchase. Candidate D says 60 days and includes proof of purchase.
Literal exact matching would accept only A, producing one pass out of four, or 25%. A meaning-based rubric requiring both correct conditions would accept A and B, producing two passes, or 50%. Under that rubric, C is incomplete and D is incorrect. These two rates do not show that the model changed; they reflect different scoring criteria. If the policy later changes to 45 days, the old reference becomes stale and can incorrectly reward outdated answers. The evaluation therefore needs a policy version or effective date. The example also shows why a high text-similarity score alone is insufficient: D can share many words with the reference while changing the most important number.

Illustrative case study

A fictional travel-support team assembles golden responses for common itinerary questions. Reviewers discover that several answers assume a particular starting city even though the prompts never specify one. The team revises those test cases so an appropriate clarifying question can count as successful behavior. It retains reference answers for fully specified requests and writes separate expectations for ambiguous ones. This prevents the evaluation from rewarding confident guesses simply because the original reference made the same unsupported assumption.

Common misconceptions and mistakes

A golden response is not necessarily a unique answer, a direct quotation, or a complete definition of quality. It can contain mistakes or become outdated. Using the same examples for training and final evaluation can inflate results through contamination. A model-based judge can still miscompare a candidate with the reference. Adding many stylistic details to a golden response can accidentally penalize concise correct answers. Conversely, a short reference may omit requirements that matter in the real task.

Business use

Use golden responses to make evaluation expectations reviewable and consistent. Assign ownership for corrections and updates, especially when the answer depends on changing business rules. Store the input, relevant context, supporting evidence, accepted alternatives, and scoring rationale together. Sample disagreements for human review and track whether failures arise from the application, the reference, or the judge. Keep a reserved evaluation set so repeated optimization against familiar examples does not become the only evidence of quality.

People also ask

Must a human write every golden response from scratch?

No. Drafting can be assisted, but the reference needs appropriate review before it is trusted as an evaluation target.

Should every candidate match the reference word for word?

Only when exact form is part of the task. Many language tasks allow several correct expressions.

Can the golden response be a clarifying question?

Yes. When essential information is missing, asking for it may be the expected behavior rather than providing a speculative answer.

Check your understanding

A candidate uses different wording but includes all required facts accurately. Should it fail solely because it differs from the golden response?

Answer: Not unless exact wording is an explicit task requirement. The scoring method should reflect the intended meaning and behavior rather than accidental phrasing.

Related terms

• Ground truth
• Exact match
• Semantic similarity
• Human evaluation
• LLM-as-a-judge
• Reference answer
• evaluation dataset
• rubric
• expected facts
• test contamination

Learn next

Write one reference answer and list the facts or behaviors that make it acceptable. Create a correct paraphrase, an incomplete answer, and a plausible incorrect answer, then test whether the rubric distinguishes them. Study reference-based evaluation and evaluator calibration next, with particular attention to versioning and valid alternative responses.

More in Generative AI and language

• Generative model - A generative model represents a process or distribution that can produce data, such as text, images, sounds, or structured records.
• Gibbs sampling - Gibbs sampling is a Markov chain Monte Carlo method for drawing approximate samples from a joint probability distribution by repeatedly sampling variables...
• Glitch token - A Glitch token is a token associated with unusually unreliable or anomalous behavior in a language model, such as difficulty reproducing its text or...
• Graph Language Model - A Graph Language Model combines language representations with information about graph structure, such as entities and their relationships.
• Grouped Query Attention - Grouped Query Attention, usually written grouped-query attention or GQA, is an attention design in which several query heads share a smaller set of key and...
• Head - A head is a model component that performs a particular computation or produces a particular kind of output.

Sources

Google for Developers, Machine Learning Glossary, Generative AI: https://developers.google.com/machine-learning/glossary/generative Google Cloud, Playbook evaluations: https://docs.cloud.google.com/dialogflow/cx/docs/concept/playbook/evaluation Databricks, Evaluation dataset reference: https://docs.databricks.com/gcp/en/mlflow3/genai/eval-monitor/concepts/eval-datasets

Previous: Golden record (informatics) Next: Golden-section search