AI Glossary / Causal inference and experimentation / Intermediate

Evaluation

Definition

Evaluation is the systematic assessment of an AI model, output, or complete system against defined purposes and criteria. It uses evidence such as test results, human judgments, observed outcomes, and operational measurements to determine how well the system performs and where it falls short. Evaluation is broader than calculating one metric. A useful evaluation specifies the intended use, the conditions tested, the quality dimensions examined, and the limits of the conclusions that the evidence supports.

What it means

The question is not simply “How smart is this model?” A more useful question is whether a particular system can perform a particular task for particular users under realistic conditions. A model may answer benchmark questions well but fail when documents are missing, instructions conflict, or a workflow requires accurate citations. Evaluation connects capability measurements to those practical requirements. It can happen during development, before release, and after deployment as the system and its environment change.

How it works

Start with the decision the evaluation must inform: compare candidates, identify failures, set a release condition, or monitor performance. Define representative cases, important edge cases, expected outcomes, and acceptance criteria. Select complementary methods, including automatic tests and human review where appropriate. Keep assessment data separate from inappropriate tuning, record system and evaluator versions, and analyze both averages and failure categories. Interpret results in the context of sampling, uncertainty, and untested conditions. Turn findings into a documented decision or a specific improvement to investigate.

Three practical examples

• Document assistant: assess answer correctness, citation support, handling of missing evidence, and response time on realistic queries.
• Predictive model: examine discrimination, calibration, and errors at the threshold used by the actual workflow.
• AI-supported process: measure whether users complete their task successfully and how much correction or review the system requires.

Think of it this way

Think of evaluating a vehicle for a delivery route. A top speed measurement is informative, but the route also requires capacity, reliability, maneuverability, and suitable operating conditions. The analogy shows why one impressive number is insufficient. AI evaluation likewise needs criteria tied to the actual job, without assuming that a test in one environment certifies every other use.

Worked example

A fictional document assistant is tested on 200 held-out questions. Reviewers find 170 answers factually correct, giving 85% correctness. The answers contain 400 factual statements, of which 340 have adequate cited support under the team's rule, giving 85% citation recall. The matching percentages do not make the measures interchangeable: one counts answers, while the other counts statements.
The team also records whether each complete answer is correct, sufficiently cited, and free of a predefined critical omission. Only 150 answers meet all three conditions, so joint pass rate is 150/200 = 75%. Median response time is assessed separately because speed is not included in those counts. A release decision must consider the joint requirement and the critical failures rather than assuming that two 85% scores imply 85% complete success. These invented numbers illustrate evaluation design, not measured product performance or a recommended acceptance threshold.

Illustrative case study

A fictional service team initially evaluates its assistant using clean, complete documents. After a pilot, it encounters outdated versions, missing pages, and ambiguous user requests. The team adds those conditions to its evaluation and checks whether the assistant acknowledges uncertainty or asks for necessary information. It retains the original test for comparison while labeling the expanded set clearly. Evaluation becomes an ongoing description of supported and unsupported use conditions, rather than a one-time certificate attached to a model version.

Common misconceptions and mistakes

Evaluation is not synonymous with a benchmark leaderboard, a demonstration, or a test-set score. A few persuasive examples do not establish typical performance. Human judgments are not automatically consistent or free of error, so rubrics and agreement checks matter. Automatic scoring can be repeatable yet measure the wrong property. Passing an offline test does not prove a system will remain reliable after deployment. Conversely, a failed test should be diagnosed before concluding that every possible use of the system is unsuitable.

Business use

Use evaluation to make decisions traceable to evidence. Connect each criterion to a requirement someone actually cares about, and name the tradeoffs among quality, speed, effort, and cost. Keep important failures visible even when average performance improves. Involve people who understand the domain and the workflow when interpreting ambiguous outcomes. Record what was not tested and what data would resolve important uncertainty. Repeat or update evaluation when models, prompts, retrieval data, integrations, user populations, or operating conditions materially change.

People also ask

Is evaluation only needed at the end of development?

No. Early evaluation can reveal that the task, data, or success criteria need revision before substantial effort is spent.

Must every evaluation use a numeric score?

No. Structured qualitative findings can identify failure modes that a summary number misses.

Can a model and a system need different evaluations?

Yes. A system includes retrieval, tools, interfaces, and human processes that can improve or undermine the model's behavior.

Check your understanding

Why is the 75% joint pass rate in the worked example useful when separate criteria each score 85%?

Answer: The complete workflow requires several conditions to hold for the same answer. Separate averages do not reveal how often all required conditions are satisfied together.

Related terms

• Evaluation metric
• Benchmarks
• Test set
• Validation
• Human evaluation
• Automatic evaluation
• benchmark
• error analysis
• acceptance criteria
• monitoring
• task success
• system evaluation

Learn next

Write one intended use as a concrete task with a target user and operating conditions. Define three requirements, a test for each, and at least one case that could pass the tests while still failing the user. Then study evaluation validity, sampling, and uncertainty to improve the connection between measured results and the decision they are meant to support.

More in Causal inference and experimentation

• E-optimal design - E-optimal design chooses an experimental allocation that maximizes the smallest eigenvalue of its information matrix.
• Error rate - Error rate is the proportion of evaluated units counted as incorrect under a defined rule.
• Estimand - An estimand is the precise quantity that a study aims to learn about.
• Evaluation function - An evaluation function is a rule or program that assigns a value to something being assessed.
• Exact match - Exact match is an evaluation rule that gives credit only when a prediction equals an accepted reference answer under the specified comparison procedure.
• Experiment - An experiment is a planned investigation in which a researcher deliberately changes one or more conditions and observes the resulting outcomes.

Sources

NIST, AI measurement and evaluation: https://www.nist.gov/ai-measurement-and-evaluation NIST, AI RMF Core and Measure function: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/ Scikit-learn, model evaluation guide: https://scikit-learn.org/stable/modules/model_evaluation.html

Previous: Evaluating parsers Next: Evaluation Bias