Can a verified system fail validation?

Yes. It may meet specified requirements while those requirements omit a real user need or operating condition.

Must validation wait until development ends?

No. Prototypes, simulations and early workflow exercises can test assumptions before costly design choices become fixed.

Does V&V guarantee an AI system is safe?

No universal guarantee follows. It provides evidence for specified properties and uses under stated conditions, with remaining uncertainty and limitations.

AI Glossary / Causal inference and experimentation / Intermediate

Verification and validation

Definition

Verification and validation, often abbreviated V&V, are complementary activities for assessing a system. Verification checks whether specified requirements are satisfied. Validation checks whether the system is suitable for its intended purpose, users and operating environment. Both require evidence matched to a clearly defined scope.

What it means

A system can follow its specification while solving the wrong problem. It can also appear useful in a demonstration while failing an important requirement. The two perspectives help expose these different gaps.

For AI, the system includes more than the model. Inputs, preprocessing, interfaces, tools, people and operating procedures can determine whether a prediction leads to a useful action. Evidence about a classifier alone should not silently become a claim about an entire service.

In machine learning, validation also names a development-data activity used for model selection. That narrower meaning is relevant but does not cover every question about intended use. Specify the meaning when a report says a system has been validated.

How it works

Begin with stakeholder needs, intended operating conditions and explicit requirements. Make each requirement precise enough to assess, and connect it to an appropriate evidence method. Depending on the claim, verification can use testing, inspection, analysis, demonstration or formal reasoning.

Plan validation around realistic tasks and conditions. Examine whether typical users can achieve the intended outcome, including when information is missing, predictions are wrong or external services fail. Early prototypes and simulations can provide useful evidence before a complete deployment exists.

Record the system version, assumptions, test conditions, acceptance criteria, observations and unresolved issues. Trace findings back to the requirement or user need they address. A failure may require an implementation repair, a better requirement or reconsideration of the intended workflow.

Repeat relevant checks after changes. Passing selected tests does not prove every possible behavior, and formal verification establishes only the properties represented under its assumptions. The strength of the conclusion must match the method and coverage.

Three practical examples

A document service verifies that every exported record contains a source reference, then validates whether reviewers can use those references to resolve ambiguous extracts.
A scheduling system verifies its stated conflict rules and validates whether the resulting schedules work with actual handover practices.
An assistant verifies that an approval control blocks unauthorized sending, then validates whether users understand which draft and recipient they are approving.

Think of it this way

Think of ordering a desk. Verification checks that its dimensions and materials match the order. Validation checks whether it fits the room and supports the person's work.

A desk is much simpler than an adaptive information system. The analogy separates compliance from suitability; it does not imply that either can be settled by one measurement for a complex AI service.

Worked example

One requirement passes, one need remains unmet

An invented support tool has a requirement to display a suggested category within two seconds for at least 95% of requests under a specified test load. In 200 measured requests, 194 meet the time limit. The observed proportion is 97%, satisfying that stated sample-based criterion.

A workflow trial then finds that the suggestion appears after agents have already committed their selection. Users must undo their work to use it. The timing requirement can pass while the intended assistance is poorly served.

The team revises the interaction design and reassesses both the requirement and the workflow. The 97% measurement is evidence under the tested conditions, not a guarantee for every future request. Meeting a numerical specification did not answer whether that specification was sufficient.

Illustrative case study

A fictional purchasing service, Rowan Orders, introduces an AI assistant that prepares replenishment requests. Verification shows that the assistant cannot submit an order without an approval action recorded by the application.

During validation, staff approve a summary without noticing that the underlying product quantities have changed after the summary was generated. The application has an approval event, but the workflow does not reliably connect approval to the content being ordered.

The team changes the design so approval identifies a specific order version and displays the consequential details. It verifies the version checks and validates the revised review task with representative users. The lesson is to investigate the meaning and adequacy of a requirement rather than celebrating a technically satisfied but incomplete rule.

Common misconceptions and mistakes

V&V is not simply running a benchmark and receiving a pass badge. Different claims need different evidence, and a score may cover only one component.

Verification does not always mean formal proof, and validation does not always mean a user survey. Both may use several methods with different strengths and limitations.

Do not assume that a previous assessment remains valid after a major change in model, data, permissions or operating conditions. Also avoid treating absence of observed failures as proof that failure is impossible. Document what was examined and what remains outside the evidence.

Business use

Use V&V to make release and change decisions traceable. Separate requirement compliance from fitness for the intended workflow, while connecting both to stakeholder needs.

Assign ownership for unresolved findings and state which conditions the system is intended to handle. Ask suppliers for the specific claims supported, the tested configuration and the evidence limitations. For consequential functions, arrange appropriately independent challenge of the work. The aim is a defensible assessment, not a broad assurance label that hides the underlying scope.

Check your understanding

A model meets its specified accuracy target, but users misunderstand its output and make incorrect workflow decisions. Has model accuracy alone established system suitability?

Answer: No. The accuracy result supports its defined predictive requirement. Validation must also address the interaction and intended use that produced the misunderstanding.

Related terms

Model validation
Traceability
Requirements
system validation
formal verification
testing
acceptance criteria
intended use
test, evaluation, verification and validation

Learn next

Write one requirement and one user need for an AI service. Describe separate evidence that would assess each, and identify how one could pass while the other fails. Then map both to the complete workflow and the conditions that would trigger reassessment.

Sources

Sources: NASA Systems Engineering Handbook, verification and validation distinctions: https://www.nasa.gov/reference/2-4-distinctions-between-product-verification-and-product-validation/ NASA, product verification: https://www.nasa.gov/reference/5-3-product-verification/ NASA, product validation: https://www.nasa.gov/reference/5-4-product-validation/ NIST AI RMF, AI actor tasks and lifecycle TEVV: https://airc.nist.gov/airmf-resources/airmf/appendices/app-a-descriptions-of-ai-actor-tasks/

Previous: Verification Next: Version space