Can trustworthiness be measured with one number?

Some individual properties have useful measures, but combining them requires justified choices and can conceal important weaknesses.

Is it solely a technical property?

No. The surrounding workflow and organizational practices also matter.

Can the same system deserve different assessments in different settings?

Yes. Tasks, users, conditions, and consequences can differ.

AI Glossary / Governance, ethics and safety / Advanced

Trustworthiness

Definition

Trustworthiness is the extent to which an AI system merits reliance for a specified purpose, supported by relevant evidence about its behavior, limitations, and surrounding practices. It concerns reasons for justified confidence, rather than how reassuring a system looks or how strongly its users believe in it.

What it means

Trustworthiness is contextual. A tool suitable for suggesting informal meeting titles may lack the evidence needed to summarize complicated technical instructions. A favorable assessment should identify the use, conditions, and system version to which it applies.
Trust is an attitude held by people. Trustworthiness concerns whether that attitude has a sound basis. The two can diverge: users may be confident in an unsuitable system or skeptical of a carefully evaluated one.
The assessment also extends beyond a model's prediction score. Data handling, access controls, user interaction, operational support, and responsibility for correcting problems can all affect whether the complete service deserves reliance. Calling a model trustworthy without examining its deployment can leave important questions unanswered.

How it works

Start with explicit claims. Instead of saying “the assistant is trustworthy,” specify what it should do, for whom, under which conditions, and what unacceptable outcomes must be prevented or managed.
NIST's AI Risk Management Framework discusses multiple characteristics, including reliability, safety, security, transparency, explainability, privacy, and management of harmful bias. Their importance and evaluation depend on context. Strong performance in one area does not establish the others.
Gather evidence suited to each claim. This may include representative testing, assessments of foreseeable misuse, review of organizational responsibilities, and observations of the actual workflow. Record limits in the evidence and interactions between requirements.
An assessment should be revisited as conditions change. A new data source, user group, interface, or model can affect earlier conclusions even if the product keeps the same name.

Three practical examples

A document assistant correctly summarizes most public manuals but retrieves restricted documents for unauthorized users. Its summarization score provides evidence about one capability; it does not resolve the access-control failure.
An internal classification tool performs consistently on the organization's familiar request types. When a new service introduces unfamiliar requests, the team evaluates that use separately before extending its claims.
A recommendation service gives staff a clear way to report incorrect suggestions. The organization assigns responsibility for investigating reports and tracking corrections. That process supports assurance only if it operates effectively; the existence of a reporting button alone is insufficient.

Think of it this way

Think of evaluating a delivery service for fragile goods. Fast arrival is useful, but so are careful handling, correct destinations, and a workable response when something goes wrong. A strong result on speed cannot erase a serious weakness in handling. Similarly, AI trustworthiness requires relevant evidence across the demands of its intended use.

Worked example

Consider an invented procurement evaluation for an internal knowledge assistant. The buyer defines two separate test requirements: at least 95% correct answers on a specified 200-question set, and no unauthorized document disclosures in a specified 50-attempt access test.
Candidate A answers 194 of 200 questions correctly: 194/200 = 97%. It discloses restricted content in two of the 50 access attempts.
Candidate B answers 192 questions correctly: 96%. It produces no unauthorized disclosures in that access test.
Both meet the stated answer-quality threshold. Only B meets both test requirements. Averaging A's 97% answer score with its 96% access-test success rate would produce 96.5%, but that average would conceal a requirement the buyer explicitly treats as necessary.
B's result still does not establish complete trustworthiness. Fifty unsuccessful attempts do not prove that every possible access failure is absent. The buyer must consider test coverage, other requirements, and the limits of the evidence. These thresholds and observations are illustrative, not universal standards or a certification method.

Illustrative case study

Fictional company Cedar Reference pilots an assistant that locates operating instructions. Demonstrations use public documents, and staff praise the quality of its summaries. The company initially treats that reaction as evidence that the service is ready for wider use.
Before expanding, the project team maps the proposed workflow. Some documents have restricted access, older instructions remain in the archive, and users need to distinguish current guidance from superseded material.
The team adds tests for permissions and document currency, identifies who will investigate reported errors, and limits the pilot to the conditions it has assessed. It keeps separate records of user confidence and technical findings.
This fictional case illustrates how an attractive demonstration can leave important assurance questions open. It does not claim that a particular checklist makes every deployment trustworthy or that positive user feedback has no value.

Common misconceptions and mistakes

“High accuracy establishes trustworthiness.” Other requirements can remain unmet.
“Users trust it, so it deserves trust.” Confidence can exceed the evidence.
“One successful assessment lasts indefinitely.” Changes can invalidate earlier assumptions.
“A single combined score settles every tradeoff.” An average can hide failure of a necessary condition.
“Trustworthy means incapable of failure.” Real assessments must acknowledge uncertainty, limits, and remaining risk.

Business use

Ask suppliers and internal teams to connect important claims with evidence. Request the tested conditions, relevant limitations, and responsibilities when failures occur. Distinguish measured results from assurances based only on product descriptions.
Use the assessment to define deployment boundaries and improvement priorities. Document which requirements are necessary, which involve judgment, and which remain unresolved. Make the reasoning understandable to people responsible for the service and those affected by its use. Reassess when meaningful changes occur rather than relying on an unchanged marketing label.

Check your understanding

Why does Candidate A fail the buyer's stated evaluation?

Answer: It misses a separate access requirement despite meeting the answer-quality threshold.

Why is Candidate B's result limited?

Answer: The tests cover specified cases, not every possible failure.

What should accompany a trustworthiness claim?

Answer: Its intended use, supporting evidence, conditions, and limitations.

Related terms

Learn next

Select one AI service and write three specific claims that would justify reliance. For each, identify relevant evidence, an unresolved limitation, and a change that would require reassessment.

Sources

NIST, AI Risk Management Framework 1.0, AI risks and trustworthiness: https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/ NIST, AI Risk Management Framework 1.0, core functions for governing, mapping, measuring, and managing risk: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/

Previous: Trust region policy optimization Next: Trustworthy AI