AI Glossary / Governance, ethics and safety / Intermediate

Reliability

Definition

Reliability is the ability of a system to perform its required function without failure over a specified period and under stated conditions. For AI, it concerns the dependable operation of the whole system in its expected use, including data processing, model behavior, software, infrastructure, and relevant human handoffs. A reliability claim is incomplete unless the function, conditions, time period, and meaning of failure are clear.

What it means

A system can be available and still give unusable answers, or make accurate predictions when running but frequently fail to respond. Reliability therefore cannot be inferred from model accuracy or uptime alone. The requirement might be that a document workflow produces a correctly formatted, checked result within a deadline, or that a monitoring service maintains a defined function throughout a shift. Different requirements create different failure definitions. Consistent repetition of an incorrect answer is not sufficient evidence of reliability for a task requiring correctness.

How it works

Reliability work defines the required service, identifies potential failures, gathers evidence under relevant conditions, and monitors continued operation. Engineers examine dependencies, failure handling, version changes, data quality, and recovery procedures. In time-to-failure analysis, a reliability or survival function describes the probability of continuing without the defined failure beyond a given time. In service evaluation, teams may also track task-success or incident measures, but these should be labeled precisely rather than assumed to be interchangeable. Evidence from one environment or short test period may not support a long-term claim in a different environment.

Three practical examples

• A document assistant is evaluated for completing valid requests within a stated deadline while correctly flagging unsupported files.
• An AI monitoring service is tested for continued operation during expected network interruptions and recovery when a dependency returns.
• A prediction pipeline is monitored for missing or stale input data that could silently degrade its required output. These examples consider system behavior, not just the model's score on a static benchmark.

Think of it this way

Think of a delivery service that must bring the right parcel to the right place within an agreed window. A truck that starts every morning is useful evidence, but it does not establish the whole service's reliability. The analogy is limited because AI failures may be subtle, such as plausible but incorrect outputs or degraded performance for particular inputs. Detecting them can require outcome checks as well as operational alarms.

Worked example

Suppose a hypothetical assistant receives 1,000 valid requests. It responds to 990 before the deadline. Of those timely responses, 950 meet the defined content and format requirements. The timely-response rate is 99%, but the observed end-to-end success rate is 950/1,000 = 95%. Conditional on a timely response, the success rate is 950/990, approximately 95.96%. These numbers describe different denominators and should not be substituted for one another. They also do not directly estimate the probability of operating without any failure for an entire month. If failures cluster during one outage or affect a particular document type, the aggregate request rate hides that structure. The test should preserve timing, failure categories, and conditions needed to interpret the result.

Illustrative case study

Fictional case: Rowan Intake reports excellent model accuracy but receives complaints about incomplete overnight processing. Investigation finds that a data-feed interruption leaves old records in place without an alert. The team adds freshness checks, a defined degraded mode, and an operational owner for failed runs. It evaluates the revised workflow under expected interruption scenarios. This invented case illustrates that reliability failures can arise outside the predictive model and that a high benchmark score cannot substitute for end-to-end evidence.

Common misconceptions and mistakes

Reliability is not identical to availability, robustness, resilience, or safety. Availability concerns readiness for use; robustness concerns performance under specified variation; resilience concerns handling and recovering from disruption; safety concerns avoiding unacceptable harm. These properties can support one another but require distinct evidence. Redundancy does not automatically improve reliability if components share the same failure mode. A reliable-looking average can hide severe rare failures, and a deterministic system can fail consistently. A lack of observed incidents in a small test is not proof that the failure probability is zero.

Business use

Define reliability requirements in terms of user outcomes and operating conditions. Track failures at appropriate stages and retain enough context to identify clustering, dependency problems, and affected groups. Test fallback and recovery procedures rather than only documenting them. Establish monitoring that can detect content-quality failures as well as outages where feasible. Review reliability evidence after model, data, infrastructure, or workflow changes. Communicate the scope and uncertainty of the claim, including unsupported conditions. Set escalation and rollback responsibilities so a known reliability problem leads to a concrete response.

People also ask

Does high accuracy mean an AI service is reliable?

Not necessarily. The service can fail through delays, outages, invalid inputs, or surrounding workflow problems.

Can a system be reliable but unsafe?

Yes. It could consistently perform a function that creates unacceptable harm; safety requires separate assessment.

Is reliability a single permanent score?

No. It depends on requirements, conditions, time, and the evidence available, and it can change as the system or environment changes.

Check your understanding

A service answers 99% of requests on time, but only 95% of all requests receive a timely and acceptable result. Which figure describes the observed complete-task success rate?

Answer: 95%. The 99% figure describes timely response only and does not include the full quality requirement.

Related terms

• Availability
• Robustness
• Resilience
• Safety
• Dependability
• Survival function
• failure mode
• service-level objective
• monitoring
• operational reliability

Learn next

Write a reliability requirement for a specific AI workflow that names the function, conditions, time window, and failure criterion. Identify one silent failure that an uptime monitor would miss and design an additional check that could reveal it.

More in Governance, ethics and safety

• Referential transparency - Referential transparency concerns whether an expression can be replaced by an equivalent expression or value without changing the meaning or relevant...
• Regularized empirical risk minimization - Regularized empirical risk minimization, or RERM, is a model-training principle that balances average loss on training data with a penalty expressing a...
• Relative risk or risk ratio - Relative risk, or risk ratio, compares the probability of a specified event in one group or condition with its probability in another.
• Reporting bias - Reporting bias is a systematic difference between what people record or describe and what actually occurs in the world.
• Representation Bias - Representation bias occurs when the data used to develop or evaluate a system inadequately represent the population or conditions to which the system is...
• Representational harm - Representational harm is harm arising from how an AI system depicts, describes, categorizes, or makes people and social groups visible.

Sources

NIST AI RMF, valid and reliable AI systems: https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/ NIST Engineering Statistics Handbook, reliability or survival function: https://www.itl.nist.gov/div898/handbook/apr/section1/apr122.htm Avizienis and colleagues, Basic Concepts and Taxonomy of Dependable and Secure Computing: https://www.landwehr.org/2004-aviz-laprie-randell.pdf

Previous: Relevance Next: Relief (feature selection)