AI Glossary / Governance, ethics and safety / Intermediate

Resilience

Definition

Resilience is the capacity of a system and its supporting organization to prepare for disruption, withstand or limit its effects, recover useful operation, and adapt when necessary. For an AI service, disruption may involve unavailable tools, corrupted or missing data, unexpected inputs, security incidents, or changes in operating conditions. Resilience concerns how the service behaves through adversity, not just how well it performs when everything is normal.

What it means

A resilient service does not have to continue every function unchanged during a problem. It may deliberately reduce functionality, pause a risky action, or move work to a safe alternative. The goal is to preserve appropriate essential outcomes and restore operation in a controlled way. This differs from merely restarting a model process. Users, queues, records, permissions, and downstream systems may also be affected. Recovery should therefore be defined in terms of the service's purpose and consequences rather than only whether a server is running again.

How it works

Resilience work identifies critical functions, dependencies, plausible disruptions, and acceptable degraded modes. The design may include isolation, redundancy, backups, input checks, fallback workflows, and clear escalation. Detection triggers a response that contains the problem and preserves important state. Recovery restores service while checking consistency and preventing duplicate or unsafe actions. Teams then investigate the incident and adapt procedures where needed. No single mechanism guarantees resilience: redundant components can share a failure, backups can be unusable, and automatic retries can worsen overload or repeat consequential actions.

Three practical examples

• A summarization service falls back to displaying source documents when its generation component is unavailable, making the limitation clear.
• A document-processing pipeline queues valid work during an outage and resumes without losing or duplicating completed records.
• An AI-assisted planning tool detects stale data, pauses automated recommendations, and provides a defined manual workflow until fresh data is available. These examples prioritize appropriate service behavior rather than uninterrupted automation at any cost.

Think of it this way

Think of a transport network that can reroute passengers after a line closes and then restore normal service safely. The network may run at reduced capacity while still meeting essential needs. The analogy is limited because AI disruptions can be silent or semantic, such as plausible outputs based on corrupted context. Resilience therefore requires checks on information and decisions as well as visible infrastructure recovery.

Worked example

Suppose a hypothetical intake service receives 60 jobs per hour. During a two-hour outage, 120 jobs accumulate in a durable queue. After recovery, the system processes 100 jobs per hour while new jobs continue arriving at 60 per hour. Its net backlog reduction is 40 jobs per hour, so clearing the 120-job backlog takes three additional hours, assuming constant rates and no retries or failures. Restoring the processor after two hours therefore does not mean all user work is immediately restored to normal. The team should define which recovery measure matters: service availability, backlog clearance, oldest-job delay, or another outcome. It should also verify that queued jobs remain valid and that any external actions are not duplicated.

Illustrative case study

Fictional case: Juniper Reports has a backup model endpoint but discovers during a rehearsal that both endpoints depend on the same unavailable data store. The team adds a clearly bounded degraded mode, tests restoration of the data dependency, and assigns responsibility for queued work. It measures recovery using completed user tasks rather than endpoint status alone. This invented case illustrates common-mode failures and the need to exercise the entire recovery path. It does not imply that every service needs multiple models or elaborate failover infrastructure.

Common misconceptions and mistakes

Resilience is not the same as invulnerability. A resilient system can experience failures while limiting their impact and recovering appropriately. It is related to reliability, robustness, and security, but none alone establishes it. Automatically switching to a less capable model can introduce new errors, so fallback quality needs evaluation. Restoring a backup can reintroduce stale or compromised data if verification is weak. Self-healing behavior is one possible mechanism, not a substitute for accountable incident handling. Sometimes the most resilient response is to stop a risky function and preserve evidence.

Business use

Set recovery objectives around the needs of users and the consequences of delay or error. Identify shared dependencies and test whether fallback paths are truly usable. Exercise restoration, replay, and reconciliation procedures with controlled scenarios. Keep users informed about degraded behavior and provide alternatives where practical. Record incidents and near misses, including failures that monitoring did not detect promptly. Review lessons after changes to models, integrations, or traffic. Ensure resilience measures do not bypass authorization or data-protection controls simply to restore throughput faster.

People also ask

Must a resilient AI system always remain fully operational?

No. A controlled degraded mode or safe pause can be appropriate.

Does a backup guarantee resilience?

No. It must be usable, sufficiently independent, and integrated into a tested recovery process.

How is resilience different from reliability?

Reliability concerns required operation without failure under stated conditions; resilience emphasizes handling, recovering from, and adapting to disruptions.

Check your understanding

A service restarts after an outage but has hours of queued work and duplicated downstream actions. Has full recovery been demonstrated?

Answer: No. Recovery must address the relevant user outcomes, backlog, and state consistency, not merely the restarted process.

Related terms

• Reliability
• Robustness
• Availability
• Fault tolerance
• Incident response
• graceful degradation
• disaster recovery
• recovery objective
• fallback
• common-mode failure

Learn next

Choose an AI workflow and remove one critical dependency in a controlled thought experiment. Specify what should stop, what can continue, how users are informed, and how recovery is verified. Calculate whether post-recovery capacity can clear the expected backlog.

More in Governance, ethics and safety

• Representation Bias - Representation bias occurs when the data used to develop or evaluate a system inadequately represent the population or conditions to which the system is...
• Representational harm - Representational harm is harm arising from how an AI system depicts, describes, categorizes, or makes people and social groups visible.
• Residual - A residual is the part of an observed value left unexplained by a fitted statistical model.
• Responsible AI - Responsible AI is the practice of developing, deploying, using, and governing AI with attention to its effects on people, society, and the environment.
• Retention limit - A retention limit is a defined boundary on how long data or records are kept for a particular purpose.
• Retraining - Retraining updates a previously trained machine-learning model through another training process using different training data.

Sources

NIST CSRC, resilience definitions and source contexts: https://csrc.nist.gov/glossary/term/resilience NIST AI RMF, secure and resilient AI characteristics: https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/ Avizienis and colleagues, original dependable-computing taxonomy: https://www.landwehr.org/2004-aviz-laprie-randell.pdf

Previous: Residual variance Next: Resistance