AI Glossary / Governance, ethics and safety / Intermediate
Fault tolerance
Definition
Fault tolerance is a system's ability to continue providing an acceptable service when one or more components fail. It is achieved through deliberate design, such as redundancy, failure detection, isolation, and controlled recovery. The claim must specify which faults are covered and what level of service remains acceptable. Fault tolerance does not mean that every possible failure is prevented or invisible to users.
What it means
An AI application depends on more than a model. It may use servers, storage, a document index, an authentication service, and external tools. A failure in any necessary component can interrupt the overall workflow even if the model itself is operating normally.
Fault tolerance asks what happens after a component stops working or behaves incorrectly. Can another component take over? Can the system provide a reduced but clearly described service? Can it avoid spreading the failure to otherwise healthy parts?
The appropriate behavior depends on the task. A search feature might tolerate temporarily losing suggestions, while a workflow that records an important transaction needs strong protection against duplicate or incomplete actions.
How it works
Start by defining the service requirement and the faults the design must tolerate. Identify dependencies and common failure points, then provide suitable detection and response mechanisms. Redundant components can take over work if routing and state management allow it.
Use timeouts and bounded retry policies so an unavailable dependency does not hold resources indefinitely. Isolate failing components and consider whether a reduced-function fallback is appropriate. The fallback must preserve the task's essential requirements rather than merely returning something that looks successful.
Test the design under controlled failures and realistic load. Verify both continuity and correctness, including how the system returns to normal operation. Two copies of a component do not provide independent protection if both depend on the same failed service or carry the same defective configuration.
Three practical examples
1. An AI inference service routes requests away from a failed worker to healthy workers with sufficient spare capacity.
2. A document portal continues to offer ordinary search when an optional generated-summary feature is unavailable, clearly distinguishing the available functionality.
3. A workflow records completed action identifiers so retrying a request after a network interruption does not unintentionally repeat an already completed action.
Think of it this way
Think of a building with a backup power supply for essential services. When the main supply fails, selected functions can continue, provided the backup has sufficient capacity and the switching mechanism works.
The analogy has limits. Software faults may corrupt shared data or affect multiple replicas at once, and detecting a partial failure can be difficult. A second component is useful only when the whole arrangement, including shared dependencies, can support the intended recovery behavior.
Worked example
Suppose an illustrative service has three identical workers, each tested to handle 40 units of a defined workload per second. The normal demand is 70 units per second. Total nominal capacity is 120 units, and after one worker fails, 80 units remain.
On this simplified capacity calculation, the remaining workers can handle the demand with 10 units of spare capacity. If normal demand were 90 instead, the same one-worker failure would leave a shortfall of 10 units per second.
The calculation does not prove fault tolerance. Routing must detect the failure, requests in progress must be handled correctly, and shared dependencies must remain available. Real AI requests also vary in computational cost, so the workload unit must represent the tested mix rather than assuming all requests consume equal resources.
Illustrative case study
Fictional case: Stonebridge Help runs an assistant that summarizes internal documents. A controlled test disables one inference worker while keeping a representative stream of requests running.
The team checks whether traffic moves to healthy workers, whether timeouts remain acceptable, and whether duplicate retries cause repeated downstream actions. A second test disables the document index and verifies that the assistant reports the unavailable evidence instead of inventing a document-based answer.
This fictional exercise distinguishes service continuity from merely producing text. It reports no uptime improvement and does not claim coverage of faults the team has not tested.
Common misconceptions and mistakes
Do not confuse backups with immediate continuity. A backup may support later recovery while leaving the live service unavailable during restoration.
Do not assume redundancy protects against every fault. A common software bug, shared power dependency, or incorrect configuration can affect multiple copies together.
Avoid unlimited retries, which can add load to an already struggling service. Also distinguish hardware or service failure from model-quality failure: serving a hallucinated answer reliably is not the same as preserving correct application behavior. The acceptable response to missing evidence may be a clear limitation rather than a lower-quality substitute.
Business use
Fault-tolerance planning helps organizations make service commitments that match their architecture and operational capacity. Decide which functions must continue, which can degrade, and which should stop when essential guarantees cannot be maintained.
Include spare capacity, monitoring, recovery procedures, and controlled failure tests in the cost assessment. Document the faults covered and the observable success criteria. Review the design when adding new model services or tools, because a new shared dependency can undermine resilience that existed in an earlier version.
People also ask
Does fault tolerance mean zero downtime?
No. It describes behavior under specified faults and requirements. Detection, switching, and recovery can still cause disruption.
Are two replicas automatically fault tolerant?
No. They need effective routing, appropriate state handling, enough capacity, and protection against relevant shared failures.
Is a fallback model always the right response?
No. Its behavior must meet the task's requirements. If it cannot preserve essential quality or constraints, the application may need a different limited-service response.
Check your understanding
Three workers can handle the normal load, but two cannot. Does adding a third worker establish tolerance of any one worker's failure?
Answer: Not at that load. The remaining capacity is insufficient. Fault tolerance requires acceptable operation after the specified failure, not only during normal conditions.
Related terms
- Reliability
- Availability
- Redundancy
- Resilience
- failover
- graceful degradation
- fault isolation
- recovery
- timeout
- idempotency
Learn next
Learn redundancy and failover next, then study common-cause failures and graceful degradation. Map one AI workflow's dependencies and define what users should experience when each dependency becomes unavailable before choosing a recovery mechanism.
More in Governance, ethics and safety
- Facial recognition - Facial recognition is the automated comparison of facial information to help verify or identify a person.
- Fairness constraint
- Fairness metric - A fairness metric is a quantitative measure used to examine whether an AI system meets a specified fairness criterion.
- Favorable label - A favorable label is the outcome category designated as desirable or beneficial for the people considered in a fairness analysis.
- Feedback Loop Bias - Feedback Loop Bias occurs when a system's outputs influence the behavior or observations that later become its learning data, reinforcing a distorted pattern.
- Fraud detection - Fraud detection is the process of identifying activity that may involve deception for an improper benefit, such as unauthorized transactions or false claims.
Sources
NIST CSRC Glossary, fault tolerance: https://csrc.nist.gov/glossary/term/fault_tolerance
Google Site Reliability Engineering, Handling Overload: https://sre.google/sre-book/handling-overload/
Primary definitions and the engineering discussion of capacity, degraded service, and resource restrictions were read. The worker arithmetic, workflow suggestions, and Stonebridge Help case are original teaching material. No universal availability guarantee is implied.
Previous: Fat Relation Next: Favorable label