Does a fail-stop component restart itself?

The basic abstraction concerns halting. Restart and recovery behavior require additional specification.

Is fail-stop the same as Byzantine failure?

No. Byzantine models allow arbitrary faulty behavior; fail-stop constrains the failure to halting rather than visible erroneous computation.

Does stable storage mean ordinary memory?

No. In the classical model it is a separate abstraction whose contents survive the specified failures.

AI Glossary / Data engineering and architecture / Intermediate

Fail-stop

Definition

Fail-stop describes a failure behavior in which a component halts instead of continuing to produce externally visible erroneous results. In the classical fail-stop processor abstraction, other processors can determine that it has halted, and designated stable storage survives the failure.

What it means

The purpose is to simplify reasoning about failures. A stopped component is unavailable, but it does not continue corrupting shared information or sending arbitrary answers. Other components can focus on recovering unfinished work instead of deciding which of its apparently valid outputs to trust.

Terminology is sometimes used loosely for any crash. The stronger abstraction matters: an ordinary process that becomes silent across a network is not automatically known to have stopped. It may be slow, disconnected or still operating elsewhere.

Fail-stop also differs from fail-safe. Fail-safe concerns whether a system enters an acceptable condition when something goes wrong. Halting one component is not necessarily safe for the wider process. Nor does fail-stop mean fault-free: failures still happen, but their allowed visible behavior is constrained.

How it works

A model specifies normal operations and the allowed failure transition. Under fail-stop assumptions, a failure causes halting rather than an incorrect state change visible outside the component. Classical formulations also distinguish volatile state, which can be lost, from stable storage used for recovery.

A surviving processor can use recorded state and a recovery protocol to continue the work. The protocol must distinguish completed operations from unfinished ones. Otherwise, a clean halt followed by an unsafe restart can still produce duplicate effects.

Real implementations approximate the abstraction using mechanisms appropriate to their fault assumptions. Detection, redundant execution or protected storage can contribute, but no single mechanism proves that every possible error becomes a clean halt. Common defects or failures outside the assumed boundary can invalidate the claim.

The detection assumption deserves particular attention in distributed systems. A timeout reports that an expected response did not arrive within a chosen period. Without additional timing or system assumptions, it cannot establish with certainty that the remote process has stopped.

Three practical examples

A fault-tolerance model treats a worker as stopping on failure and recovers its unfinished calculation from a checkpoint.
A storage service design separates a failed processor's volatile state from durable records needed by a replacement processor.
A distributed-algorithm review checks whether its proof assumes reliably detectable halts, rather than merely missing heartbeat messages.

Think of it this way

Think of a measuring instrument that switches off when it detects an internal fault instead of displaying a plausible but incorrect value. The absence of a reading is inconvenient, but a misleading reading could be harder to manage.

The analogy has limits: a real instrument may fail without detecting its own defect. The fail-stop abstraction states an expected failure behavior; it is not evidence that an implementation always achieves it.

Worked example

Recovering work after a clean halt

An invented batch job processes 100 records. A stable checkpoint confirms that records one through forty are complete. The worker then processes records forty-one through fifty in volatile memory and halts before saving another checkpoint or publishing those ten results.

A replacement reads the checkpoint and starts from record forty-one. It performs sixty records of work to complete the batch. Ten of those computations repeat work lost in the halt, while fifty concern records the original worker had not reached.

This is correct only under the stated publication assumptions. If the first worker had already sent external effects for records forty-one through fifty, replaying them could duplicate those effects. Recovery would then need durable completion information or a mechanism that makes retries safe.

A clean halt therefore simplifies the fault model, but it does not by itself provide exactly-once business outcomes. The recovery protocol and the boundary of committed effects still matter.

Illustrative case study

A fictional document-processing company uses workers to extract fields from uploaded forms. Its design review initially calls every worker crash fail-stop and assumes that replacement workers can safely repeat interrupted jobs.

Testing reveals a gap: a worker can submit an extracted record to another service and then stop before recording that submission locally. Repeating the job creates a second record even though the worker never produced corrupted extraction data.

The team introduces a stable operation identifier and a receiving-side duplicate check. It also records that network silence is only a failure suspicion, not proof that a worker has halted. The case illustrates two separate concerns: constraining failure behavior and recovering application effects correctly.

Common misconceptions and mistakes

Any crash satisfies the full fail-stop abstraction. Reliable detection and state assumptions may also be required.
Fail-stop means the service remains available. The component stops; continued service requires additional arrangements.
A timeout proves the remote component has halted. Delay or disconnection can produce the same observation.
Clean halting makes retries automatically safe. Previously completed external effects still need appropriate handling.

Business use

The concept helps teams examine what their resilience design assumes. Ask which faults are covered, what observers can know, which state survives and how unfinished operations are recovered.

For services supporting AI workflows, a worker restart can affect model calls, database updates and notifications differently. Recomputing a local value may be harmless while repeating an external action is not. Define those boundaries before claiming reliable recovery.

When evaluating a supplier's fail-stop claim, request the stated fault model and supporting evidence. A successful demonstration of one controlled crash does not establish behavior under silent corruption, shared software defects or every storage failure.

Check your understanding

A worker becomes unreachable for five seconds. Does that observation alone prove a fail-stop halt?

Answer: No. It may have halted, but delay or a network problem can look the same. A reliable conclusion requires assumptions or mechanisms beyond the timeout itself.

Related terms

Failure detector
Checkpoint
Crash failure
fault model
Byzantine failure
stable storage
recovery protocol

Learn next

Read failure detector and checkpointing. Trace one interrupted operation through your system and identify what is known, what is merely suspected and which external effects may already have occurred.

Failure detector

Sources

Schlichting and Schneider, Fail-Stop Processors: An Approach to Designing Fault-Tolerant Computing Systems: https://pages.cs.wisc.edu/~remzi/Classes/739/Papers/fail-stop.pdf. Original research defining the processor abstraction and recovery approach.

Cornell University, Byzantine Generals In Action: Implementing Fail-Stop Processors: https://ecommons.cornell.edu/entities/publication/acfcb50e-c796-4604-9118-f66643c1fbe7. Primary research record describing halting, failure detection and stable storage assumptions. The batch calculation and document-processing case are original illustrations.

Previous: Factuality Next: Failure bias