Skip to main content

Faults, failures, errors; resilience

Short answer

A fault is a defect, an error is an incorrect state caused by a fault, and a failure is a service deviation due to an error. Handling faults uses avoidance, detection, masking through redundancy, and recovery. Process resilience uses replicated groups (flat or hierarchical) to mask failures; K+1 members survive K crashes, 2K+1 survive K Byzantine failures.

What do you mean by faults, failures and errors? How do you handle faults in Distributed System? Explain process resilience approach in brief.

Fault, error, failure

  • Fault: the cause of a problem, a defect in hardware, software or a link (e.g. a bug, a broken wire).
  • Error: the part of the system state that is wrong because of a fault (e.g. a wrong value in a variable).
  • Failure: the system does not deliver its specified service because of an error; this is visible to the user.

A fault causes an error, which may lead to a failure. Example: a software bug (fault) produces a wrong total (error), so the printed report is wrong (failure).

Handling faults

  • Fault avoidance/prevention: good design and testing.
  • Fault detection: heartbeats, timeouts, checksums.
  • Fault masking by redundancy: replication, voting such as TMR, retransmission.
  • Recovery: backward recovery (checkpoints, logs) and forward recovery (error-correcting codes).
  • Reliable communication and atomic commit protocols.

Process resilience

Process resilience means protecting a system against process failures by replicating a process into a process group that works as a single logical process: if a member fails, another takes over, so the service continues.

Group organization

  • Flat group: all processes are equal and decide collectively (e.g. by voting). There is no single point of failure, but decision making is more complex and has more overhead.
  • Hierarchical group: one coordinator decides and the workers carry out its instructions. It is simple and fast, but if the coordinator fails, the group must elect a new one.
Flat group        Hierarchical group
 P1--P2            Coordinator
  \  /              /   |   \
   P3--P4         W1   W2   W3

Failure masking and group size: a group that can survive K failures is K-fault tolerant. With crash failures, K+1 members are enough; with arbitrary (Byzantine) failures, 2K+1 members are needed (with voting), and 3K+1 to reach agreement.

Agreement in faulty systems: members must agree on decisions even when some members are faulty (consensus, Byzantine agreement). Membership changes need a group membership service that adds and removes members and detects failed ones (by pinging and timeouts).

Example: a replicated database group of 3 servers answers client requests; if one crashes, the other two keep serving.

✓ Answer checked by Claude Opus 5.5 (high effort)

Related questions

All Distributed System chapter 9 questions →