--- title: "Availability and fault tolerance in GFS" subject: "Big Data Technologies" chapter: 2 key: "semester_8/big_data_technologies:gfs-availability-fault-tolerance" source: https://www.bctengineer.com/q/semester_8/big_data_technologies/gfs-availability-fault-tolerance site: BCT Engineering --- # Availability and fault tolerance in GFS Big Data Technologies · Chapter 2: Google File System - Asked in: 2074 Bhadra, 5 marks - Video: [The Google File System (GFS) Explained | Distributed Systems Paper Summary](https://www.youtube.com/watch?v=3VksDXKxP98) — Nabadeep Das - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_8/big_data_technologies/gfs-availability-fault-tolerance - Whole chapter: https://www.bctengineer.com/md/semester_8/big_data_technologies/chapter-2 ## Question What is availability and fault tolerance in Google File System? ## Short answer Availability is achieved by fast recovery, three-way chunk replication, replicated master state, and shadow masters for read-only access. Fault tolerance uses re-replication after failures, chunk version numbers, per-block checksums, and the operation log for recovery. ## Answer Because GFS runs on thousands of commodity machines, component failure is the normal case. It therefore builds **high availability** (the system keeps serving requests) and **fault tolerance** (it keeps working correctly despite failures) into its design. #### Availability The system should remain usable even while parts of it fail or are being upgraded. - **Fast recovery:** master and chunkservers are designed to restore their state and restart in seconds, whatever the cause of shutdown. Clients retry. - **Chunk replication:** every chunk is stored on several chunkservers (three by default, configurable per file) on different racks. If one fails, reads go to another replica. - **Master replication:** the operation log and checkpoints are replicated on multiple machines. If the master fails, monitoring infrastructure starts a new master elsewhere. **Shadow masters** provide read-only access during master downtime. #### Fault tolerance - **Re-replication:** the master tracks the replica count through HeartBeat messages. When a chunkserver dies or a replica is corrupted, it copies the chunk to other servers to restore the replication level. - **Chunk version numbers** detect stale replicas that missed mutations while a server was down; stale replicas are garbage collected. - **Data integrity by checksums:** each 64 KB block of a chunk has a 32-bit checksum, verified on reads. A corrupted replica is replaced from a good one. - **Operation log + checkpoints:** metadata changes are logged durably before being visible to clients, so the master state can be recovered. - **Diagnostic logging** helps in problem isolation and debugging. #### Example If a chunkserver holding chunk C crashes, the master notices missing heartbeats, sees C has only 2 replicas, and instructs another chunkserver to copy it from a surviving replica. Clients meanwhile read from the remaining replicas without interruption. ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. A chunkserver holding chunk C crashes and C now has only 2 replicas. What happens? A. The data is lost B. The master notices missing heartbeats and instructs another chunkserver to copy C from a surviving replica C. Clients stop reading the file D. The client re-creates the chunk Answer: B. The master tracks replica counts through HeartBeat and restores the replication level. (Key point: Re-replication) 2. What is the purpose of chunk version numbers? A. To detect corrupted data in a chunk B. To choose the nearest chunkserver C. To compress chunks D. To detect stale replicas that missed mutations while a server was down Answer: D. A replica with an old version number is stale and is garbage collected; checksums are for corruption. (Key point: Chunk version numbers) 3. How does GFS keep the master available? A. It uses no replication B. Operation log and checkpoints are replicated, a new master can be started, and shadow masters give read-only access C. Chunkservers act as masters D. Clients keep the metadata Answer: B. Replicated logs allow a new master to start elsewhere, and shadow masters provide read-only access meanwhile. (Key point: Master availability) ## Related in Big Data Technologies - [Google File System architecture](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Agfs-architecture) — Big Data Technologies, chapter 2, asked 2× - [GFS vs other file systems](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Agfs-differ-other-fs) — Big Data Technologies, chapter 2 ## Related in other subjects - [Replication, fault tolerance, scalability](https://www.bctengineer.com/question.md?key=semester_7%2Fdistributed_system%3Areplication-scalability-ft) — Distributed System, chapter 7 - [Fault tolerance and types of faults](https://www.bctengineer.com/question.md?key=semester_7%2Fdistributed_system%3Afault-types) — Distributed System, chapter 9 - [Fault tolerance techniques](https://www.bctengineer.com/question.md?key=semester_7%2Fdistributed_system%3Afault-tolerance-techniques) — Distributed System, chapter 9 - [Fault tolerance and redundancy for reliability](https://www.bctengineer.com/question.md?key=semester_5%2Finstrumentation_ii%3Afault-tolerance-schemes) — Instrumentation II, chapter 7, asked 4×