--- title: "Google File System architecture" subject: "Big Data Technologies" chapter: 2 key: "semester_8/big_data_technologies:gfs-architecture" source: https://www.bctengineer.com/q/semester_8/big_data_technologies/gfs-architecture site: BCT Engineering --- # Google File System architecture Big Data Technologies · Chapter 2: Google File System - Asked in: 2074 Bhadra, 8 marks; 2073 Bhadra, 5 marks (asked 2 times) - Video: [Cloud Computing | Tutorial #28 | Google File System](https://www.youtube.com/watch?v=CTdc67MMaL8) — RANJI RAJ - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_8/big_data_technologies/gfs-architecture - Whole chapter: https://www.bctengineer.com/md/semester_8/big_data_technologies/chapter-2 ## Question Explain the architecture of Google File System (GFS). ## Short answer GFS uses a single master holding metadata and multiple chunkservers storing 64 MB chunks with triple replication. Clients contact the master for chunk locations and then directly read/write data to chunkservers, separating control and data flow for scalability and fault tolerance. ## Answer **Google File System (GFS)** is a scalable distributed file system developed by Google for large, data-intensive applications. It runs on thousands of cheap commodity machines and provides fault tolerance and high aggregate throughput. It is optimised for huge files, sequential reads, and append-heavy writes. #### Architecture A GFS cluster has a **single master**, many **chunkservers**, and many **clients**. ```text +--------+ 1. file name, chunk idx +-----------+ | Client |-------------------------->| Master | | (app) |<--------------------------| metadata | +---+----+ 2. chunk handle+locations+-----+-----+ | heartbeat | 3. data read/write | +---v-------+ +-----------+ +-------------v+ |Chunkserver| |Chunkserver| | Chunkserver | | (Linux | | | | | | files) | | | | | +-----------+ +-----------+ +--------------+ ``` #### Components 1. **Master** - One master holds all metadata: namespace, file-to-chunk mapping, access control and the locations of chunk replicas. - Namespace and mapping are persisted in an **operation log** with checkpoints, and replicated. Chunk locations are not stored permanently; the master asks chunkservers at start-up and through regular **HeartBeat** messages. - It also controls chunk lease management, garbage collection, re-replication and chunk migration. 2. **Chunkservers** - Store chunks as Linux files on local disks. Each chunk is a fixed **64 MB** block with a unique 64-bit **chunk handle**. - Each chunk is replicated on (by default) **three** chunkservers. Checksums protect against corruption. 3. **Client** - A library linked into the application. It asks the master for metadata, then talks **directly** to chunkservers for the data, so the master is not in the data path. Clients do not cache file data, but cache metadata. #### Features - Large chunk size reduces metadata and client-master traffic. - Relaxed consistency model with atomic record append. - Snapshot and replication for fault tolerance. - Control flow (master) is separated from data flow (chunkservers). #### Basic read Client converts (file, byte offset) to a chunk index, asks the master, receives the chunk handle and replica locations, and then reads from the nearest chunkserver. #### Data and metadata at a glance | Item | Held by | Notes | |---|---|---| | Namespace, file-to-chunk map | Master (memory + log) | Persistent via operation log | | Chunk replica locations | Master (memory) | Rebuilt from chunkserver reports | | Chunk data | Chunkservers | 64 MB, 3 replicas by default | | Checksums | Chunkservers | Per 64 KB block | #### Fault tolerance and consistency - Replicas are placed on different racks; the master re-replicates when copies are lost. - Writes use a lease-holding **primary** chunkserver that orders mutations; **record append** is atomic at least once. - Heartbeat messages carry state and commands between master and chunkservers. ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. What are the three types of components in a GFS cluster? A. NameNode, DataNodes, JobTracker B. A single master, many chunkservers and many clients C. Many masters and one chunkserver D. Primary, secondary and tertiary masters only Answer: B. A GFS cluster has one master, many chunkservers and many clients. (Key point: GFS components) 2. What are the default chunk size and replication in GFS? A. 128 MB and 2 replicas B. 64 KB and 3 replicas C. 4 KB and 1 replica D. 64 MB and 3 replicas Answer: D. Chunks are fixed 64 MB blocks, each replicated on three chunkservers by default. (Key point: Chunk size and replication) 3. How does the master learn the locations of chunk replicas? A. By asking chunkservers at start-up and through HeartBeat messages, not storing them permanently B. From the operation log only C. From the client cache D. From checksums Answer: A. Chunk locations are not stored persistently; they are rebuilt from chunkserver reports. (Key point: Chunk locations at master) 4. How does a client read data in GFS? A. All data flows through the master B. It gets data from the master's disk C. It caches file data and never contacts the master D. It asks the master for metadata, then reads directly from a chunkserver Answer: D. The master supplies only chunk handle and locations; data goes directly between client and chunkserver, so the master is not a bottleneck. (Key point: Master out of data path) ## Related in Big Data Technologies - [GFS vs other file systems](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Agfs-differ-other-fs) — Big Data Technologies, chapter 2 - [Availability and fault tolerance in GFS](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Agfs-availability-fault-tolerance) — Big Data Technologies, chapter 2 - [GFS master role in read/write](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Agfs-master-read-write-flow) — Big Data Technologies, chapter 2 ## Related in other subjects - [GFS vs HDFS architecture](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Agfs-vs-hdfs) — Information System, chapter 8 - [DFS requirements and file service architecture](https://www.bctengineer.com/question.md?key=semester_7%2Fdistributed_system%3Adfs-requirements-architecture) — Distributed System, chapter 2 - [File system layout, i-node, backup](https://www.bctengineer.com/question.md?key=semester_6%2Foperating_system%3Afs-layout-inode-backup) — Operating System, chapter 5 - [File system layout; i-node vs linked list](https://www.bctengineer.com/question.md?key=semester_6%2Foperating_system%3Afs-layout-inode-vs-linked-list) — Operating System, chapter 5