Google File System architecture

Google File System (GFS) is a scalable distributed file system developed by Google for large, data-intensive applications. It runs on thousands of cheap commodity machines and provides fault tolerance and high aggregate throughput. It is optimised for huge files, sequential reads, and append-heavy writes.

Architecture

A GFS cluster has a single master, many chunkservers, and many clients.

  +--------+  1. file name, chunk idx  +-----------+
  | Client |-------------------------->|  Master   |
  |  (app) |<--------------------------| metadata  |
  +---+----+  2. chunk handle+locations+-----+-----+
      |                                   heartbeat
      | 3. data read/write                    |
  +---v-------+  +-----------+  +-------------v+
  |Chunkserver|  |Chunkserver|  | Chunkserver  |
  |  (Linux   |  |           |  |              |
  |  files)   |  |           |  |              |
  +-----------+  +-----------+  +--------------+

Components

1. Master
   • One master holds all metadata: namespace, file-to-chunk mapping, access control and the locations of chunk replicas.
   • Namespace and mapping are persisted in an operation log with checkpoints, and replicated. Chunk locations are not stored permanently; the master asks chunkservers at start-up and through regular HeartBeat messages.
   • It also controls chunk lease management, garbage collection, re-replication and chunk migration.
2. Chunkservers
   • Store chunks as Linux files on local disks. Each chunk is a fixed 64 MB block with a unique 64-bit chunk handle.
   • Each chunk is replicated on (by default) three chunkservers. Checksums protect against corruption.
3. Client
   • A library linked into the application. It asks the master for metadata, then talks directly to chunkservers for the data, so the master is not in the data path. Clients do not cache file data, but cache metadata.

Features

• Large chunk size reduces metadata and client-master traffic.
• Relaxed consistency model with atomic record append.
• Snapshot and replication for fault tolerance.
• Control flow (master) is separated from data flow (chunkservers).

Basic read

Client converts (file, byte offset) to a chunk index, asks the master, receives the chunk handle and replica locations, and then reads from the nearest chunkserver.

Data and metadata at a glance

Fault tolerance and consistency

• Replicas are placed on different racks; the master re-replicates when copies are lost.
• Writes use a lease-holding primary chunkserver that orders mutations; record append is atomic at least once.
• Heartbeat messages carry state and commands between master and chunkservers.