Skip to main content

GFS vs HDFS architecture

Short answer

GFS is Google's proprietary, HDFS is open-source. Both have master (GFS: Master, HDFS: NameNode) and chunkservers/DataNodes. Chunk size: GFS 64MB, HDFS 128MB. Both replicate data thrice. Architecture: client contacts master for metadata, then reads/writes directly to chunkservers/DataNodes.

Compare GFS Vs HDFS and their corresponding architecture.

GFS (Google File System) is Google's proprietary distributed file system. HDFS (Hadoop Distributed File System) is the Apache open-source file system modelled on GFS.

BasisGFSHDFS
OwnerGoogle, proprietaryApache, open source
Master nodeMasterNameNode
Slave nodesChunk serversDataNodes
Unit of dataChunk, 64 MBBlock, 128 MB (64 MB in old versions)
Replication3 by default3 by default
WritesMultiple writers, record append supportedWrite-once-read-many, single writer; append supported later
Metadata logOperation log and checkpointsEditLog and FsImage
HeartbeatMaster and chunk servers exchange HeartBeatNameNode and DataNodes exchange heartbeats and block reports
LanguageC++Java
Processing partnerGoogle MapReduceHadoop MapReduce

Architecture

 GFS:                      HDFS:
 Client                    Client
  |  \                     |  \
  |  Master (metadata)      |  NameNode (metadata)
  |                         |
 Chunk servers             DataNodes
 [chunks x3]               [blocks x3]

In both, the client asks the master/NameNode for locations, then reads and writes data directly with the chunk servers/DataNodes, so the master is not a bottleneck for data transfer.

✓ Answer checked by Claude Opus 5.5 (high effort)

Related questions

All Information System chapter 8 questions →