Large fixed-size chunks in GFS

GFS stores each file as fixed-size chunks of 64 MB, much larger than the few-KB blocks of ordinary file systems. Each chunk is a Linux file on a chunkserver, identified by a 64-bit chunk handle.

Why large, fixed-size chunks (advantages)

1. Less metadata: fewer chunks mean the master's table is small. One 64 MB chunk needs under 64 bytes of metadata, so all metadata fits in the master's memory, enabling fast operations.
2. Fewer client-master interactions: reads and writes on the same chunk need only one request to the master. A client can cache chunk locations for a multi-TB working set.
3. Reduced network overhead: a client doing many operations on a chunk can keep a persistent TCP connection to one chunkserver.
4. Suits the workload: applications read/write multi-GB files sequentially, so large chunks match the access pattern.
5. Simple allocation and lookup: fixed size makes it easy to compute the chunk index from a byte offset (offset / 64 MB), and simplifies replica placement, load balancing and re-replication.
6. Lazy space allocation: chunk files are allocated only as needed, so internal fragmentation is avoided.

Demerits

1. Hot spots: a small file may consist of one chunk. If many clients access it at the same time (e.g. an executable file being started by hundreds of machines), the chunkservers holding it are overloaded. GFS mitigates this with a higher replication factor and staggered start times.
2. Wasted space for small files: a small file still takes a whole chunk entry, and with many small files the master's metadata grows without the benefit of large chunks (lazy allocation limits the disk waste, not the metadata).
3. Inefficiency for random access of small data: a whole large chunk must be managed even for small reads.
4. Slower re-replication and recovery: copying a 64 MB chunk takes more time and bandwidth than a small block.
5. Load imbalance: a few big chunks make it harder to balance load evenly.
6. Unsuitable for many small files and for low-latency random workloads.

Summary

Large fixed-size chunks reduce metadata size and master traffic and suit large sequential files, but they cause hot spots for small, popular files and are poor for small-file workloads. GFS reduces the hot-spot problem by higher replication of popular files and by throttling start-up reads.