---
title: "DFS definition and HDFS client write"
subject: "Big Data Technologies"
chapter: 2
key: "semester_8/big_data_technologies:define-dfs-hdfs-write"
source: https://www.bctengineer.com/q/semester_8/big_data_technologies/define-dfs-hdfs-write
site: BCT Engineering
---

# DFS definition and HDFS client write

Big Data Technologies · Chapter 2: Google File System

- Asked in: 2075 Bhadra, 10 marks
- Video: [HDFS Architecture 🔥🔥](https://www.youtube.com/watch?v=oWefH_4ftUY) — Perfect Computer Engineer
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_8/big_data_technologies/define-dfs-hdfs-write
- Whole chapter: https://www.bctengineer.com/md/semester_8/big_data_technologies/chapter-2

## Question

Define DFS. How client writes data in HDFS? Explain with the help of suitable block diagram.

## Short answer

A DFS stores files across networked machines and presents them as one local file system. In HDFS, the client contacts the NameNode for a DataNode pipeline, streams packets to the first DataNode, which forwards them along the pipeline; acknowledgements return through the chain.

## Answer

#### Distributed File System (DFS)
A **DFS** is a file system whose files are stored across several networked machines but accessed by users and programs as though they were on one local file system. It gives location transparency, replication, scalability and fault tolerance. Examples: NFS, AFS, GFS and HDFS.

#### HDFS client write process
HDFS has a **NameNode** (master, holds metadata) and many **DataNodes** (store blocks, default block size 128 MB, replication factor 3). The client writes as follows.

1. The client calls `create()` on the DistributedFileSystem object. It contacts the NameNode, which checks permissions and that the file does not already exist, and records the new file.
2. The client receives an FSDataOutputStream. As the client writes, data is split into packets and put in an internal **data queue**.
3. For each block, the NameNode selects a list of DataNodes (a **pipeline**) to hold the three replicas, using rack awareness (first replica on the client's node/rack, second on a different rack, third on another node of that second rack).
4. The client sends packets to the first DataNode, which stores them and forwards them to the second, which forwards them to the third.
5. Each DataNode sends an **acknowledgement** back along the pipeline. Packets move from the data queue to an **ack queue** until acknowledged by all.
6. When a block is full, the client asks the NameNode for the next set of DataNodes and repeats.
7. When writing finishes, the client calls `close()`, flushes the remaining packets, waits for acknowledgements, and tells the NameNode the file is complete.

```text
 +--------+ 1 create  +----------+
 | Client |---------->| NameNode |
 | (JVM)  |<----------| metadata |
 +---+----+ 2 DN list +----------+
     | 3 write packets
     v
 +--------+  4 forward  +--------+  forward  +--------+
 |DataNode|------------>|DataNode|---------->|DataNode|
 |   1    |<------------|   2    |<----------|   3    |
 +--------+  5 ack      +--------+    ack    +--------+
     6 close / complete --> NameNode
```

If a DataNode fails during writing, the pipeline is closed, the failed node is removed, the remaining packets are put back on the data queue, and the NameNode later re-replicates the block to maintain the replication factor.

#### Features of a DFS
- **Transparency:** location, access and replication are hidden from the user.
- **Scalability:** more storage by adding nodes.
- **Fault tolerance:** replicas keep data available if a node fails.
- **Concurrent access** by many clients.

#### Rack awareness in write
The default placement is: replica 1 on the writer's node (or a random node if the client is outside the cluster), replica 2 on a node in a different rack, replica 3 on another node in the same rack as replica 2. This balances reliability against cross-rack bandwidth.

#### Key points
- Only the **metadata** passes through the NameNode; data goes straight from the client to DataNodes.
- A block is considered written when the minimum number of replicas (dfs.namenode.replication.min, default 1) have acknowledged it; remaining replicas are created asynchronously if needed.
- HDFS files are write-once: only append is allowed, no random update.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. What is a DFS?
   A. A file system on one disk only
   B. A file system whose files are stored across networked machines but accessed as if local
   C. A backup program
   D. A database engine
   Answer: B. A DFS gives users location transparency over files spread across machines. (Key point: Definition of DFS)

2. What does the NameNode do when the client calls create()?
   A. Receives and stores the file data
   B. Forwards packets to DataNodes
   C. Deletes older replicas
   D. Checks permissions and that the file does not exist, then records the new file
   Answer: D. The NameNode handles only metadata: permission checks, existence check and recording the file. (Key point: Role of NameNode in write)

3. How are packets replicated during an HDFS write?
   A. Client sends packets to the first DataNode, which forwards to the second, then third; acks return along the pipeline
   B. Client sends each packet separately to all three DataNodes
   C. NameNode copies packets to DataNodes
   D. Packets go to one DataNode and are replicated later by backup
   Answer: A. A pipeline of DataNodes forwards packets, and packets move from the data queue to the ack queue until all acknowledge. (Key point: Write pipeline and acknowledgements)

4. What is the default replica placement?
   A. All three on the same node
   B. 1st on writer's node, 2nd on a different rack, 3rd on another node in that second rack
   C. All three on different racks
   D. 1st on a different rack, 2nd and 3rd on the writer's rack
   Answer: B. This placement balances reliability against cross-rack bandwidth. (Key point: Rack-aware replica placement)

## Related in Big Data Technologies

- [Clock synchronization in DFS](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Aclock-synchronization-dfs) — Big Data Technologies, chapter 1
- [Short note: Client-Server and Master-Slave](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Ashort-note-client-server-master-slave) — Big Data Technologies, chapter 1
- [Role of HDFS in Hadoop](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Ahdfs-role-in-hadoop) — Big Data Technologies, chapter 6

## Related in other subjects

- [Digital data acquisition system block diagram](https://www.bctengineer.com/question.md?key=semester_4%2Finstrumentation_i%3Adigital-data-acquisition-system) — Instrumentation I, chapter 6
- [Data logger block diagram and operation](https://www.bctengineer.com/question.md?key=semester_5%2Finstrumentation_ii%3Adl-block) — Instrumentation II, chapter 5, asked 6×
- [Block diagram of data logger](https://www.bctengineer.com/question.md?key=semester_2%2Fbasic_electronics_engineering%3Adata-logger-block-diagram) — Basic Electronics Engineering, chapter 7
- [Compact data logger characteristics and block diagram](https://www.bctengineer.com/question.md?key=semester_5%2Finstrumentation_ii%3Acompact-dl-block) — Instrumentation II, chapter 5