---
title: "Components of Hadoop"
subject: "Big Data Technologies"
chapter: 6
key: "semester_8/big_data_technologies:hadoop-components"
source: https://www.bctengineer.com/q/semester_8/big_data_technologies/hadoop-components
site: BCT Engineering
---

# Components of Hadoop

Big Data Technologies · Chapter 6: Case Study: Hadoop

- Asked in: 2073 Bhadra, 10 marks
- Video: [Hadoop Components Explained | Hadoop Ecosystem | Hadoop Architecture | Hadoop Tutorial | Edureka](https://www.youtube.com/watch?v=ll_O9JsjwT4) — edureka!
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_8/big_data_technologies/hadoop-components
- Whole chapter: https://www.bctengineer.com/md/semester_8/big_data_technologies/chapter-6

## Question

What are the components of Hadoop? Explain each in brief.

## Short answer

Hadoop core components are HDFS for distributed replicated storage, MapReduce for parallel processing, YARN for cluster resource management, and Hadoop Common for shared libraries. The ecosystem adds Hive, Pig, HBase, Sqoop, Flume, ZooKeeper, Oozie and others.

## Answer

**Apache Hadoop** is an open-source framework for distributed storage and processing of very large data sets on clusters of commodity hardware. It has core components and an ecosystem of related tools.

#### Core components

1. **HDFS (Hadoop Distributed File System):** the storage layer, based on the Google File System. Files are split into large blocks (128 MB) replicated (default 3 times) across DataNodes. The NameNode keeps metadata. It provides fault tolerance and high-throughput access.
2. **MapReduce:** the processing layer. A job is split into **map** tasks (process input splits in parallel, emit key-value pairs) and **reduce** tasks (aggregate by key). Computation moves to the data.
3. **YARN (Yet Another Resource Negotiator):** the resource-management layer (Hadoop 2+). The ResourceManager and NodeManagers allocate CPU and memory containers to applications, so that MapReduce, Spark and others can share the cluster.
4. **Hadoop Common:** shared Java libraries and utilities (I/O, RPC, serialization, security) used by all modules.

```text
 +--------------------------------------+
 | Ecosystem: Hive Pig HBase Sqoop ...  |
 +--------------------------------------+
 | MapReduce (processing)               |
 +--------------------------------------+
 | YARN (resource management)           |
 +--------------------------------------+
 | HDFS (storage)                       |
 +--------------------------------------+
```

#### Ecosystem components

| Component | Purpose |
|---|---|
| Hive | SQL-like queries (HiveQL) over HDFS data |
| Pig | Data-flow scripting (Pig Latin) |
| HBase | Column-oriented NoSQL database on HDFS |
| Sqoop | Import/export between RDBMS and Hadoop |
| Flume | Collecting streaming log data into HDFS |
| ZooKeeper | Coordination and configuration service |
| Oozie | Workflow scheduler for Hadoop jobs |
| Mahout/Spark | Machine learning and in-memory processing |

#### Key features

Scalable (add nodes), fault tolerant (replication, task re-execution), cost-effective (commodity hardware), and works with structured and unstructured data.

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. How does HDFS store a large file?
   A. As one file on the NameNode
   B. In a relational table
   C. On a single DataNode with a backup tape
   D. Split into large blocks (128 MB) replicated, by default 3 times, across DataNodes
   Answer: D. Block splitting and replication give fault tolerance and high throughput. (Key point: HDFS blocks and replication)

2. How does MapReduce process a job?
   A. Reduce tasks run first and map tasks aggregate by key
   B. Map tasks process input splits in parallel and emit key-value pairs; reduce tasks aggregate by key
   C. All data is moved to one machine for processing
   D. Only one task runs at a time
   Answer: B. Map is parallel per split; reduce aggregates by key, with computation moved to the data. (Key point: MapReduce model)

3. What is YARN's role in Hadoop 2+?
   A. Storing metadata
   B. Importing data from RDBMS
   C. Resource management: allocating CPU and memory containers so MapReduce, Spark and others share the cluster
   D. Providing SQL-like queries
   Answer: C. YARN's ResourceManager and NodeManagers allocate containers to applications. (Key point: Role of YARN)

4. Which pairing of ecosystem component and purpose is correct?
   A. Hive: SQL-like queries over HDFS data
   B. Sqoop: collecting streaming logs
   C. Flume: import/export with RDBMS
   D. ZooKeeper: SQL-like queries
   Answer: A. Hive gives HiveQL; Sqoop handles RDBMS transfer; Flume handles streaming logs; ZooKeeper coordinates. (Key point: Ecosystem tools)

## Related in Big Data Technologies

- [Hadoop daemons explained](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Ahadoop-daemons) — Big Data Technologies, chapter 6, asked 2×

## Related in other subjects

- [Components of a VR system](https://www.bctengineer.com/question.md?key=semester_5%2Fcomputer_graphics%3Avr-system-components) — Computer Graphics, chapter 1
- [Major components of a power station and functions](https://www.bctengineer.com/question.md?key=semester_4%2Fpower_system_analysis_i%3Apower-station-components-functions) — Power System Analysis I, chapter 1