--- title: "Distributed system and Hadoop" subject: "Information System" chapter: 8 key: "semester_8/information_system:distributed-system-hadoop-voluminous-data" source: https://www.bctengineer.com/q/semester_8/information_system/distributed-system-hadoop-voluminous-data site: BCT Engineering --- # Distributed system and Hadoop Information System · Chapter 8: Scalable and Emerging Information System techniques - Asked in: 2072 Asoj, 2+2+4 marks - Video: [What is Apache Hadoop?](https://www.youtube.com/watch?v=JWX5Inb--ig) — IBM Technology - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_8/information_system/distributed-system-hadoop-voluminous-data - Whole chapter: https://www.bctengineer.com/md/semester_8/information_system/chapter-8 ## Question What is distributed system? Describe Hadoop systems. How voluminous data are handled? ## Short answer A distributed system is a network of independent computers working as one. Hadoop is an open-source framework with HDFS for storage, YARN for scheduling, and MapReduce for processing. Voluminous data is split into blocks, stored replicated across nodes, with computation moved to data for parallel processing. ## Answer #### Distributed system A **distributed system** is a collection of independent computers connected by a network that work together and appear to the user as a single coherent system. They share resources and coordinate by passing messages. Features: resource sharing, scalability, concurrency, fault tolerance and transparency. Examples: the web, cloud platforms, Hadoop clusters. #### Hadoop **Apache Hadoop** is an open-source framework for storing and processing very large data on clusters of commodity hardware. - **HDFS:** distributed storage; data split into blocks, replicated across nodes. - **YARN:** resource management and job scheduling. - **MapReduce:** parallel processing model. - **Hadoop Common:** shared libraries. - **Ecosystem:** Hive (SQL), Pig (scripts), HBase (NoSQL), Sqoop, Flume, Spark. - Features: scalable, fault tolerant, low cost, flexible with structured and unstructured data. #### How voluminous data are handled 1. The data is divided into large blocks and stored across many nodes in HDFS, each block replicated (usually 3 copies) for reliability. 2. Computation is sent to the data (data locality), instead of moving data to the computation. 3. MapReduce jobs run map tasks in parallel on all blocks, then reduce tasks aggregate the results. 4. Failed tasks or nodes are detected through heartbeats and work is re-run on other nodes. 5. Capacity grows by simply adding more nodes (horizontal scaling). 6. Higher-level tools (Hive, Pig, Spark) simplify queries and speed up analysis. ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. What defines a distributed system? A. One powerful computer with many users B. Independent computers connected by a network that appear to the user as a single coherent system C. A database with many tables D. A set of unconnected machines Answer: B. Independent networked computers coordinate by message passing and appear as one system. (Key point: Distributed system definition) 2. Which Hadoop component handles resource management and job scheduling? A. HDFS B. YARN C. Hive D. Hadoop Common Answer: B. HDFS is storage, YARN manages resources and scheduling, MapReduce processes, Common holds shared libraries. (Key point: Hadoop components) 3. How does Hadoop reduce network movement when handling volumes of data? A. It copies all data to one central server B. It sends computation to the nodes holding the data C. It compresses data to zero size D. It avoids replication Answer: B. Data locality: computation moves to the data instead of the reverse. (Key point: Data locality) 4. How is capacity increased and failure handled? A. Buying a bigger single machine; manual restart B. Adding more nodes (horizontal scaling); heartbeats detect failures and work is re-run elsewhere C. Reducing replication; ignoring failed nodes D. Stopping the cluster; restoring from tape Answer: B. Horizontal scaling plus heartbeat-based failure detection and task re-execution on other nodes. (Key point: Scaling and failure) ## Related in Information System - [Handling voluminous data](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Avoluminous-data-techniques-challenges) — Information System, chapter 8 - [MapReduce vs Hadoop; IR process](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Amapreduce-vs-hadoop-ir) — Information System, chapter 8 - [MapReduce vs Hadoop](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Amapreduce-vs-hadoop) — Information System, chapter 8 ## Related in other subjects - [Distributed system in big data example](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Adistributed-system-example) — Big Data Technologies, chapter 1 - [Role of HDFS in Hadoop](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Ahdfs-role-in-hadoop) — Big Data Technologies, chapter 6 - [pipe(); semaphore in distributed systems](https://www.bctengineer.com/question.md?key=semester_6%2Foperating_system%3Apipe-semaphore-distributed) — Operating System, chapter 3 - [Why distributed computing for big data](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Adistributed-computing-necessary) — Big Data Technologies, chapter 1