--- title: "MapReduce vs Hadoop; IR process" subject: "Information System" chapter: 8 key: "semester_8/information_system:mapreduce-vs-hadoop-ir" source: https://www.bctengineer.com/q/semester_8/information_system/mapreduce-vs-hadoop-ir site: BCT Engineering --- # MapReduce vs Hadoop; IR process Information System · Chapter 8: Scalable and Emerging Information System techniques - Asked in: 2074 Bhadra, 4+4 marks - Video: [What is MapReduce♻️in Hadoop🐘| Apache Hadoop🐘](https://www.youtube.com/watch?v=pcTFiU7wKkQ) — Gate Smashers - Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14 - Page: https://www.bctengineer.com/q/semester_8/information_system/mapreduce-vs-hadoop-ir - Whole chapter: https://www.bctengineer.com/md/semester_8/information_system/chapter-8 ## Question Differentiate between Map reduce and Hadoop systems. Explain the process of Information Retrieval in detail with example. ## Short answer MapReduce is a programming model; Hadoop is a framework including HDFS and MapReduce. IR process: collect, preprocess, build inverted index, match query, rank results. For 'cloud storage', D2 'cloud storage is cheap and safe' ranks higher than D1 'cloud computing is cheap'. ## Answer #### Difference between MapReduce and Hadoop | Basis | MapReduce | Hadoop | |---|---|---| | **Nature** | A programming model and processing technique | A complete open-source framework or ecosystem | | **Origin** | Introduced by Google (2004) | Apache project, inspired by Google's MapReduce and GFS | | **Purpose** | Process large data in parallel using map and reduce functions | Store (HDFS) and process (MapReduce, YARN) big data on clusters | | **Components** | Map, shuffle/sort, reduce | HDFS, YARN, MapReduce, plus tools such as Hive, Pig, HBase | | **Storage** | Does not store data itself | HDFS stores data | | **Relation** | Is a component of Hadoop | Implements MapReduce | #### Process of information retrieval **Information retrieval (IR)** finds documents in a collection that are relevant to a user's query. 1. **Document collection and acquisition:** crawl or collect documents. 2. **Preprocessing:** tokenisation, stop word removal, stemming, normalisation. 3. **Indexing:** build an inverted index (term -> documents with frequency/positions). 4. **Query formulation:** the user states a need; the query is processed in the same way. 5. **Matching and retrieval:** the engine finds documents containing the terms. 6. **Ranking:** documents are scored (TF-IDF, BM25, PageRank) and sorted. 7. **Presentation and feedback:** show results; relevance feedback may refine the query. ```text Docs -> Preprocess -> Index Query -> Preprocess -> Match -> Rank -> Results ``` #### Example Documents: D1 "cloud computing is cheap", D2 "cloud storage is cheap and safe". Inverted index: cloud -> D1, D2; computing -> D1; storage -> D2; cheap -> D1, D2; safe -> D2. Query "cloud storage" matches D2 on two terms and D1 on one, so D2 is ranked first, then D1. ## Check yourself The answer's key points as multiple-choice questions, with the correct option and why. 1. Which statement correctly describes the relation between MapReduce and Hadoop? A. MapReduce is a complete ecosystem and Hadoop is only one of its functions B. MapReduce is a programming model that is a component of the Hadoop framework C. Hadoop is a programming model and MapReduce is the storage layer D. They are the same thing developed by Apache Answer: B. MapReduce is a processing model introduced by Google; Hadoop is the wider framework (HDFS, YARN, MapReduce, tools) that implements it. (Key point: MapReduce vs Hadoop nature) 2. Which of the two stores data itself? A. MapReduce, through its shuffle phase B. Neither of them stores data C. Hadoop, through HDFS D. MapReduce, through YARN Answer: C. MapReduce does not store data; Hadoop provides HDFS for storage. (Key point: Storage) 3. In the IR process, what is built after preprocessing (tokenisation, stop word removal, stemming)? A. A ranked result list B. An inverted index mapping terms to documents C. A relevance feedback loop D. A crawler queue Answer: B. Indexing builds an inverted index of term -> documents with frequency/positions, which is used later for matching. (Key point: Preprocessing and indexing) 4. With D1 'cloud computing is cheap' and D2 'cloud storage is cheap and safe', why is D2 ranked first for the query 'cloud storage'? A. D2 is longer B. D2 matches both query terms while D1 matches only one C. D2 contains the stop word 'and' D. D2 was indexed later Answer: B. D2 contains both 'cloud' and 'storage', D1 only 'cloud', so D2 scores higher. (Key point: Ranking example) 5. How is the user's query treated before matching? A. It is processed in the same way as documents (tokenised, stemmed, etc.) B. It is matched raw without any processing C. It is stored in HDFS D. It is converted into a PageRank score Answer: A. The query goes through the same preprocessing so its terms match index terms. (Key point: Query processing) ## Related in Information System - [MapReduce vs Hadoop](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Amapreduce-vs-hadoop) — Information System, chapter 8 - [Information retrieval in cloud](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Air-importance-in-cloud) — Information System, chapter 7 - [Distributed system and Hadoop](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Adistributed-system-hadoop-voluminous-data) — Information System, chapter 8 - [MIS vs DSS information and processing](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Amis-vs-dss-information-processing) — Information System, chapter 4 - [MapReduce working with example](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Amapreduce-working-big-data) — Information System, chapter 8 - [MapReduce with GFS and HDFS](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Amapreduce-gfs-hdfs) — Information System, chapter 8 ## Related in other subjects - [Web documents, browser, information retrieval](https://www.bctengineer.com/question.md?key=semester_7%2Fweb_technology%3Aweb-documents-browser-retrieval) — Web Technology, chapter 2 - [Differentiate wired and wireless systems](https://www.bctengineer.com/question.md?key=semester_2%2Fbasic_electronics_engineering%3Awired-vs-wireless-diff) — Basic Electronics Engineering, chapter 5, asked 2× - [MapReduce worker failure tolerance](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Amapreduce-fault-tolerance) — Big Data Technologies, chapter 3 - [MapReduce workflow with example](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Amapreduce-workflow-example) — Big Data Technologies, chapter 3, asked 2×