---
title: "Big data and its processing"
subject: "Information System"
chapter: 8
key: "semester_8/information_system:big-data-processing"
source: https://www.bctengineer.com/q/semester_8/information_system/big-data-processing
site: BCT Engineering
---

# Big data and its processing

Information System · Chapter 8: Scalable and Emerging Information System techniques

- Asked in: 2072 Magh, 2+4 marks
- Video: [Big Data In 5 Minutes | What Is Big Data?| Big Data Analytics | Big Data Tutorial | Simplilearn](https://www.youtube.com/watch?v=bAyrObl7TYE) — Simplilearn
- Answer checked by Claude Opus 5.5 (high effort) on 2026-10-14
- Page: https://www.bctengineer.com/q/semester_8/information_system/big-data-processing
- Whole chapter: https://www.bctengineer.com/md/semester_8/information_system/chapter-8

## Question

Explain Big Data. How is it processed?

## Short answer

Big data is datasets too large, fast, and varied for traditional tools, characterized by volume, velocity, variety. Processing involves ingestion via Flume/Kafka, distributed storage in HDFS/NoSQL, batch processing with MapReduce or stream processing with Spark, then analysis and visualization.

## Answer

#### Big Data
**Big data** means datasets so large, fast-growing and varied that traditional database tools cannot store, manage or process them efficiently. They come from social media, sensors, e-commerce, logs and mobile devices.

#### Characteristics (the Vs)
- **Volume:** terabytes to petabytes.
- **Velocity:** speed at which data is generated and must be processed.
- **Variety:** structured, semi-structured and unstructured data (text, images, video).
- **Veracity:** uncertainty and quality of data.
- **Value:** useful insight that can be extracted.

#### How big data is processed
1. **Data collection / ingestion:** from sources using tools such as Flume, Kafka, Sqoop.
2. **Storage:** distributed systems such as HDFS, NoSQL databases and cloud storage.
3. **Processing:**
   - **Batch processing:** Hadoop MapReduce works on large stored datasets in parallel (map, shuffle, reduce).
   - **Stream processing:** Storm, Spark Streaming, Flink handle data in real time.
   - **In-memory processing:** Spark for fast iterative analysis.
4. **Analysis:** querying (Hive, Pig), data mining, machine learning, text and graph analytics.
5. **Visualisation and reporting:** dashboards and charts to support decisions.

```text
 Sources -> Ingest -> Store (HDFS/NoSQL)
        -> Process (MapReduce/Spark)
        -> Analyse -> Visualise
```

## Check yourself

The answer's key points as multiple-choice questions, with the correct option and why.

1. Why is a dataset considered big data?
   A. It is stored on the cloud
   B. It is so large, fast-growing and varied that traditional database tools cannot handle it efficiently
   C. It contains only images
   D. It is older than ten years
   Answer: B. Big data exceeds the capability of traditional tools to store, manage or process. (Key point: Definition)

2. Which V refers to structured, semi-structured and unstructured data types?
   A. Volume
   B. Velocity
   C. Variety
   D. Veracity
   Answer: C. Variety is about different forms of data; veracity is quality/uncertainty, velocity is speed. (Key point: Variety)

3. Which choice matches real-time processing?
   A. Hadoop MapReduce batch jobs
   B. Storm, Spark Streaming or Flink stream processing
   C. Data warehouse nightly loads
   D. Manual spreadsheets
   Answer: B. Stream processing tools handle data in real time; MapReduce is batch. (Key point: Batch vs stream)

4. What is the correct order of the big data pipeline?
   A. Analyse, ingest, store, visualise, process
   B. Ingest, store, process, analyse, visualise
   C. Store, visualise, ingest, process, analyse
   D. Process, ingest, analyse, store, visualise
   Answer: B. Sources -> ingest -> store -> process -> analyse -> visualise. (Key point: Processing pipeline order)

## Related in Information System

- [MIS vs DSS information and processing](https://www.bctengineer.com/question.md?key=semester_8%2Finformation_system%3Amis-vs-dss-information-processing) — Information System, chapter 4

## Related in other subjects

- [Role of data scientist in big data](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Arole-of-data-scientist) — Big Data Technologies, chapter 1, asked 2×
- [Why distributed computing for big data](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Adistributed-computing-necessary) — Big Data Technologies, chapter 1
- [Distributed system in big data example](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Adistributed-system-example) — Big Data Technologies, chapter 1
- [Big data trends, challenges, characteristics](https://www.bctengineer.com/question.md?key=semester_8%2Fbig_data_technologies%3Abig-data-trends-challenges) — Big Data Technologies, chapter 1