Skip to main content

Data analytics process and distributed computing

Short answer

Data analytics is needed to turn raw big data into better decisions, predictions, and fraud detection. Distribute computing provides storage (HDFS splitting files across nodes), parallel processing (MapReduce/Spark), fault tolerance through replication, and cost-effectiveness by using commodity hardware with data locality.

Why do we need data analytics process? Explain the role of Distributed computing in Big data.

Why do we need the data analytics process?

Data analytics is the process of collecting, cleaning, transforming and modelling data to discover useful information and support decisions. A defined process is needed because raw data by itself has no value.

  1. Better decisions: replaces guesswork with evidence (e.g. which product to stock).
  2. Finding patterns and trends: sales seasonality, fraud patterns, customer behaviour.
  3. Prediction: forecast demand, churn or failures using historical data.
  4. Efficiency and cost reduction: find waste in operations and supply chains.
  5. Customer understanding: personalised offers and recommendations.
  6. Risk and fraud detection: spot unusual transactions in time.
  7. Competitive advantage: firms that learn faster from data win.
  8. Handling big data: an organised process (collect, store, clean, analyse, visualise, act) keeps results reliable and repeatable at large scale.

Typical steps: define the objective, collect data, clean and prepare, explore, model, validate, visualise/report, and deploy.

Role of distributed computing in big data

Big data cannot be stored or processed on one machine, so distributed computing is its foundation.

  • Distributed storage: files are split into blocks and spread across nodes (HDFS, GFS), giving petabyte capacity.
  • Parallel processing: frameworks like MapReduce and Spark run the same task on many data blocks at once, reducing time from hours to minutes.
  • Scalability: add commodity nodes when data grows (horizontal scaling).
  • Fault tolerance: replication of data and re-execution of failed tasks keep jobs running when nodes fail.
  • Data locality: computation moves to the node holding the data, which saves network bandwidth.
  • Cost reduction: cheap commodity servers replace expensive supercomputers.
  • Support for variety and velocity: different nodes and tools (Hive, HBase, Kafka, Storm) handle structured, unstructured and streaming data.
  • High availability: no single machine failure stops the service.

Example: a telecom firm analyses billions of call records stored on HDFS using MapReduce to find churn patterns in a few hours.

✓ Answer checked by Claude Opus 5.5 (high effort)

Related questions

All Big Data Technologies chapter 1 questions →