Why distributed computing for big data

Distributed computing means splitting data and computation over many connected machines that work together as one system. It is necessary for big data because one computer cannot store or process data of such volume, speed and variety in reasonable time.

Reasons

1. Volume exceeds a single machine: data is in terabytes to petabytes. One disk or server cannot hold it, but many commodity nodes together can.
2. Processing speed: reading 1 TB from one disk at about 100 MB/s takes around 3 hours. With 1000 disks reading in parallel it takes about 10 seconds. Parallel processing gives the speed-up.
3. Scalability: capacity grows cheaply by adding more commodity nodes (horizontal scaling) instead of buying a costly supercomputer, whose limit is fixed.
4. Fault tolerance: with thousands of machines, failures are normal. Data is replicated (e.g. three copies) and failed tasks are re-run elsewhere, so the system keeps working.
5. Cost: cheap commodity hardware is much less expensive than one high-end server.
6. Data locality: moving code to the data is cheaper than moving huge data to the code, which saves network bandwidth.
7. Availability and reliability: no single point of failure for the data, and the service stays up when some nodes are down.
8. Variety and velocity: different nodes can handle different data types, and streaming data can be ingested by many nodes at once.

Example

Google indexes billions of web pages. This is only possible by spreading storage (GFS) and computation (MapReduce) over thousands of machines.