Skip to main content

Chapter 10 · Watch, then practise

Case Studies

Apply the preceding ideas to HDFS and ZooKeeper. These are supplementary practice case studies; the repository has no chapter-10 notes or playlist identifying the course’s prescribed systems.

3 questions · 3 with related videos. Matches are based on playlist titles; broader background matches are labeled.

What to study

  • HDFS metadata and blocks
  • ZooKeeper coordination data
  • Workload-specific design

Chapter playlists

Supplementary playlist — Big Data with Hadoop & Spark Training | Live Sessions | CloudxLab

Apache Zookeeper & HDFS | Big Data Hadoop Spark | CloudxLab

CloudxLab Official · 2:52:59

Supplementary CloudxLab lecture lists NameNodes, DataNodes and file reads/writes in its published outline.

1. HDFS request path

What different responsibilities do an HDFS NameNode and DataNodes have?

The NameNode manages the namespace and metadata mapping blocks to DataNodes. DataNodes store and serve file blocks. A client obtains placement information and transfers file data with DataNodes. Separating metadata from bulk data avoids routing every data byte through the NameNode.

Introduction to Apache ZooKeeper | Big Data Hadoop Spark | CloudxLab

CloudxLab Official · 3:04:47 · Background lecture

Supplementary ZooKeeper introduction; the session-expiration rule for ephemeral znodes is stated in the answer.

2. Session-scoped registration

How can a ZooKeeper ephemeral znode represent a running service instance, and what event removes it?

The instance creates an ephemeral znode in its session. ZooKeeper removes it when that session ends. Session expiry, rather than an instantaneous observation of a process crash, determines disappearance; clients must account for the detection delay.

Introduction to Apache ZooKeeper | Big Data Hadoop Spark | CloudxLab

CloudxLab Official · 3:04:47 · Background lecture

Choose a video · 2 lectures

ZooKeeper and HDFS introductions provide background for separating coordination from bulk file storage.

3. Choose the role

Would you place large video files in ZooKeeper or use it to coordinate services that store those files elsewhere?

Use it for small coordination records, such as configuration and service locations. HDFS illustrates a system designed for large distributed files. This division follows the documented workloads: metadata coordination and bulk storage have different access patterns and performance priorities.

References