HBase architecture

HBase is an open-source, distributed, column-oriented NoSQL database built on top of HDFS, modelled on Google's BigTable. It stores huge tables (billions of rows, millions of columns) and gives random, real-time read/write access.

Data model

A table has rows identified by a sorted row key. Columns are grouped into column families; each cell is versioned by a timestamp. Tables are split horizontally into regions (ranges of row keys).

Architecture components

        Client  <----->  ZooKeeper
          |                 |
          v                 v
      HMaster  <-- assigns regions -->
          |
   +------+-------+---------------+
   |              |               |
RegionServer1  RegionServer2  RegionServer3
 [Region][Region] ...
   |  WAL | MemStore | HFiles
   +--------------+---------------+
                  v
            HDFS (DataNodes)

1. HMaster: the master server. It assigns regions to region servers, handles load balancing, region splits/merges, and schema operations (create, delete, alter tables). It monitors region servers. It does not serve data reads and writes.
2. Region Server (HRegionServer): runs on the HDFS DataNodes. It hosts many regions, serves client read/write requests directly, and splits regions when they grow large.
3. Region: a contiguous range of rows of a table. Each region has one Store per column family. A Store consists of:
   • MemStore: in-memory write buffer; data is sorted and flushed to disk when full.
   • HFiles: immutable sorted files on HDFS. Several HFiles are merged by compaction.
4. WAL (Write-Ahead Log / HLog): every write is first appended to the WAL on HDFS, so data can be recovered if the region server crashes before MemStore is flushed.
5. ZooKeeper: coordination service. It keeps track of live servers, elects the active HMaster, and stores the location of the hbase:meta table.
6. META table: a special catalog table listing which region server holds which region. The client finds it via ZooKeeper, caches the location, then talks directly to the region server.
7. HDFS: the underlying storage that provides replication and fault tolerance.

Write path

Client -> Region Server -> append to WAL -> write to MemStore -> acknowledge -> flush to HFile later.

Read path

Client -> ZooKeeper -> META -> Region Server -> check MemStore and block cache, then HFiles -> return result.