This talk will explore how factual uses some of the advanced features in version .98 and beyond to build a robust and flexible data store.

1. HBase and Batch Processing
Molly O’Connor
Factual

2. HBase at Factual
Support API for global location queries and accept live
writes of new supporting data
Batch updates: ingesting large amounts of new data,
pushing out new versions of the data (improvements in
algorithms for data cleaning, verification, clustering)

3. Overview
1. HBase Architecture
2. Batch Workflow
3. Lessons/Challenges

4. HBase Intro -- Data Model
● Column families for HBase table are specified at
creation time
● Arbitrary byte sequences for column qualifiers (unlimited
and created as data is written)
● Data organized by column families and sorted by key

5. HBase Intro -- HFile Format
● Sorted lexicographically with secondary indices inline
with data
● Block size: memory tradeoffs. Choose based on
expected read access
● Compression: experiment: lzo, snappy, gz
● Index Size: don’t make keys and column names longer
than needed.

6. HBase Intro -- Region Servers

7. HBase Intro -- Locality
● Region servers write new data locally
● Compaction further promotes data locality
● Metrics: at the region server level. In 1.0, “Block
Locality”, in 0.94, “hdfsBlocksLocalityIndex”
● Enable short circuit reads for additional benefits

8. HBase Intro -- Consistency
● Single row atomicity across column families guaranteed
● checkAndPut -- single row, checks on the value of a
single column only
● mutateRowsWithLocks --via coprocessor
○ within a region: need clever row key design, region
split policy

9. Overview
1. HBase Architecture
2. Batch Workflow
3. Lessons/Challenges

10. ● Better performance for large scale updates
● Quality analysis and metrics on all data before adopting
● Perform computations that are not possible or
prohibitively expensive in a live hbase setting
● Data is already on HDFS
HBase and Batch

11. Batch Workflow
1. Snapshot
2. MapReduce
3. Bulkloading

12. HBase Snapshots
● Copy on write (HFile links)
● Per table
● Rolling
○ HBase’s guarantee of consistency within a row
● Use cases: backup/recovery, export, mapreduce

13. Snapshots and MapReduce
● Definitely use Mapreduce over snapshots, if possible
(HBASE-8369)
○ before this feature, issues with reading HFiles
directly because of compaction
○ Advantages: Job is faster and puts less pressure on
region servers
○ Caveat: Not reading live data

14. Locality and Mapreduce
Tradeoffs: want to colocate computation with data. But, this
causes contention with HBase
○ Don’t run mapreduce on HBase nodes?
○ Mitigated somewhat with Yarn?

15. Federated Namespaces for Multi-Purpose Cluster

16. Bulkloading
● Additional path to ingesting data with HBase. Create
HFiles directly, and HBase adopts the files
● Bulkload is atomic at the region level
○ single row consistency (across CF) guaranteed

17. Locality after Bulkloading
● Look at current region locations, and try to produce new
HFiles on those nodes
● Compaction after bulkloading: needs to be timed well,
but can eventually lead to locality
● New data will not be in the block cache!
● HBASE-11195 can promote compaction in cases where
locality is low
● HBASE-8329 throttling compaction speed

18. Overview
1. HBase Architecture
2. Batch Workflow
3. Lessons/Challenges

19. Challenges
● Does bulkloading fit your data model?
○ Replay--do you need a catchup phase after data
ingestion?
● Consistency beyond row level (using a library to
manage a secondary index or other transactional
writes)?
● Maybe use Mapreduce over live tables but throttle
requests? HBASE-11598

20. Summary
1. Ability to do bulk updates can be hugely important for
performance
2. HDFS integration and strong feature set make HBase a
good choice for batch processing
3. More features coming (today highlighted many
introduced in the new version 1.0)

21. Questions?