Presented at Java One 2015

1. 1©
Cloudera,
Inc.
All
rights
reserved.
Apache
HBase:
Overview,
Hands-­‐On,
and
Use
Cases
Apekshit Sharma
Esteban
Gutierrez

2. 2©
Cloudera,
Inc.
All
rights
reserved.
Apekshit Sharma
• Distributed
Software
Engineer,
Cloudera
• Software
Engineer,
Google
• Apache
HBase contributor
• Performance
improvements
and
configuration
framework
Esteban
Gutierrez
• Software
Engineer,
Cloudera
• Apache
HBase
committer
• +15
years
of
experience
on
Internet
scale
platforms.
Who
we
are

3. 3©
Cloudera,
Inc.
All
rights
reserved.
Contents
• Motivation
• Introduction
to
Apache
HBase
• Data
model
• Hands-­‐On:
Installation,
HBase
shell
• A
more
in-­‐depth
introduction
to
Apache
HBase
• Apache
Hadoop
HDFS
• HBase
Architecture

4. 4©
Cloudera,
Inc.
All
rights
reserved.
Motivation
http://www.internetlivestats.com/total-­‐number-­‐of-­‐websites/

5. 5©
Cloudera,
Inc.
All
rights
reserved.
Motivation
• Indexing
the
internet
has
challenges:
• Scale
• Volume
• Rate
• Diversity
of
content
• URLs
• High-­‐resolution
images
• Video

6. 6©
Cloudera,
Inc.
All
rights
reserved.
Motivation

7. 7©
Cloudera,
Inc.
All
rights
reserved.
• What
if
you’re
not
trying
to
index
the
internet?
Motivation

8. 8©
Cloudera,
Inc.
All
rights
reserved.
• Data
for
analytical
processing
• User-­‐facing
real-­‐time
platforms
Motivation

9. 9©
Cloudera,
Inc.
All
rights
reserved.
Apache
HBase
• Open
source
• Random
access,
Low
latency
and
Consistent
data
store
• Horizontally scalable
• Built
on
top
of
Apache
Hadoop

10. 10©
Cloudera,
Inc.
All
rights
reserved.
Apache
HBase
is
open
source
• Apache
2.0
License
• A
community
project
with
committers
and
contributors
from
diverse
organizations
• Cloudera,
Facebook,
Salesforce.com,
Huawei,
TrendMicro,
eBay,
HortonWorks,
Intel,
Twitter,
…
• Code
license
means
anyone
can
modify
and
use
the
code.

11. 11©
Cloudera,
Inc.
All
rights
reserved.
•CAP
theorem
•Consistency
•Availability
•Partition
tolerance
Apache
HBase
is
consistent
HBase

12. 12©
Cloudera,
Inc.
All
rights
reserved.
• Adding
more
servers
linearly increases
performance
and
capacity
• Storage
capacity
• Input/output
operations
• Store
and
access
data
on
commodity
servers • Largest
cluster:
>
3000
nodes,
>
100
PB
• Average
cluster:
10-­‐40
nodes,
100-­‐400TB
Apache
HBase
is
horizontally
scalable
0
100
200
300
400
500
600
Performance
(IOPs/Storage/Throughput)
#
of
servers

13. 13©
Cloudera,
Inc.
All
rights
reserved.
• Commodity
servers
(circa 2015)
• 12-­‐24
2-­‐4TB
hard disks
• 2
(quad/hexa/octa)-­‐core
CPUs,
2-­‐3
GHz
• 64
-­‐ 512 GBs
of
RAM
• 10
Gbps ethernet
• $5,000
-­‐ $10,000
/
machine
Apache
HBase
is
horizontally
scalable

14. 14©
Cloudera,
Inc.
All
rights
reserved.
Data
model
Row
key info:name info:age comp:base comp:stocks
121 ‘tom’ ‘28’ ‘125k’
145 ‘bob’ ‘32’ ‘110k’
‘50’
(ts=2012)
‘100’
(ts=2014)
Columns
<Column
family>:<column
qualifier>
Cells
Row
keys

15. 15©
Cloudera,
Inc.
All
rights
reserved.
Data
model
• Data
is
stored…
in
a
big
table
• Sorted
rows with
primary
row
keys.
Supports
billions
of
rows.
• Column
:
<column
family>:<column
qualifier>;
Supports
millions
of
columns
• Cell
:
intersection
of
row
and
column.
• Can
be
empty.
No
storage/processing
overheads
• Can
have
different
time-­‐stamped
values

16. 16©
Cloudera,
Inc.
All
rights
reserved.
Hands
On
Apache
HBase installation
Play
with
HBase
shell
http://pastebin.com/60EUN75f

17. 17©
Cloudera,
Inc.
All
rights
reserved.
Understanding
basic
architecture
of
Hadoop
(HDFS)

18. 18©
Cloudera,
Inc.
All
rights
reserved.
Apache
Hadoop
Open
source
Commodity servers
Horizontally scalable
Highly fault-­‐tolerant
Massive processing
power

19. 19©
Cloudera,
Inc.
All
rights
reserved.
Apache
Hadoop
MapReduce
+
YARN
2
Core
Components
HDFS
(Hadoop
Distributed
File
System)

20. 20©
Cloudera,
Inc.
All
rights
reserved.
History
2003

21. 21©
Cloudera,
Inc.
All
rights
reserved.
• Distributed file
system
• Commodity servers
• horizontally scalable
• Highly fault-­‐tolerant
• Proprietary
GFS

22. 22©
Cloudera,
Inc.
All
rights
reserved.
• Distributed file
system
• Commodity servers
• Horizontally scalable
• Highly fault-­‐tolerant
• Open
source
HDFS

23. 23©
Cloudera,
Inc.
All
rights
reserved.
HDFS
API
• File
• Open,
Close,
Read,
Write,
Move,
etc
• Directories
• Create,
Delete,
etc
• Permissions
• Owners,
Groups,
rwx permissions

24. 24©
Cloudera,
Inc.
All
rights
reserved.
Basic
Architecture
of
HDFS

25. 25©
Cloudera,
Inc.
All
rights
reserved.
File
B1 B2 B3
File
system
will
split
the
file
into
blocks
DiskB1
B2
B3
Local
file
system

26. 26©
Cloudera,
Inc.
All
rights
reserved.
DataNode
1 DataNode
2 DataNode
3 DataNode
4
HDFS
File
distributed
across
machines
B1 B2 B3

27. 27©
Cloudera,
Inc.
All
rights
reserved.
B1
DataNode
1
B2
DataNode
2
B3
DataNode
3 DataNode
4
HDFS
DataNode

28. 28©
Cloudera,
Inc.
All
rights
reserved.
DataNode
1 DataNode
2 DataNode
3 DataNode
4
HDFS
NameNode
NameNode
B1 B2 B3

29. 29©
Cloudera,
Inc.
All
rights
reserved.
DataNode
1 DataNode
2 DataNode
3 DataNode
4
HDFS
Reading
a
file
NameNode
B1 B2 B3
Client
1.
File
‘foo’ 2.
Verify
client
has
permissions
to
read
the
file
3.
List
of
foo’s
bocks
and
datanodes

30. 30©
Cloudera,
Inc.
All
rights
reserved.
DataNode
1 DataNode
2 DataNode
3 DataNode
4
HDFS
Fault
tolerance
NameNode
B1 B2 B3

31. 31©
Cloudera,
Inc.
All
rights
reserved.
DataNode
1 DataNode
2 DataNode
3 DataNode
4
HDFS
Redundancy
NameNode
B1 B2 B3B1 B1B2 B2 B3B3

32. 32©
Cloudera,
Inc.
All
rights
reserved.
DataNode
1 DataNode
2 DataNode
3 DataNode
4
HDFS
Horizontal
Scalability
NameNode
B1 B2 B3B1 B1B2 B2 B3B3
DataNode
5

33. 33©
Cloudera,
Inc.
All
rights
reserved.
Let’s
look
at
some
existing
HDFS
systems...

34. 34©
Cloudera,
Inc.
All
rights
reserved.
• Yahoo!
HDFS
Clusters
40k+
servers,
100k+
CPUs,
450PB
data
• Spam filtering,
web
indexing,
personaliztion,
etc.
• Facebook
HDFS
Cluster
15TB
new
data
per
day
1200+
machines,
30PB
in
one
cluster
• Datawarehouse,
Messages,
etc.

35. 35©
Cloudera,
Inc.
All
rights
reserved.
• 1-­‐10PB HDFS
Clusters
Production
critical
workloads
for
websites,
retail,
finance,
telcom,
government,
research
…
• Sub
PB
HDFS
Clusters
PoCs,
R&D,
production
in
early
stages
…

36. 36©
Cloudera,
Inc.
All
rights
reserved.
But….
there
are
restrictions!
It’s
not
a
magic
wand!

37. 37©
Cloudera,
Inc.
All
rights
reserved.
Files
are append
only
• Access
Model
:
Write-­‐once-­‐read-­‐many
• Can
not
change
existing
contents

38. 38©
Cloudera,
Inc.
All
rights
reserved.
Not
designed
for
small
files
• HDFS
block
sizes
are
in
MB
(default
128MB)
• Designed
for
typical
GBs
/
TBs
of
file
sizes
• Normal
files
system
have
4kb
block
size!
• Name
node
becomes
bottleneck
because
metadata
grows
too
big

39. 39©
Cloudera,
Inc.
All
rights
reserved.
Summary
HDFS
is
a
great
distributed
file
system!
• Store
massive
data
• Scalable
• High
throughput
• Fault
tolerance

40. 40©
Cloudera,
Inc.
All
rights
reserved.
MapReduce
• Distributed
processing
framework
• Commodity
machines
• Fault
tolerance

41. 41©
Cloudera,
Inc.
All
rights
reserved.
MapReduce
• HBase
uses
and
extends
MR
APIs
for
data
access
Further
reading:
http://hadoop.apache.org

42. 42©
Cloudera,
Inc.
All
rights
reserved.

43. 43©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
Unique
id Name price weight store1 store2 store3
“1000000” snickers $9.99 4
Oz Yes Yes Yes
“3000000” almonds $9.99 8
Oz Yes No Yes
“8000000” coke $9.99 16
Oz Yes Yes Yes

44. 44©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
Unique
id Name price weight store1 store2 store3
“1000000” snickers $9.99 4
Oz Yes Yes Yes
“3000000” almonds $9.99 8
Oz Yes No Yes
“8000000” coke $9.99 16
Oz Yes Yes Yes
“4000000” new $34.63 16
Oz No Yes Yes

45. 45©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
Unique
id Name price weight store1 store2 store3
“1000000” snickers $9.99 4
Oz Yes Yes Yes
“3000000” almonds $9.99 8
Oz Yes No Yes
“8000000” coke $9.99 16
Oz Yes Yes Yes
“4000000” foo $34.63 16
Oz No Yes Yes
“5000000” bar $22.54 16
Oz Yes Yes Yes

46. 46©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
Unique
id Name price weight store1 store2 store3
“1000000” snickers $9.99 4
Oz Yes Yes Yes
“3000000” almonds $9.99 8
Oz Yes No Yes
“8000000” coke $9.99 16
Oz Yes Yes Yes
“4000000” foo $34.63 16
Oz No Yes Yes
“5000000” bar $22.54 16
Oz Yes Yes Yes
“9000000” new1 $2.5 16
Oz Yes Yes Yes
“7000000” new2 $6.4 16
Oz Yes Yes Yes
“2000000” new3 $6.4 16
Oz Yes Yes Yes

47. 47©
Cloudera,
Inc.
All
rights
reserved.
Row Key Name brand price weight store1 store2 store3
“1000000” snickers xxx $9.99 4
Oz Yes Yes Yes
“2000000” new3 xxx $6.4 16
Oz Yes Yes Yes
“3000000” almonds xxx $9.99 8
Oz Yes No Yes
“4000000” foo xxx $34.63 16
Oz No Yes Yes
Row
Key Name brand price weight store1 store2 store3
“5000000” bar xxx $22.54 16
Oz Yes Yes Yes
“7000000” new2 xxx $6.4 16
Oz Yes Yes Yes
“8000000” coke xxx $9.99 16
Oz Yes Yes Yes
“9000000” new1 xxx $2.5 16
Oz Yes Yes Yes
[
“”,
“5000000”)
[
“5000000”,
“”)
HBase
Architecture
|
Regions

48. 48©
Cloudera,
Inc.
All
rights
reserved.
Row Key Name price weight ….
“1000000” snickers $9.99 4
Oz ….
“2000000” new3 $6.4 16
Oz ….
“3000000” almonds $9.99 8
Oz ….
“4000000” foo $34.63 16
Oz ….
Row Key Name price weight ….
“5000000” bar $22.54 16
Oz ….
“7000000” new2 $6.4 16
Oz ….
“8000000” coke $9.99 16
Oz ….
“9000000” new1 $2.5 16
Oz ….
HBase
Architecture
|
RegionServer
Server
12
Server
7

49. 49©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Regions
• Horizontal
split
of
tables
• Regions
are
served
by
RegionServers
• Automatically
(based
on
configurations).
Can
be
done
manually
too.
Relationship:
• A
Region
is
only
served
by
a
single
RegionServer
at
a
time.
• A
RegionServer
can
serve
multiple
Regions.
Region
Server
X
Region
[""
,
10)
Region
[40,
50)
Region
Server
Y
Region
[10,
30)
Region
[30,
40)
Region
[50
"")

50. 50©
Cloudera,
Inc.
All
rights
reserved.
Row Key Name price weight store1 store2 store3
“1000000” snickers $9.99 4
Oz Yes Yes Yes
“2000000” new3 $6.4 16
Oz Yes Yes Yes
“3000000” almonds $9.99 8
Oz Yes No Yes
“4000000” foo $34.63 16
Oz No Yes Yes
“5000000” bar $22.54 16
Oz Yes Yes Yes
“7000000” new2 $6.4 16
Oz Yes Yes Yes
“8000000” coke $9.99 16
Oz Yes Yes Yes
“9000000” new1 $2.5 16
Oz Yes Yes Yes
HBase
Architecture
What
if
you
have
many
more
columns?

51. 51©
Cloudera,
Inc.
All
rights
reserved.
Row Key info:
name
info:
price
info:
weight
availability:
store1
availability:
store2
availability:
store3
“1000000” snickers $9.99 4
Oz Yes Yes Yes
“2000000” new3 $6.4 16
Oz Yes Yes Yes
“3000000” almonds $9.99 8
Oz Yes No Yes
“4000000” foo $34.63 16
Oz No Yes Yes
“5000000” bar $22.54 16
Oz Yes Yes Yes
“7000000” new2 $6.4 16
Oz Yes Yes Yes
“8000000” coke $9.99 16
Oz Yes Yes Yes
“9000000” new1 $2.5 16
Oz Yes Yes Yes
HBase
Architecture
|
Column
family

52. 52©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Column
family
• Set
of
columns
that
have
similar
access
patterns
• Data
stored
in
separate
files
• Tune
performance
• In-­‐memory
• Compression
• Version
retention
policies
• Cache
priority
• Needs
to
be
specified
by
the
user

53. 53©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
info:
name
info:
price
info:
weight
“1000000” snickers $9.99 4
Oz
“2000000” new3 $6.4 16
Oz
“3000000” almonds $9.99 8
Oz
available:
store1
available:
store2
available:
store3
“1000000” Yes Yes Yes
“2000000” Yes Yes Yes
“3000000” Yes No Yes
Region
Column
Family
Column
Family

54. 54©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Put
1. Client
creates
a
row
to
put.
2. Client
checks
which
RegionServer
hosts
this
row
3. Row
is
written
into
write-­‐ahead
log
(WAL)
on
disk
for
persistence
4. Row
is
written
to
MemStore
(in
memory
storage)

55. 55©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Put
RegionServer
2
Put
Region
Column
Family Column
Family
MemStore (RAM) MemStore (RAM)
WAL
(disk)

56. 56©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
HFile
Region
RegionServer
2
Put
Column
Family Column
Family
FlushFlush
MemStore (RAM) MemStore (RAM)
WAL
(disk)
On
Disk On
Disk
HFiles

57. 57©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
HFile
• As
more
rows
are
added
to
the
table
• Write
to
WAL
• Write
to
MemStore
• When
MemStore
gets
full,
its
contents
are
flushed
to
HDFS
as
Hfiles

58. 58©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
HFile
Region
RegionServer
2
Column
Family Column
Family
MemStore (RAM) MemStore (RAM)
On
Disk On
Disk
• Side
effect
of
more
HFiles in
a
Region
• More
disk
seeks
on
read
• More
disk
usage
(overwritten
value,
deletes,
etc)

59. 59©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Compactions
Merging
multiple
Hfiles into
a
single
Hfile.
• Minor
compactions
• Major
compactions

60. 60©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
Region
RegionServer
2
Column
Family Column
Family
On
Disk On
Disk
MemStore (RAM) MemStore (RAM)

61. 61©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
Region
RegionServer
2
Column
Family Column
Family
On
Disk On
Disk
MemStore (RAM) MemStore (RAM)

62. 62©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
Region
RegionServer
2
Column
Family Column
Family
On
Disk On
Disk
MemStore (RAM) MemStore (RAM)

63. 63©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Compactions
• Minor
compactions
• Can
be
finely
tuned
using
multiple
configurations
• Controlled
by
policy
(pluggable)
• Major
compactions
• Periodic
(set
from
configuration)
or
manually
triggered.
• Tend
to
be
run
during
off-­‐peak
times.

64. 64©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Splits
• Eventually,
Regions
become
imbalanced.
• HBase
can
split
a
Region
into
two.
• Daughter
Regions
are
can
be
moved
to
a
different
RegionServer,
if
necessary.
Gives
HBase
horizontal
scalability!

65. 65©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Splits
Region
RegionServer
2
10
20
30
40
50
60
70
80
90
100
• Region
has
rows
[0,
100)
• Splitting
• Middle
of
current
Region’s
row
range
• All
column
families
are
split

66. 66©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Splits
Region
RegionServer
2
10
20
30
40
50
• Splits
• [0,
50)
• [50,
100)
60
70
80
90
100

67. 67©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Bulk
Import
• This
write
path
is
durable,
but
if
you’re
importing
a
lot
of
data
• Every
put
goes
into
WAL,
which
means
disk
seeks.
Lots
of
puts
mean
lots
of
disk
seeks.
• Lots
of
data
into
MemStores means
lots
of
flushing
to
disk.
• Lots
of
flushing
to
disk
might
mean
lots
of
compactions.
So
we
have
other
better
options
like….

68. 68©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
Bulk
Loading
• Bypass
conventional
write
path
• Extract
data
from
source
• Transform
data
into
HFiles (done
with
MapReduce job)
directly
• Tell
RegionServers
to
serve
these
HFiles

69. 69©
Cloudera,
Inc.
All
rights
reserved.
HBase
Architecture
|
APIs
• Java
API
• Most
full-­‐featured.
• REST
API
• Easily
accessible.
• Thrift
API
• Support
for
many
languages
(e.g.
C,
C++,
Perl,
Ruby,
Python).

70. 70©
Cloudera,
Inc.
All
rights
reserved.
Enough
of
Architecture

71. 71©
Cloudera,
Inc.
All
rights
reserved.
That’s
all
folks!

72. 72©
Cloudera,
Inc.
All
rights
reserved.10/17/14 Strata+Hadoop world 2014. George and Hsieh
Try
Hadoop
Now
cloudera.com/live

73. 73©
Cloudera,
Inc.
All
rights
reserved.10/17/14 Strata+Hadoop world 2014. George and Hsieh
Join
the
Discussion
Get community
help or provide
feedback
cloudera.com/community

74. 74©
Cloudera,
Inc.
All
rights
reserved.
Sources
• A
Survey
of
HBase
Application
Archetypes
• Lars
George,
Jon
Hsieh
• http://www.slideshare.net/HBaseCon/case-­‐studies-­‐session-­‐7
• Hadoop
and
HBase:
Motivations,
Use
cases
and
Trade-­‐offs
• Jon
Hsieh’s
slides

75. 75©
Cloudera,
Inc.
All
rights
reserved.
Questions
?