Hadoop, Hbase and Hive- Bay area Hadoop User Group

Hive/HBase Integration or, MaybeSQL? April 2010 John Sichi Facebook +

Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Motivations ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

How Can HBase Help? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Use Case 1: HBase As ETL Data Target Facebook HBase Hive INSERT … SELECT … Source Files/Tables

Use Case 2: HBase As Data Source Facebook HBase Other Files/Tables Hive SELECT … JOIN … GROUP BY … Query Result

Use Case 3: Low Latency Warehouse Facebook HBase Other Files/Tables Periodic Load Continuous Update Hive Queries

HBase Architecture Facebook From http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html

Hive CLI With HBase ,[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Storage Handler ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Column Mapping ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Load Via INSERT ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Map-Reduce Job for INSERT Facebook HBase From http://blog.maxgarfinkel.com/wp-uploads/2010/02/mapreduceDIagram.png

Map-Only Job for INSERT Facebook HBase

Query Processing ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Metastore Integration ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Bulk Load ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Range Partitioning During Sort Facebook A-G H-Q R-Z HBase (H) (R) TotalOrderPartitioner loadtable.rb

Sampling Query For Range Partitioning ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Sorting Query For Bulk Load ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Deployment ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Questions? ,[object Object],[object Object],[object Object],[object Object],[object Object],Facebook

Hey, What About HBQL? ,[object Object],[object Object],Facebook

Hadoop, Hbase and Hive- Bay area Hadoop User Group

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Hadoop, Hbase and Hive- Bay area Hadoop User Group

Similar to Hadoop, Hbase and Hive- Bay area Hadoop User Group (20)

More from Hadoop User Group

More from Hadoop User Group (18)

Recently uploaded

Recently uploaded (20)

Hadoop, Hbase and Hive- Bay area Hadoop User Group