Session Presented at 5th IndicThreads.com Conference On Java held on 10-11 December 2010 in Pune, India WEB: http://J10.IndicThreads.com ------------ Hbase is an open-source, non-relational, distributed, sparse, column-oriented data-store modeled after Google’s BigTable and is written in Java. In this presentation we will talk about how to migrate a RDBMS based Java application to Hbase based application. We will have a discussion on following points: • Hbase schema design (a paradigm shift from the way we think about data-storage right now) compared to RDBMS based schema design. • The challenges faced while porting the application with HBase. • Introduction to HBql to query the data from Hbase. • Monitoring example application for Hbase (JMX APIs exposed) and Machine’s performance with Gangila. • Discussion on Thrift interface and how can we used Rest interface to integrate hbase with non java based applications. • Cluster replication and what is coming in the next major 0.90 release of Hbase. • We will end up the session, with the demo of ported application. Takeaways for the Audience 1. When is Hbase appropriate and when not? 2. Hbase architecture and schema design 3. RDBMS vs Hbase 4. Interfacing Hbase with applications using Thrift or REST 5. Hbase cluster and Replication 6. Hbase monitoring

1. Gaurav Kohli Xebia Breaking with DBMS and Dating with Relational Hbase

2. me Gaurav Kohli [email_address] Consultant Xebia IT Architects About

4. Something about RDBMS

5. Limitations of RDBMS

6. Why Hbase or any NoSql solution

7. Overview of Hbase

8. Specific Use cases

9. Paradigm shift in Schema Design

10. Architecture of Hbase

11. Hbase Interface – Java API, Thrift

12. Conclusion Agenda

13. Databases Relational

14. Relational Databases have a lot of limitations

18. Master Slave Replication Replication Master Slave Scaling Out

20. All Slaves must have the same write capacity as master

21. Single point of failure, no easy failover Master Reads Writes Slave nodes Master - Many Slave Scaling Out

22. Master Master Slave Replication Dual Master

23. NoSQL

28. High-Performance

29. Storage System Hbase

31. Denormalized data

32. Wide and sparsely populated data structure(key-value)

33. No DBA needed Hbase is Not

36. Write availability

37. No Single point of failure Use Case

39. Real-time inserts, updates, and queries.

40. Fraud detection by comparing transactions to known patterns in real-time.

41. Analytics - Use MapReduce, Hive, or Pig to perform analytical queries Specific Use Case

43. Table are sorted by Row

45. Storage Model

46. Storage Model

49. Row Key+ Column Key + timestamp => value Row Key Column Key Timestamp Value 1 info:name 1273516197868 Gaurav 1 info:age 1273871824184 28 1 info:age 1273871823022 34 1 info:sex 1273746281432 Male 2 info:name 1273863723227 Harsh 3 Info:name 1273822456433 Raman 2 Versions of this row Timestamp is a long value Column Qualifier/Name Sorted by Row key and column key Column family Student table Schema Design

54. Attribute Possible Values Default COMPRESSION NONE,GZ,LZO NONE VERSIONS 1+ 3 TTL 1-2147483647(seconds) 2147483647 BLOCKSIZE 1 byte – 2 GB 64k IN_MEMORY true,false false BLOCKCACHE true,false true Column families attributes

56. Each Table is partitioned into Regions Regions

57. Regions and row200 row201 row500 row1 new row Splitting

58. Regions and row200 row201 row350 row1 row 351 row 501 Splitting

60. Zookeeper

61. RegionServers

62. HDFS

63. MapReduce Architecture

64. Architecture

65. – Java API, Thrift... Tools

67. Thrift ( Ruby, Php, Python, Perl, C++... )

68. REST

69. Groovy DSL

70. MapReduce

71. Hbase Shell Tools

73. Put

74. Delete

75. Scan

76. IncrementalColumnValue Tools

79. Solves only a small subset(~5%) Conclusion

81. Secondary Indexing

82. Referential Integrity (updates)

84. Read/Write scale

85. Replication

86. Batch analysis Conclusion

90. Hbase Wiki (wiki.apache.org/hadoop/Hbase)

91. Hbase blog (blog.hbase.org)

92. Images from Google Search

93. http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html

94. http://highscalability.com/blog/2010/12/6/what-the-heck-are-you-actually-using-nosql-for.html References & Credit