NoSQL Landscape and a Solution to Polyglot Persistence

NoSQL Landscape and a
Solution to Polyglot
Persistence

© Impetus Technologies
Agenda
• Big Data Problems
• Transition from RDMS to NoSQL
• NoSQL Landscape
• Challenges in transition
• Tools for NoSQL
• Kundera – an open source polyglot solution
Recorded version available at http://bit.ly/1hfz4Tn

BIG Data Problem

Why not RDBMS?
Scalability
Data format
High availability
 Data volume in zeta
byte, yottabyte
 Horizontal scaling
would be expensive
 Data format can be
static or dynamic
 Relational / Non-
relational
 Data locality
 No single point
of failure

Non-RDBMS way
Scale out
Scale up Static schema
Dynamic schema
Centralized
Decentralized

Introduction to NoSQL
“An approach to storing and retrieving data with horizontal scaling, simple
design and high availability”
Data format driven
processing
Distributed with No
single point of
failure(SPOF)
Thinking out of SQL
box

NoSQL :A Pragmatic Solution?
With NOSQL data can be consistent, highly available
and with no SPOF!
But not 100%!

CAP Theorem
Consistency
Availability
Partition
Tolerance N/A

Thinking NoSQL?
Size
Format
VelocityFiltering
Large Data

Size
High data growth ! scalability is an issue?
Traditional RDBMS based solutions will not work!
xxx
xxx
xxx

Velocity
Near real time/Big Data analytics
Parallel processing, ready-for-read design is required
Traditional RDBMS solutions are not
fast enough to meet the SLAs !

Filtering
Filtering. Fraud detection
Risk management analysis
Traditional RDBMS may work on small
scale but not with large data !

Format
Non relational data format.
Different nature of data set: graph based, key-value based access
Traditional database is limited to static tables!
lo
g
s

NoSQL Landscape
NOSQL

Transition to NoSQL
Datastore
Selection
API
exploration
Landscape
Understanding
Implementation

Selecting a NoSQL Datastore
Neo4j, Titan,
Objectivity,
Orient DB,
Vertex DB
Cassandra,
HBase,
Hypertable,
BigTable
Oraclekv, Redis,
Couch DB, Riak
MongoDB,
Couch base
Graph Columnar
Key-value Document

High Level APIs
 Kundera
 Kundera
 Kundera
 Hector
 Easy Cassandra
 Datastax java
driver
 Astyanax
 Morphia
 Data Nucleus
 Jongo
 Spring data
 Spring data
 Neo4j
 Hibernate OGM
 Data nucleus
 Hbase api
 Spring data
 Kundera

Hybrid Design
Cassandra, HBase RDBMS Redis
MongoDB, Couchbase Neo4J, Titan Hadoop, Spark

Bumpy Ride!
Unlearn and Learn
new APIs!
Index based retrieval
over multiple NOSQL
data stores
Atomic operations
NOSQL world is still
evolving, may need to
explore among data stores
Migration of existing
production applications and
many more…

One Stop Solution
Master key, possible?
Let’s explore!

Polyglot Way
Migrating existing
solutions
Guarantee
atomicity
Switch
databases

High Level Polyglot API
 Spring data
 Kundera
 Spring data
 Kundera
 Spring data
 Kundera
 Spring data
 Kundera
Let’s implement in JPA way!

Kundera to the Rescue!!
Supports 8 data stores –
Cassandra, Hbase,
MongoDB, Redis, Neo4j,
Oracle NoSQL, CouchDB
and any RDBMS
CRUD / Strong Query
Support
Object Relationships
Handling
Datastore-Optimized
Persistence and Query
Approach
Interceptors / Events /
Caching
Connection Pool / Fallback
(Lucene) Indexing
Flexibility

Getting Started

User Logs Sample App
@Entity
@Table(name = "user“)
@IndexCollection(columns = { @Index(name = "emailId") })
public class User {
@Id
@Column(name = "user_id")
private String userId;
@Column(name = "first_name")
private String firstName;
@Column(name = "last_name")
private String lastName;
@Column(name = "emailId")
private String emailId;
@OneToMany(cascade = CascadeType.ALL, fetch = FetchType.LAZY)
@JoinColumn(name = "user_id")
private Set<userLogs> logs;
@Embedded
private PersonalDetail personalDetail;
public User() {
// Default constructor.
}
//Setters and Getters
@Entity
@Table(name = “logs”)
@Index(columns = { "body", “created_at" }, index = true)
public class UserLogs {
@Id
@Column(name = “log_id")
private String logId;
@Column(name = "body")
private String body;
@Column(name = “created_at")
@Temporal(TemporalType.DATE)
private Date createdDate;
public UserLogs() {
// Default constructor.
}
// Setters and Getters
User Entity UserLogs Entity

User Logs Sample App
Configuration : Persistence.xml

<persistence-unit name=“logCassandra">
<provider>com.impetus.kundera.KunderaPersistence</provider>
<class>com.impetus.kvapps.entities.UserLogs</class>
<exclude-unlisted-classes>true</exclude-unlisted-classes>
<properties>
<property name="kundera.nodes" value="localhost" />
<property name="kundera.port" value="9160" />
<property name="kundera.keyspace" value=“userstore" />
<property name="kundera.dialect" value="cassandra" />
<property name="kundera.client.lookup.class"
value="com.impetus.client.cassandra.thrift.ThriftClientFactory" />
<property name="kundera.ddl.auto.prepare" value="create" />
<property name="index.home.dir" value="lucene"/>
</properties>
</persistence-unit>

<persistence-unit name=“logRdbms">
<class>com.impetus.kvapps.entities.User</class>
<properties>
value="com.impetus.client.rdbms.RDBMSClientFactory" />
<property name="hibernate.hbm2ddl.auto" value="create" />
<property name="hibernate.show_sql" value="false" /><property
name="hibernate.format_sql" value="false" />
<property name="hibernate.dialect"
value="org.hibernate.dialect.MySQL5Dialect" />
<property name="hibernate.connection.driver_class"
value="com.mysql.jdbc.Driver" />
<property name="hibernate.connection.url"
value="jdbc:mysql://localhost:3306/userstore" />
<property name="hibernate.connection.username" value="root" />
<property
name="hibernate.connection.password" value="root" />
</propertie>
</persistence-unit>

Switching Data stores

<persistence-unit name=“logCassandra">
<class>com.impetus.kvapps.entities.userLogs</class>
<properties>
<property name="kundera.keyspace" value=“userstore" />
<property name="kundera.dialect" value="cassandra" />
value="com.impetus.client.cassandra.thrift.ThriftClientFactory" />
<property name="index.home.dir" value="lucene"/>
</properties>
</persistence-unit>

<persistence-unit name=“logMongo">
<class>com.impetus.kvapps.entities.User</class>
<properties>
<property name="kundera.keyspace" value=“userlstore" />
<property name="kundera.dialect" value="mongodb" />
value="com.impetus.client.mongodb.MongoDBClientFactory" />
</properties>
</persistence-unit>
//create entity manager factory.
EntityManagerFactory emf = Persistence.createEntityManagerFactory(“logCassandra,logMongo”, properties);
EntityManager em = emf.createEntityManager();
…..
em.persist(user);
Configuration : Persistence.xml
Persist Data

Performance & Benchmarks

Technical Challenges Addressed!
• Distributed indexing over multiple NOSQL database e.g. Solr,
Elastic search
• Plugin Kundera powered ES or Lucene indexer
• Build your own library and simply plugin
• Unlearn and Learn new APIs!
• Based on most popular JPA 2.0 specification
• Atomicity guarantee and Transaction management
• Built in support for JPA/JTA transaction and batch operations
• NOSQL world is evolving, plan to switch databases?
• Since it’s a JPA powered solution, reuse same code with almost no changes

Q&A
Big Data Solutions and Services partner for Enterprises
bigdata@impetus.com

ThankYou!
• Meet us at
• Hadoop Summit, San Jose
• CIO Big Data Summit, Texas
• Strata Conference + Hadoop World, New York
• Gartner Symposium, Orlando
• Try / Recommend Kundera
• https://github.com/impetus-opensource/Kundera
• @impetustech

NoSQL Landscape and a Solution to Polyglot Persistence

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Mehr von Impetus Technologies

Mehr von Impetus Technologies (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

NoSQL Landscape and a Solution to Polyglot Persistence

Hinweis der Redaktion