Сергей Ковалев (Senior Java Developer в Altoros)

1. © ALTOROS Systems | CONFIDENTIAL
NoSQL Databases and Polyglot
Persistence
Sergey Kovalev, Senior Java Developer, sergey.kovalev@altoros.com

2. © ALTOROS Systems | CONFIDENTIAL 2
The Challenge
“One Size Does Not Fit
All”

3. © ALTOROS Systems | CONFIDENTIAL 3
• What is Polyglot Persistence?
• RDBMS vs NoSQL
• NoSQL DB types
• Polyglot Persistence Patterns
• Real project experience
Agenda:

4. © ALTOROS Systems | CONFIDENTIAL
Polyglot Persistence -
4
Using multiple data storage technologies, chosen
based upon the way data is being used by individual
applications or components of single application.
Martin Fowler
http://martinfowler.com/articles/nosql-intro.pdf
What is Polyglot Persistence?

5. © ALTOROS Systems | CONFIDENTIAL 5
NoSQL Zooo
What is Polyglot Persistence?
http://nosql-databases.org 150+ NoSQL databases

6. © ALTOROS Systems | CONFIDENTIAL 6
What is Polyglot Persistence?
Who use polyglot persistence?

7. © ALTOROS Systems | CONFIDENTIAL 7
• What is Polyglot Persistence?
• RDBMS vs NoSQL
• NoSQL DB types
• Polyglot Persistence Patterns
• Real project experience
Agenda:

8. © ALTOROS Systems | CONFIDENTIAL
RDMS are great
8
• Widely used and understood
• Tested in real environments
• Efficient use of storage space if data normalized properly
• Great tools support ACID semantics
• Incredibly flexible and powerful query language
• Great framework support
RDBMS vs NoSQL

9. © ALTOROS Systems | CONFIDENTIAL
RDBMS have limitations
9
• Complex object graphs does not map very well with flat
tables.
• Strict Schema
• Difficult to evolve Schema with time.
• Data constraints and JOINs can be expensive at runtime.
• Difficult to scale horizontally.
RDBMS vs NoSQL

10. © ALTOROS Systems | CONFIDENTIAL
NoSQL features
10
• Flexible schema
• Easy to scale horizontally to add processing power and
storage
• High Performance
RDBMS vs NoSQL

11. © ALTOROS Systems | CONFIDENTIAL 11
• What is Polyglot Persistence?
• RDBMS vs NoSQL
• NoSQL DB types
• Polyglot Persistence Patterns
• Real project experience
Agenda:

12. © ALTOROS Systems | CONFIDENTIAL
Key Value Stores
12
• A key-value store is a simple hash table
• Primarily used when all access to the database is via
primary key
• Value is a blob with the data store not caring or knowing
what is inside
NoSQL DB types

13. © ALTOROS Systems | CONFIDENTIAL
Column-Family Stores
13
• Store data in column families as rows that have many
columns associated with a row key
• Column families are groups of related data, often accessed
together
• Aggregate-Oriented
NoSQL DB types

14. © ALTOROS Systems | CONFIDENTIAL 14
NoSQL DB types
Document Databases
• Stores and retrieves documents, which can be XML, JSON,
BSON, …
• Documents are self-describing, hierarchical tree data
structures which can consist of maps, collections and scalar
values
• Aggregate-Oriented

15. © ALTOROS Systems | CONFIDENTIAL 15
NoSQL DB types
Graph Databases
• Allow to store entities and relationships between these
entities
• Entities are known as nodes, which have properties
• Relations are known as edges, which also have properties

16. © ALTOROS Systems | CONFIDENTIAL 16
• What is Polyglot Persistence?
• RDBMS vs NoSQL
• NoSQL DB types
• Polyglot Persistence Patterns
• Real project experience
Agenda:

17. © ALTOROS Systems | CONFIDENTIAL
RDBMS
17
• Transactional data (payment and account) in a ACID compliant traditional
RDBMS stores such as MySQL or Oracle.
• Reporting Tools
Polyglot Persistence Patterns
RDBMS
Web application
Reporting Tools
Transactional
data

18. © ALTOROS Systems | CONFIDENTIAL
Key Value Stores
18
• Session data in Redis or Hazelcast. Session data requires faster read and write
but not durability.
• Shopping cart data in a high availability and fault-tolerance data store such as
Riak.
Polyglot Persistence Patterns
RDBMS
Web application
Reporting
Tools
Transactional
data
Key-Value
store
Sessions and
Shopping cart

19. © ALTOROS Systems | CONFIDENTIAL
Column-Family Stores
19
• Log level data (audit and activity) in a very high write throughput data store such
as Cassandra.
• Also good for analytic and real-time data mining such as product ranking
Polyglot Persistence Patterns
RDBMS
Web application
Reporting
Tools
Transactional
data
Key-Value
store
Sessions and
Shopping cart
Column-Family
store
Logging
Analytic

20. © ALTOROS Systems | CONFIDENTIAL
Document Databases
20
• Product catalogue in a document oriented data store such as MongoDB.
Requirement include high read throughput, frequent data change (stock level
information)
• Customer profile data including purchase history, shipping & billing address
Polyglot Persistence Patterns
Web application
Reporting
Tools
Transactional
data
Key-Value
store
Sessions and
Shopping cart
Column-Family
store
Logging
Analytic
Document
store
RDBMS
Product
catalogue

21. © ALTOROS Systems | CONFIDENTIAL
Graph Databases
21
• Data for product recommendations, related products and similar products in a
graph database such as Neo4j.
Polyglot Persistence Patterns
Web application
Reporting
Tools
Transactional
data
Key-Value
store
Sessions and
Shopping cart
Column-Family
store
Logging
Analytic
Document
store
RDBMS
Product
catalogue
Graph
Databases
Social graph

22. © ALTOROS Systems | CONFIDENTIAL
Summary
22
Polyglot Persistence Patterns

23. © ALTOROS Systems | CONFIDENTIAL 23
• What is Polyglot Persistence?
• RDBMS vs NoSQL
• NoSQL DB types
• Polyglot Persistence Patterns
• Real project experience
Agenda:

24. © ALTOROS Systems | CONFIDENTIAL
ScienceWarehouse challenges
24
• Hundreds of thousands of records are added to the
database daily
• Visitors’ activities logging generates huge amount of write
queries
• Price table contains approximately 100,000,000 records
• Price history table contains approximately 500.000.000
• Data rapidly grows
Real project experience

25. © ALTOROS Systems | CONFIDENTIAL 25
• Upgrade hardware (vertical scaling)
• Upgrade to the Oracle Database Enterprise Edition
Options for ScienceWarehause
• Use NoSQL and Polyglot Persistence
or
Real project experience

26. © ALTOROS Systems | CONFIDENTIAL 26
Real project experience
Solutions for ScienceWarehouse
• Add a caching layer
• Write the log data on visitors’ activities to a NoSQL database
(Cassandra or HBase)
• Move price history(500,000,000 records table) information to
NoSQL data base(Cassandra or HBase)

27. © ALTOROS Systems | CONFIDENTIAL 27
Real project experience
ScienceWarehouse High Level Architecture
Web application
Legacy DB
Key-Value
store
Sessions and
Shopping cart
Column-Family
store
Logging
Analytic
RDBMS

28. © ALTOROS Systems | CONFIDENTIAL 28
Sergey Kovalev, Senior Java Developer, sergey.kovalev@altoros.com
NoSQL Databases and Polyglot
Persistence