This webinar discusses polyglot persistence, which is the strategy of using multiple data storage technologies together to solve different data problems. It explains that while relational databases are good for transactions and consistency, NoSQL databases are better for scale and unstructured data. The webinar shows how to integrate SQL and NoSQL databases by routing requests based on data type or synchronizing data automatically between the databases. It provides an example architecture using a SQL database for legacy apps and reporting with a NoSQL database for mobile and web apps, and discusses benefits like scalability, accelerated development, and leveraging existing tools.
2. SQL, NOSQL, AND POLYGLOT PERSISTENCE
• Microsoft published a book
on 10/15/13 on the benefits
of combining SQL data and
unstructured data
• This webinar explores the
practical implementation of
this strategy
3. QUESTIONS TO ANSWER
•
•
•
•
•
What is NoSQL?
What is Polyglot Persistence (aka hybrid)?
How do you integrate SQL and NoSQL?
Why is it important?
How does a hybrid .NET, SQL and NoSQL Database benefit
your applications?
• What are the steps to enable your apps to use a
hybrid solution?
4. WHAT IS NOSQL?
A NoSQL (Not Only SQL) database provides a mechanism for storage and
retrieval of data that employs less constrained consistency models than
traditional relational databases. Motivations for this approach include
simplicity of design, horizontal scaling and finer control over availability.
(Source: Wikipedia)
•
•
•
•
•
Distributed computing – scale out versus scale up
Fault tolerance through mirroring/replication
Ability to be schema-less and handle unstructured data
Eventually consistent as opposed to fully consistent
Several approaches – Key Value, Document, Graph etc
6. COMPARISON OF SQL AND NOSQL
A NoSQL database, as the name implies, provides a different storage and processing model than a
traditional SQL based relational database.
RELATIONAL
DOCUMENT
{
“_id” : ObjectId(“4fehbf26
s
{
“frt_name” : “Jeremy”,
i
“_id” : ObjectId(“4fehbf26
s
“last_name” : “Smith”,
{
“frt_name” : “Jeremy”,
i “address” : {
“_id” : ObjectId(“4fehbf26 Glendon
s
“last_name” ::“Smith”,
“street”
“1100
{
“frt_name” : “Jeremy”,
i “address” : {
Avenue”,
“_id” : ObjectId(“4fehbf26 Glendon
s
“last_name” ::“Smith”,Angeles”,
“street”
“city” : “Los
“1100
“frt_name” :“state” : “CA”
i “address”“Jeremy”,
Avenue”,{
:
“last_name” }:“Smith”,Angeles”,
“street”: : “LosGlendon
“city” “1100
“address” : }
Avenue”,{
“state” : “CA”
“street”}: : “LosGlendon Avenue”,
“city” “1100 Angeles”,
“city” :}“Los “CA”
“state” : Angeles”,
“state” : “CA”
}
}
}
}
KEY-VALUE
7. NOSQL STRENGTHS AND WEAKNESSES
Strength
Multi-node scalability
Weakness
Guaranteed data consistency
Source: Data Access for Highly-Scalable Solutions:
Using SQL, NoSQL, and Polyglot Persistence
8. COMPARISON OF SQL AND NOSQL
SQL Server
NoSQL
Consistency Model
Fully consistent
Eventually Consistent
Scalability
Single Server (Scale up)
Multiple Servers (Scale out)
Fault Tolerance
HA through Secondary fail over
Server
Fault tolerance through
Mirroring/Replication
Data Design
DBA does data design first
Start schema-less and implement data
structure as you code
Data types
SQL/Relational Data
Table and/or unstructured data
Strengths
Transactions/data
consistency/Robust BI/Reporting
and tooling
Scale/performance/ease of
development and adaptation
So, which is best? BOTH!
9. WHAT IS POLYGLOT PERSISTENCE?
Polyglot Persistence, like polyglot programming, is all about choosing the
right persistence option for the task at hand.
Scott Leberknight
Different databases are designed to solve different problems. Using a single
database engine for all of the requirements usually leads to non- performant
solutions.
Martin Fowler
Polyglot Persistence is about using hybrid storage approach (RDBMS,
NOSQL,BLOB,FILE) that allows you to use the best tool for the job versus
being locked into one approach.
FatCloud
10. WHAT DOES POLYGLOT PERSISTENCE LOOK LIKE?
Source: Data Access for Highly-Scalable Solutions:
Using SQL, NoSQL, and Polyglot Persistence
11. METHOD 1: HOW TO INTEGRATE SQL WITH NOSQL
Data Driven Request Routing
Calls for relational and
transactional data
SQL Server
Invoice
Invoice
Invoice
024
175
832
Web / Mobile or
other Client app
Calls for unstructured
and scale sensitive data
NoSQL Server Cluster
Invoice
Invoice
Invoice
936
492
751
Invoice
Invoice
Invoice
595
037
275
Invoice
Invoice
Invoice
275
Invoice
037
Invoice
936
Invoice
275
Invoice
037
Invoice
936
Invoice
275
037
936
Invoice
Invoice
Invoice
832
Invoice
492
Invoice
024
Invoice
832
Invoice
492
Invoice
024
Invoice
832
492
024
Invoice
Invoice
Invoice
751
Invoice
175
Invoice
595
Invoice
751
Invoice
175
Invoice
595
Invoice
751
175
595
12. METHOD 2: HOW TO INTEGRATE SQL WITH NOSQL
Automatic Synchronization
FatDB Server Cluster
SQL Server
Invoice
Invoice
Invoice
024
175
832
Invoice
Invoice
Invoice
936
492
751
Invoice
Invoice
Invoice
595
037
275
Automate routine
synchronization
tasks for effortless
data consistency.
SQL Write Back
Invoice
Invoice
Invoice
275
Invoice
037
Invoice
936
Invoice
275
Invoice
037
Invoice
936
Invoice
275
037
936
Invoice
Invoice
Invoice
832
Invoice
492
Invoice
024
Invoice
832
Invoice
492
Invoice
024
Invoice
832
492
024
Invoice
Invoice
Invoice
751
Invoice
175
Invoice
595
Invoice
751
Invoice
175
Invoice
595
Invoice
751
175
595
1. Changes to SQL Server automatically sent to FatDB
2. Changes to FatDB automatically sent to SQL Server
13. EXAMPLE OF A POLYGLOT DATA STORE
Mobile Clients
1. Legacy apps continue unchanged
updating SQL Server
2. Changes in SQL Server are
transmitted to FatDB
3. FatDB is accessed by
mobile, web, cloud clients delivering
scale and fault tolerance
3
New Web App
FatDB Server Cluster (In house or In cloud)
Invoice
Invoice
Invoice
Invoice
275
Invoice
037
Invoice
936
Invoice
275
Invoice
037
Invoice
936
275
037
936
Invoice
Invoice
Invoice
Invoice
751
Invoice
175
Invoice
595
Invoice
751
Invoice
175
Invoice
595
751
175
595
Invoice
Invoice
Invoice
832
Invoice
492
Invoice
024
Invoice
832
Invoice
492
Invoice
024
832
492
024
2
4. Updates to FatDB are transmitted
back to SQL Server
4
SQL Server
Invoice
Invoice
Invoice
024
175
832
5. BI and Reporting tools can continue
accessing SQL Server as an
accurate data repository
1
Legacy Apps
Invoice
Invoice
Invoice
936
492
751
Invoice
Invoice
Invoice
595
037
275
BI & Reporting
5
14. WHY IS POLYGLOT (a.k.a HYBRID) IMPORTANT?
“…the advent of NoSQL databases has meant that many application
developers, designers, and architects are now looking to apply the most
appropriate means of data storage to each specific aspect of their systems,
and this may involve implementing multiple types of database and
integrating them into a single solution.”
Microsoft, “Data Access for Highly Scalable Solutions”, 2013
• One size may not fit all that well
• We can now use the best tools or toolset for the job
15. HOW DOES A HYBRID SOLUTION BENEFIT YOU
1.
2.
3.
4.
5.
6.
7.
More scalable, less cost
Preserve legacy apps
Accelerate time to market
New application functionality
Preserve proven BI and reporting tools
Improved data mining
Superior elasticity
16. 10 TIPS FOR CREATING A HYBRID SOLUTION
1. Think holistically, do your traffic and data analysis
2. Test and meter
3. Minimize architectural complexity, “Missions” trump “Services”
4. Minimize “glue” code and impedance mismatches, standardize
5. Don’t paint yourself into a corner, be “Agile”
6. Minimize maintenance and configuration overhead
7. Look for portability and flexibility
8. Look for strong RDBMS integration
9. Look for an integrated extensible framework
10.Look for good support and resources
Not only SQL, a different way of thinking about storage. Not relational, but relations can, in fact exist, but they do in code.Designed for unstructured data…Uses a cluster of low cost servers to drive performance, mirror the data, and scale-out the database. This method also allows for high availability and fault tolerance, whereas individual compute nodes may fail, but the data is still available within the clusterRelational - Highly-structured table organization with rigidly-defined data formats and record structure.Document - Collection of complex documents with arbitrary, nested data formats and varying “record” format.Key-Value - An item of data that is identified by an arbitrary name. The key is the name, and the value is the content.BenefitsFLEXIBEL DATA MOdelMANAGING BIG DATASCALABILITY AND ELASTICITYLOWER TCOSql database and relation strengths are for OLTP and ACID requirements.It has been estimated 80% data stored in SQL does not require a relation model and would benefit from the NoSQL model
We cantake the first diagram and make a generic e-commerce version out of easily enough. And draw and analogy to how it would be done with straight SQL
Relational - Highly-structured table organization with rigidly-defined data formats and record structure.Document - Collection of complex documents with arbitrary, nested data formats and varying “record” format.Key-Value - An item of data that is identified by an arbitrary name. The key is the name, and the value is the content.BenefitsFLEXIBEL DATA MOdelMANAGING BIG DATASCALABILITY AND ELASTICITYLOWER TCOSql database and relation strengths are for OLTP and ACID requirements.It has been estimated 80% data stored in SQL does not require a relation model and would benefit from the NoSQL model