Slides: Moving from a Relational Model to NoSQL

Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved.
Moving from a Relational Model
to NoSQL
Leveraging a Flexible JSON Schema
Matthew Groves – Product Marketing Manager

Confidential and Proprietary. Do not distribute without Couchbase consent. © Couchbase 2020. All rights reserved. 2
Objectives
• Give an overview of migrating an existing relational database to
NoSQL (Couchbase)
• Give you a sense for some of the technical issues / differences
• We will not get into deep details (i.e. dialect details, different ways
to write a query, how indexes are handled, etc)

1
2
3
4
5
Quick Intro to NoSQL
Quick Intro to Couchbase
Migration Options
Optimizing
Demo
AGENDA

Quick Intro to NoSQL

Consider + Purchase Operate + Service
Build + Fulfill
Massively
Interactive
Enterprises
e-Commerce Supply Chain Internet of Things
CUSTOMER EMPLOYEE MACHINE
Drivers of NoSQL

Application Requirements Are Dramatically Different
NoSQL
Transactions
Interactions
Transactional Analytical
SCALABLE
SECURE
FLEXIBILE
DISTRIBUTED
LOW TCO

Application Requirements Are Dramatically Different
Oracle
1K:1
Consumer
1:1
1M:1 IoT
Legacy Databases Insufficient
Relational divide
Couchbase

What is JSON?
JavaScript Object Notation
• Lightweight data interchange format
• Based on a subset of JavaScript
• Uses a hierarchical key/value syntax
Data Types
• Object: Unordered collection of key/value pairs
• Array: ordered list of 0 or more values
• Number: Integer or Float
• String: Unicode in double quotes
• Boolean: true or false
• Null: Empty Value <"">
{
"JSON": "JavaScript Object Notation",
"Desc": [
"Lightweight data interchange format",
"Based on a subset of JavaScript",
"Uses a hierarchical key/value syntax"
],
"DataTypes": {
"Object": {
"Desc": "Unordered collection of key/value pairs",
"Ex01": {
"key": "value"
}
},
"Array": {
"Desc": "Ordered list of 0 or more values",
"Ex01": [
"apples", 12, true
]
},
"Number": {
"Desc": "Integer or Float",
"Ex01": 12,
"Ex02": 1234.56789
},
"String": {
"Desc": "Unicode in double quotes",
"Ex01": "Hello World!"
},
"Boolean": {
"Desc": "true or false",
"Ex01": true,
"Ex02": false
},
"Null": {
"Desc": "Empty Value",
"Ex01": ""
}
}

Why JSON for Data?
JSON is easy to read and write
JSON is ubiquitous
• ~ 170 JSON parsing libs
• ~ 60 different languages/frameworks
• json.org – resource lists
• Standardizing in RFC 8259
JSON objects make great "documents"
• Object => "document" / "subdocument"
• Internal key/value pairs => "attributes"

{
'_type': 'user',
'age': 23,
'name': 'Mint, Mikey',
'admin': false
}
Tables vs document sets (simple)
USER
user_id age name admin
blue123-4 42 Blue, Betty true
orange456-7 31 Orange,
Oscar
false
{
'age': 42,
'name': 'Blue, Betty',
'admin': true
}
id: blue123-4
Relational DB
Tables
Rows
{
'_type': 'product',
'qty;: 1966,
'name': 'Blarg Nozzle',
'stock': true
}
{
'qty;: 1966,
'stock': true
}
id: abc123
PRODUCT
prod_id qty name stock
abc123 1966 Blarg Nozzle true
xyz456 5280 Hepto Shaft false
NoSQL Bucket
Collections
Documents

Schema “enforcement” vs schema agility
USER
Oscar
false
Relational
{
'_type': 'product',
'qty;: 1966,
'stock': true
}
{
'type': 'product',
'qty;: 1966,
'stock': true
}
id:
product::
abc123
PRODUCT
prod_id qty name stock
abc123 1966 Blarg Nozzle true
xyz456 5280 Hepto Shaft false
def321 ??? Foo Modulator false
{
'qty': 0,
'name': 'Foo Modulator',
'stock': false,
'restock': true
}
id: def321
restock = true ?
{
'_type': 'user',
'age': 23,
'name': 'Mint, Mikey',
'admin': false
}
{
'age': 42,
'admin': true
}
id: blue123-4
NoSQL

Timestamp format choices
• ISO 8601
• Time component array
• Epoch/Unix

Quick Intro to Couchbase

Couchbase | NoEQUAL
Agility/versatility
Performance
Easy to manage

What is Couchbase?
• NoSQL Database that stores JSON documents
• Includes a Key-Value Store for fast lookups
• Supports SQL and ACID transactions
• Includes Indexing
• Includes Full Text Search capability
• Memory-First, shared-nothing architecture
• Asynchronous architecture
• Masterless

PRIVATE
CLOUD
Cloud-Native: Deploy On Premise or In Any Cloud
OPTIMIZED FOR FLEXIBILITY,
CUSTOMIZABILITY AND PERFORMANCE
OPTIMIZED FOR SIMPLICITY,
PERFORMANCE AND EASE OF USE
PUBLIC
CLOUD
FULLY MANAGED
DBaaS

COUCHBASE LITE SYNC GATEWAY COUCHBASE SERVER
Lightweight embedded NoSQL
database with full CRUD and
query functionality.
Secure web gateway with
synchronization, data access,
and data integration APIs for
accessing, integrating, and
synchronizing data
over the web.
Highly scalable, highly available,
high performance NoSQL
database server.
Securely Manage & Sync Data from any Cloud to the Edge

Cross Data Center Replication (XDCR)

Migration Options

Scenario
Existing application architected on an
RDBMS, it works well, but there are a lot of
changes coming and new expected
demands in terms of scalability.

Issues to consider
• Risk tolerance
• Skills / expertise
• Migration cost
• Performance
• Scale

How do you migrate?
• 1 Rewrite: No migration, write the whole thing over
• 2 Redesign Schema: Keep your business logic, rewrite your data layer and
schema, totally redesign your schema on NoSQL best practices
• 3 Refactor First: Keep everything but refactor your data logic and RDBMS
schema into a best practices NoSQL schema
• 4 Optimize Later: Host your schema with as few changes as possible, get the
application running on the new technology, refactor/optimize the schema as
necessary for performance
• 5 Just Host It: Host your schema with as few changes as possible.
Risk
Effort

Migrating Without
Denormalizing

Conventional relational data
Normalize – model data as minimally
redundant tables, and rely on in-memory
JOINs
✔Storage minimized
✔Schema enforced
✔Ad hoc queries well supported by SQL
✔Record-centric, 2D data paradigm
CustomerID Name Address Email
XYZ234 Acme 9 W. 2nd al@acme
ProductID Name Description
774477 Widget Fit for all uses.
115588 Gadget Just what I need.
PayTypeID Name
001 PayPal
OrderID Customer PayType
123 XYZ234 001
OrderID ProductID Qty
123 774477 2
123 115588 3

Collection
Relational Database vs Bucket (simple)
USER
Oscar
false
Relational
table
{
'type': 'user',
'age': 31,
'name': 'Orange, Oscar',
'admin’: false
}
id:
user::orang
e456-7
{
'age': 42,
'admin': true
}
id: blue123-4

{
'type': 'user',
'age': 31,
'admin’: false
}
id:
user::orang
e456-7
Collection
Rows/records vs documents (simple)
Copyright © 2019 Couchbase,
Inc.
USER
Oscar
false
Relational
table
{
'age': 42,
'admin': true
}
id: blue123-4

Collection
Columns vs attributes/parameters/fields
USER
Oscar
false
Relational
table
{
'age': 31,
'admin’: false
}
id: orange456-7
{
'age': 42,
'admin': true
}
id: blue123-4

Document data
{
"date" : "2021-01-06",
"orderNum" : "ABC123",
"customer": {
"name": "Acme",
"address": "9 W. 2nd",
"email": "al@acme"
},
"items": [{
"qty": 2,
"name": "Widget",
"description": "Fit for all uses."
},{
"qty": 3,
"name": "Gadget",
"description": "Just what I need."
}]
}
order::123
ORDER
id date orderNum
123 2021-01-06 ABC123
ITEM
order_id qty name description
123 2 Widget Fit for all…
123 3 Gadget Just what I ...
CUSTOMER
order_id name address email
123 Acme 9 W. 2nd al@acme

Document data
De-normalize – optimize for planned application usage
✔Data can directly align with object model
✔Reduced JOIN and Tx pressure
✔Simpler data-layer code
✔Low latency CRUD and streaming APIs
✔Indexing for alternate access patterns
✔Queries well supported by SQL++ (N1QL)

The New Query
SELECT l.city
FROM Travel.dbo.LANDMARKS l
WHERE l.city IN
(SELECT city
FROM Travel.dbo.AIRPORTS)
RDBMS Queries
SELECT l.city
FROM Travel._ default.landmark l
WHEREl.city IN
(SELECT VALUE city
FROM Travel._ default.airport )
N1QL Queries

Query Differences
• Table = collection
• ANSI JOINs and other SQL conventions
are supported.
• There are other “N1QLisms” that are
mainly to do with JSON like “unnest” or
“nest” dealing with embedded
documents.
• Indexes are very important in
Couchbase
SELECT player.name,
player.level,
stats.loadtime,
SUM(CASE WHEN hist.result =
"won" THEN 1 ELSE 0 END) AS wins
FROM jungleville_stats AS stats
UNNEST stats.`pvp - hist` AS hist
JOIN jungleville AS player ON KEYS
stats.uuid
GROUP BY player, stats
ORDER BY wins DESC, player.level
DESC

Indexes
• meta(bucket) allows you
to refer to the field
names
• GSI = Global Secondary
Index
CREATE INDEX person_role on
profiles(
person_role_name
)
WHERE
meta(profiles).type="person"
USING GSI

Keys to success
• NoSQL Database feature support (SQL, ACID, Joins)
• Understanding
• Buckets
• Scopes
• Collections
• SQL Dialect transitions
• Optimize Later

Optimizing

Key considerations
• Joining and transacting is costly (especially in a distributed
architecture) and doesn’t scale as well horizontally.
• Your well-designed application code is a guide. “Person” contains
“Addresses” but looks up “Orders”
• Think in terms of Entities. In the scope of your application does this
have a “lifecycle” outside of the thing that always references it.
• Can you reduce the number of explicit transactions required?
• Can you reduce the number of joins?
• Have you met your performance/scale requirement

Key considerations
JOINs / ACID App code as
a guide
Entities

Reduce Joins/Transactions
{
'acct_no': 'ABC1',
'balances': {
'savings': 100,
'checking': 200
}
}
UPDATE Accounts
SET balances.savings = 100
AND balances.checking = 200
WHERE acct_no = "ABC1"
AND type = "account";
acct_no balance
ABC1 100
UPDATE savings
SET balance = 100
WHERE acct_no = "ABC1";
savings
acct_no balance
ABC1 200
UPDATE checking
SET balance = 200
WHERE acct_no = "ABC1";
checking
N1QL: Atomic document
update
SQL: ACID transaction
START TRANSACTION
COMMIT

Analyze your logical model
✔Order embeds Items, external Product (1:n) and Paytype (1:1) docs
Customer Order
1:n
Paytype
1:1
Items
1:n
Product
1:1
Name
Name
Quantity Name
Description
Price
Is this your object?

✔Order embeds Items, external Product (1:n) and Paytype (1:1) docs
Customer Order
1:n
Paytype
1:1
Items
1:n
Product
1:1
Name
Name
Quantity Name
Description
Price
Is this your object?
Embedding

Analyze your logical model
✔Order embeds Paytype and refs Items which embed Product
Customer Order
1:n
Paytype
1:1
Items
1:n
Product
1:1
Name
Name
Quantity Name
Description
Price
Are these your
objects?

Embedding + Referring
✔Order embeds Paytype and refs Items which embed Product
Customer Order
1:n
Paytype
1:1
Items
1:n
Product
1:1
Name
Name
Quantity Name
Description
Price
Are these your
objects?
Customer
Paytype

Objects vs. object arrays

Conclusions
• Migrate first, optimize later (reduces risk, increases performance/scale)
• Joins / Multi-document transactions are more costly in a distributed
architecture
• Expect a 1-1 migration to be slower, require more hardware
• Embed documents as an optimization
• SQL Queries can be converted
• ACID transactions are available (but not needed as often when
optimized)

Data Access: N1QL = SQL for JSON
Create User
Get User
Update User
Delete User
SELECT *
FROM users
WHERE users.lastName = “Johnson”
UPDATE users
SET status = “Platinum”
WHERE users.lastName = “Johnson”
DELETE
FROM users
WHERE users.firstName = “Shane”
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
INSERT INTO
VALUES
Users (KEY, VALUE)
(
“user::100”,
{
“firstName”: ”Shane”,
“lastName”: ”Johnson”
}
);

Couchbase Demo

https://www.couchbase.com/products/cloud

Slides: Moving from a Relational Model to NoSQL

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Slides: Moving from a Relational Model to NoSQL

Ähnlich wie Slides: Moving from a Relational Model to NoSQL (20)

Mehr von DATAVERSITY

Mehr von DATAVERSITY (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Slides: Moving from a Relational Model to NoSQL