The Why, When, and How of NoSQL - A Practical Approach

The Why, When, and How of NoSQL:
A Practical Approach
David Segleau
Dir.Technical Product Marketing
Couchbase

©2016 Couchbase Inc. 2
About the speaker – David Segleau
David Segleau
DirectorTechnical Product Marketing
Couchbase (since Nov 2015)
Experience:
- Database guy
- Oracle, Sleepycat, Informix, Illustra,Teradata
- VP Eng, Prod Mgmt, QA, Support,Training, Docs
- Technology is only useful when it’s deployed
- Expertise:
- Database server technology, RDBMS, and NoSQL

What’s Couchbase?
Couchbase is the company behind Couchbase Server & Couchbase Mobile
• Open source JSON database
• Founded 2010
• 500+ enterprise customers globally
Some of our customers:
Couchbase Server can be deployed as:
Document database Key-value store Distributed cache

Today’s agenda
 Why NoSQL?
 Identifying the right application
 Modeling your data
 Accessing your data
 Installing and scaling your database
 Migrating your data
 Q & A

What is NoSQL?
 NoSQL?
 Not only SQL?
Non relational?
 Distributed (most)
– Scaled out, not up
• Elasticity and commodity hardware
– Partitioned and replicated
• Scalability, performance, availability
 Schema-less (most)
– Flexible model
– JSON (some)
 Multi-model
– Key-value & Document
– Columnar & Graph
– Graph & Key-value

Who is using NoSQL?
Enterprises are adopting NoSQL for mission critical applications
Media & Publishing eCommerce Hospitality

Who is using NoSQL?
 Gannett, publisher of 90+ media properties, replaced relational database
technology with NoSQL to power its digital publishing platform.
 eBay, with over 2 billion page views per day, uses Couchbase + RDBMS for
their Listing cache, and Couchbase as database of record forToken
management.
 Marriott deployed NoSQL to modernize its hotel reservation system that
supports $38 billion in annual bookings.
 FHLBankTopeka leverages NoSQL on top of SQL Server to speed up
access to customer financial data for its 770 member banks.
 Cars.com, with over 30 million visits per month, replaced SQL Server with
NoSQL to store customer and vehicle data.

Why are they using NoSQL?
Technology Drivers
 Customers are going online
 The internet is connecting everything
 Big Data is getting bigger
 Applications are moving to the cloud
 The world has gone mobile
Technical Needs
 Develop with agility
– Flexibility + Simplicity
– Easier + Faster
 Operate at any scale
– Elasticity + Availability
– Performance at scale
– Always-on, global deployment
Business Needs
 Innovate and compete
– Faster time to market
– Reduced costs (operational + hardware)
– Increased revenue

NoSQL vs. RDBMS
 Replace or Complement?  It depends
– Replace: NoSQL is often the operational
database of record
– Complement: NoSQL adds perf, scale, and
availability to legacy RDBMS
 Customers use RDBMS and NoSQL
 NoSQL is adding RDBMS features
– Security,QueryLanguage,Analytics
 RDBMS is adding NoSQL features
– Sharding, JSON, Distributed Processing

Why migrate from an RDBMS to NoSQL?
 Easier to scale
3 nodes to 100s, 1 data center to many, commodity hardware
 Better performance
Integrated caching, memory-optimized indexes, memory-based replication
 Up to 40x lower cost
Open source, subscription-based, per instance (not per core)
 Cross-platform
Runs onWindows or Linux (Red Hat, Ubuntu, Debian, etc.)
 Greater agility
JSON-based data model, SQL-based query language

How do you get started?
1. Identify the right application
2. Model your data
3. Access your data
4. Install and scale your database
5. Migrate your data

Identifying the right application

Identifying the right application
Have one or more of the following characteristics or requirements:
 Innovate and iterate faster
 Send and receive JSON
 Provide low latency at any throughput
 Support many concurrent users
 Supports users anywhere and everywhere
 Be available 24x7
 Store terabytes of data
 Read and write to multiple data centers
Service
RDBMS
Service Service
NoSQL
Application
Examples:
 High performance, high availability caching service
 Independent application with a narrow scope
 Logical or physical service within a large application
 Global service that powers multiple applications

Model your data

Demystifying terminology
Relational NoSQL (Couchbase)
Failover Cluster Cluster
Availability Group Cluster
Database Bucket
Table Bucket
Row (Tuple) Document (JSON)
Primary Key Object ID
IDENTITY or Sequence Counter
Indexed View View
SQL N1QL

Modeling your data: Fixed vs. self-describing schema

Modeling your data:The flexibility of JSON
Same document type,
Different fields
• Different types
• Optional
• On demand
Tip:Add a version field to track changes.
{“docType”: “user”, “docVersion”: “1”, …}
{“docType”: “user”, “docVersion”: “2”, …}

Modeling your data: Changing the data model
Relational database
• Modify the database schema
• Modify the application code (e.g., Java)
• Modify the interface (e.g., HTML5/JS)
Document database
• Modify the interface (e.g., HTML5/JS)

Modeling your data: Object IDs
Best Practices
• Natural Keys
• Human Readable
• Deterministic
• Semantic
Examples
• author::shane
• author::shane::blogs
• blog::nosql_fueled_hadoop
• blog::nosql_fueled_hadoop::comments
What about identity columns?
1. Document<Long> nextAuthorIdDoc = bucket.counter(“authorIdCounter”, 1);
2. Long nextAuthorId = nextAuthorIdDoc.content();
3. String authDocId = “author::” + nextAuthorId; // author::101
Tip: Increment the counter by 10, 20, etc. instead of doing it for every insert.

Modeling your data: Relationships
Author
Blog (FK)Blog (FK)
Comment (FK) Comment (FK)
Author (FK x2)
BlogBlog (FK x2)
Comment Comment
Bottom up Top down

Modeling your data: Relationships

Modeling your data: Strategies and best practices
If … Then …
Relationship is one-to-one or one-to-many Store related data as nested objects
Relationship is many-to-one or many-to-many Store related data as separate documents
Data reads are mostly parent fields Store children as separate documents
Data reads are mostly parent + child fields Store children as nested objects
Data writes are mostly parent or child (not both) Store children as separate documents
Data writes are mostly parent and child (both) Store children as nested objects

Modeling your data: Strategies and best practices
 Are there a lot of concurrent writes, continuous updates?
 Store children as separate documents
Blog
 Thread
 Comment
 Comment
 Thread
 Comment
 Comment
Blog
{
“docType”: “blog”,
“author”: “author::shane”,
“title”: “Couchbase Wins”,
“threads”: [
“blog::couchbase_wins::threads::001”,
“blog::couchbase_wins::threads::002”
}
Thread
{
“docType”: “thread”,
“comments”: [
{
“visitor”: “Brendan Bond”,
“text”: “This blog is amazing!”
“replies”: [
{
“user”: “Dustin Johnson”,
“text”: “No, it is not.”
}]
}
}

Access your data

Accessing your data: Options
Key-Value
(CRUD)
N1QL
(Query)
Views
(Query)
Documents
Indexes MapReduce
FullText
(Search)
Geospatial
(Search)
We’ll focus on these three for now.
Indexes MapReduce

Accessing your data: Connecting to the database
 Access data via topology-aware smart clients
 Maintains an up-to-date cluster map
 Communicates directly with database nodes – no proxies, no routers, etc.
 Available for Java, Node.js, PHP, .NET, Python, C, Go, and more
 With standard, certified JDBC/ODBC drivers (if you want to)

Accessing your data: Domain objects vs. document objects
* JSON serialization via Boon.
Working with document objects requires
less code, provides more flexibility.

Accessing your data: Key-value operations – referenced data

Accessing your data: Key-value operations – nested data

Accessing your data: Subdocument operations

Accessing your data – N1QL queries: Capabilities
Feature SQL N1QL
JOIN ✔ ✔
TRANSFORM ✔ ✔
FILTER ✔ ✔
AGGREGATE ✔ ✔
SORT ✔ ✔
SUBQUERIES ✔ ✔
PAGINATION ✔ ✔
OPERATORS ✔ ✔
FUNCTIONS ✔ ✔

Accessing your data: N1QL queries – referenced data

Accessing your data: N1QL queries – nested data

Accessing your data: N1QL queries – CRUD

Accessing your data: LINQ
_context.Query<Users>().Where(u => u.status == “Platinum”)
from user in _context.Query<Users>()
join account in _context.Query<Account>()
on user.accountId equals N1QlFunctions.Key(account) into userGroup
from accounts in userGroup.DefaultIfEmpty()
where (account.type == “Visa” || account.type == “MasterCard”)
select new { firstName = user.firstName, lastName = user.LastName };
from user in _context.Query<Users>()
join address in _context.Query<Address>()
on user.addresses.shipping.addressId equals N1QlFunctions.Key(address) into addressGroup
from address in addressGroup.DefaultIfEmpty()
where address.state == “CA”
select new { firstName = user.firstName, lastName = user.LastName };

Accessing your data: N1QL queries – indexes
Simple
Compound
Functional
Partial

Accessing your data:Views
What if indexed views worked great with
write intensive workloads?
And you could use COUNT,ORDER BY, and everything else …

Accessing your data:Views

Accessing your data:Views – Incremental MapReduce

Accessing your data:Views – queries

Accessing your data: Strategies and best practices
Concept Strategies & Best Practices
Key-Value Operations provide the best
possible performance
• Create an effective key naming strategy
• Create an optimized data model
Incremental MapReduce (Views) are well
suited to aggregation
• Ideal for large data sets, the entire data set
• Can be used to create complex secondary
indexes
N1QL queries provide the most flexibility –
everything else
• Query data regardless of how it is modeled
• Remember to create indexes, leverage
covering indexes where possible

Install and scale your database

Installing and scaling your database
Admin screenshot – add
node
1. Download
2. Install
3. Configure
4. Cluster / Scale
5. Rebalance

Installing and scaling your Couchbase database: XDCR
 MultipleDataCenters
 Cluster per data center
 Replicate between clusters
 Unidirectional / bidirectional
 Master / Master
 Local reads and writes
 Ring
 Hub-and-spoke
 Mesh
 Combination
 Built-in

Installing and scaling your Couchbase database: MDS
 Multi-Dimensional Scaling
 SOA for databases
 Independent service scalability
 Scale out and scale out
 Scale Data, Query, Indexing and
FullText services Scale out
Scale up

Migrate your data

So many options! Remember the KISS principle
1) Identify the requirements
• ETL vs. Data cleanse vs. Data enrichment
• Duration vs. Resources
• Data governance
2) Pick your strategy
• Batch vs. Incremental
• Single threaded vs. multi-threaded
3) Pick your tools
• Data migration tools (Informatica, Looker,
Talend)
• BYO-tool (PHP & Python scripts, Hadoop, Spark)
• KISS with Couchbase
• Export to CVS; Import as documents; Use
N1QL to transform & insert into new
bucket
• Use SQL to transform & export; Insert into
Couchbase
• Best Practices
• Align with your data model
• Plan for failure (bad source data, hardware
failure, resource limitations)
• Ensure interruptible, restartable, logged,
predictable

How can you sync NoSQL and relational?
 1.ApplicationCode(Manual)
 2. Replication (Automatic)
– From NoSQL to relational
– From relational to NoSQL
Couchbase
Kafka
Queue
Producer Consumer RDBMSDCP
Stream
RDBMS Handler
CouchbaseGoldenGate
https://github.com/mahurtado/CouchbaseGoldenGateAdapter

Where to go next?
Conduct a Successful Proof of Concept
1. Select a use case and application
2. Define the success criteria
3. Understand the data
4. Identify the access patterns
5. Review the architecture
Measure your Return on Investment
 Greater agility?
 Faster time to market?
 Easier scalability?
 Better performance?
 Better availability?
 Lower costs?

Questions?

Best practices recap
• Pick the right application
• Focus on SOA, application/use case specific
• Drive data model from data access patterns
• Use Document type, Versionid
• Create optimized, understandible keys
• Weigh nested, referenced or mixed designs
• Add indexes: Simple, Compound, Functional, Partial, Covering, Memory Optimized
• Match the data access method to requirements
• Key-value, Views, N1QL
• Proof of Concept
• Focus, Success Criteria, Review Architecture

Want to learn more?
Getting Started guide:
http://www.couchbase.com/get-started-developing-nosql
Download Couchbase software:
http://www.couchbase.com/nosql-databases/downloads
Free OnlineTraining
http://training.couchbase.com/online
“Why NoSQL” white paper
http://www.couchbase.com/nosql-resources/why-nosql

Thank you

The Why, When, and How of NoSQL - A Practical Approach

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (12)

Similar to The Why, When, and How of NoSQL - A Practical Approach

Similar to The Why, When, and How of NoSQL - A Practical Approach (20)

More from DATAVERSITY

More from DATAVERSITY (20)

Recently uploaded

Recently uploaded (20)

The Why, When, and How of NoSQL - A Practical Approach