For 30 years, developers have been taught that relational data modeling was THE way to model, but as more companies adopt MongoDB as their data platform, the approaches that work well in relational design actually work against you in a document model design. In this talk, we will discuss how to conceptually approach modeling data with MongoDB, focusing on practical foundational techniques, paired with tips and tricks, and wrapping with discussing design patterns to solve common real world problems.
6. #MDBLocal
Why MongoDB?
Best way to work
with data
Intelligently put data
where you need it
Freedom
to run anywhere
Intelligent Operational Data Platform
9. #MDBLocal
MongoDB Approach
Utility of data leveraged with through query and access pattern
Data
(BSON)
Query
( CRUD /
Aggregation
Pipeline )
Access
Pattern
( Indexing )
Utility
16. #MDBLocal
Tabular ( Relational )
Capture column and value as Field Value pairs
Instead of left to right, top to bottom
Sub tables are captured as sub documents
17. #MDBLocal
Key Value
Key value captured directly as Field Value
If caching, can use TTL index
For attributes, exists as sub document
21. #MDBLocal
Combine Multiple Models in One Document
“storenumber” 101
“wifi_fence” type: "Polygon", coordinates: [ [ [ 0 , 0 ] , [ 3 , 6 ] , [ 6 , 1 ] , [ 0 , 0 ] ] ]
“amenities”
“playland” Y
“curbside_pickup” Y
“delivery” Y
“storelocation” type: "Point", [ 123, 456 ]
“sister_stores” [ 102, 104, 503 ]
“description” MongoDB’s document model is…
“employees”
“name” Bob
“name” Sue
“name” Ed
Geo
Key:value
Text
Graph
Tabular
22. #MDBLocal
Combine Data and State in the Same Document
• Relational keeps data and state information separate
• MongoDB – combine data and state in same document
• Read stateful data from Primary, read stateless data from
Secondary
25. #MDBLocal
db.customers.aggregate([
{
$unwind: "$address",
},
{
$match: {"address.location": "home"}
},
{
$group: {
_id: "$address.city",
totalSpend: {$sum: "$annualSpend"},
averageSpend: {$avg: "$annualSpend"},
maximumSpend: {$max: "$annualSpend"},
customers: {$sum: 1}
}
}
])
Versatile: Complex queries fast to create, optimize, & maintain
MongoDB’s aggregation framework has the flexibility you need to get
value from your data, but without the complexity and fragility of SQL
These “phases” are distinct and
easy to understand
They can be thought about in
order… no nesting.
26. #MDBLocal
Versatile: Rich query functionality
MongoDB
{ customer_id : 1,
first_name : "Mark",
last_name : "Smith",
city : "San Francisco",
phones: [ {
number : "1-212-777-1212",
type : "work"
},
{
number : "1-212-777-1213",
type : "cell"
}]
……...
Expressive
Queries
• Find anyone with phone # “1-212…”
• Check if the person with number “555…” is on the “do not call” list
Geospatial
• Find the best offer for the customer at geo coordinates of 42nd St.
and 6th Ave
Text Search • Find all tweets that mention the firm within the last 2 days
Aggregation
• Count and sort number of customers by city, compute min, max,
and average spend
Native Binary
JSON Support
• Add an additional phone number to Mark Smith’s record without
rewriting the document
• Update just 2 phone numbers out of 10
• Sort on the modified date
JOIN
($lookup)
• Query for all San Francisco residences, lookup their transactions,
and sum the amount by person
Graph Queries
($graphLookup)
• Query for all people within 3 degrees of separation from Mark
28. #MDBLocal
Design Queries to express meaningfulness of data
Since we are layering multiple dimensions, leverage queries to
express the concepts represented in the data.
Example:
Customer Information
Address Information
State in System
Metadata for reporting
Query – Represent customer
and address
Query – Get status
Query – Pull data for reporting
CRUD – Get All
30. #MDBLocal
Types of indexes
Compound Index
Single Field Index
Text Index
Geospatial Index (lat/lon pairs)
Unique Index (ensures uniqueness)
TTL Index (Time To Live - automatically delete
after elapsed time)
31. #MDBLocal
Fully Indexable
Fully featured secondary indexes
• Primary Index
– Every Collection has a primary key index
• Compound Index
– Index against multiple keys in the document
• MultiKey Index
– Index into arrays
• Wildcard Index
– Auto-index all matching fields, sub-documents &
arrays
• Text Indexes
– Support for text searches
• GeoSpatial Indexes
– 2d & 2dSphere indexes for spatial geometries
• Hashed Indexes
– Hashed based values for sharding
Index Types
• TTL Indexes
– Single Field indexes, when expired delete the document
• Unique Indexes
– Ensures value is not duplicated
• Partial Indexes
– Expression based indexes, allowing indexes on subsets of data
• Case Insensitive Indexes
• supports text search using case insensitive search
• Sparse Indexes
– Only index documents which have the given field
Index Features
32. #MDBLocal
{
"_id" : ObjectId("5c1d358bf383fbee028aea0b"),
"product_name" : "Blaster Gauntlet",
"product_attributes" : {
"elements" : [ "Fire" , "Water" ],
"price" : 250
...
}
},
{
"_id" : ObjectId("5c1d358bf383fbee028aea0c"),
"product_name" : "Super Suit",
"product_attributes" : {
"superFlight" : true,
"resistance" : [ "Bludgeoning", "Piercing", "Slashing" ]
...
},
}
Wildcard Indexes
Allow more natural data modeling, avoids
pre-defining indexes for every access
pattern
• Polymorphic document structures:
Product catalogs, CMS
• Ad-hoc queries & data exploration
Define a filter that indexes all matching fields,
sub-documents, and arrays
• Sparse index, omit specific fields
• Covered queries & collations
• Strongly consistent: updated
atomically with base data
Index all sub-documents &
arrays under Product Attributes
35. #MDBLocal
MongoDB Approach
Utility of data leveraged with through query and access pattern
Data
(BSON)
Query
( CRUD /
Aggregation
Pipeline )
Access
Pattern
( Indexing )
Utility
38. #MDBLocal
Easy: MongoDB Multi-Document ACID Transactions
Just like relational transactions
• Multi-statement, familiar relational syntax
• Easy to add to any application
• Multiple documents in 1 or many collections and databases
ACID guarantees
• Snapshot isolation, all or nothing execution
• No performance impact for non-transactional operations
39. #MDBLocal
Syntax
with client.start_session() as s:
s.start_transaction()
collection_one.insert_one(doc_one, session=s)
collection_two.insert_one(doc_two, session=s)
s.commit_transaction()
Natural for developers
• Idiomatic to the programming
language
• Familiar to relational
developers
• Simple
40. #MDBLocal
Master and Working Collections
Master: Customer
Order
Use
Case
Working: Order
Read
Read/Write
Master: Inventory
Read
• Master collections are stateless
• Working collections are stateful
• When initiate working document,
read as needed from Master
• Duplicate only for performance
• When Working state changes,
write back to Master
• Example: inventory consumed
• If duplicated data changes in Master,
write update to Working file
• Example: shipping address
• Use Transactions to keep it all straight
1
1
2
2
43. #MDBlocal
Practical Data Modeling
for MongoDB: Tutorial
https://www.surveymonkey.com/r/6GZ2GQR
Every session you rate enters you into a drawing for a $250
Visa gift card, sponsored by