1. Frankfurt NoSQL User Group
Chris Harris
Email : charris@10gen.com
Twitter : cj_harris5
Wednesday, 22 February 12

2. As simple as possible,
but no simpler
Memcached
Key / Value
Scalability & Performance
RDBMS
Depth of functionality
Wednesday, 22 February 12

3. Key/Value Store
MongoDB
Relational Database
Wednesday, 22 February 12

4. Terminology
RDBMS MongoDB
Table Collection
Row(s) JSON Document
Index Index
Join Embedding & Linking
Partition Shard
Partition Key Shard Key
Wednesday, 22 February 12

5. Here is a “simple” SQL Model
mysql> select * from book;
+----+----------------------------------------------------------+
| id | title |
+----+----------------------------------------------------------+
| 1 | The Demon-Haunted World: Science as a Candle in the Dark |
| 2 | Cosmos |
| 3 | Programming in Scala |
+----+----------------------------------------------------------+
3 rows in set (0.00 sec)
mysql> select * from bookauthor;
+---------+-----------+
| book_id | author_id |
+---------+-----------+
| 1| 1|
| 2| 1|
| 3| 2|
| 3| 3|
| 3| 4|
+---------+-----------+
5 rows in set (0.00 sec)
mysql> select * from author;
+----+-----------+------------+-------------+-------------+---------------+
| id | last_name | first_name | middle_name | nationality | year_of_birth |
+----+-----------+------------+-------------+-------------+---------------+
| 1 | Sagan | Carl | Edward | NULL | 1934 |
| 2 | Odersky | Martin | NULL | DE | 1958 |
| 3 | Spoon | Lex | NULL | NULL | NULL |
| 4 | Venners | Bill | NULL | NULL | NULL |
+----+-----------+------------+-------------+-------------+---------------+
4 rows in set (0.00 sec)
Wednesday, 22 February 12

6. Flexible “Schemas”
{ “author”: “brendan”,
“text”: “...” }
{ “author”: “brendan”,
“text”: “...”,
“tags”: [“mongodb”,
“nosql”] }
Wednesday, 22 February 12

7. The Same Data in MongoDB
{
"_id" : ObjectId("4dfa6baa9c65dae09a4bbda5"),
"title" : "Programming in Scala",
"author" : [
{
"first_name" : "Martin",
"last_name" : "Odersky",
"nationality" : "DE",
"year_of_birth" : 1958
},
{
"first_name" : "Lex",
"last_name" : "Spoon"
},
{
"first_name" : "Bill",
"last_name" : "Venners"
}
]
}
Wednesday, 22 February 12

8. CRUD Operations
Wednesday, 22 February 12

9. db.test.insert({fn: “Chris”, ln: “Harris”})
Wednesday, 22 February 12

10. db.test.find({fn: “Chris”})
Wednesday, 22 February 12

11. Cursors
$gt, $lt, $gte, $lte, $ne, $all, $in, $nin, $or,
$not, $mod, $size, $exists, $type, $elemMatch
> var c = db.test.find({x: 20}).skip(20).limit(10)> c.next()
> c.next()
...
query
first N results + cursor id
getMore w/ cursor id
next N results + cursor id or 0
...
Wednesday, 22 February 12

12. Creating Indexes
An index on _id is automatic.
For more use ensureIndex:
db.blogs.ensureIndex({author: 1})
1 = ascending
-1 = descending
Wednesday, 22 February 12

13. Compound Indexes
db.blogs.save({
author: "James",
ts: new Date()
...
});
db.blogs.ensureIndex({author: 1, ts: -1})
Wednesday, 22 February 12

14. Unique Indexes
db.blogs.save({
author: "James",
title: "My first blog"
...
});
db.blogs.ensureIndex({title: 1}, {unique: true})
Wednesday, 22 February 12

15. Indexing Embedded Documents
db.blogs.save({
title: "My First blog",
stats : { views: 0,
followers: 0 }
});
db.blogs.ensureIndex({"stats.followers": -1})
db.blogs.find({"stats.followers": {$gt: 500}})
Wednesday, 22 February 12

16. Indexing Embedded Arrays
db.blogs.save({
title: "My First blog",
comments: [
{author: "James", ts : new Date()} ]
});
db.blogs.ensureIndex({"comments.author": 1})
db.blogs.find({"comments.author": "James"})
Wednesday, 22 February 12

17. Multikeys
db.blogs.save({
title: "My Second blog",
tags: ["mongodb", "NoSQL"]
});
db.blogs.ensureIndex({tags: 1})
db.blogs.find({tags: "NoSQL"})
Wednesday, 22 February 12

18. Covered Indexes
• From 1.8.0
• Query resolved in index only
• Need to exclude _id from items projected
db.blogs.save({
author: "James",
title: "My first blog"
});
db.blogs.ensureIndex({author: 1})
db.blogs.find({author: "James"},
{author: 1, _id:0}))
Wednesday, 22 February 12

19. Sparse Indexes
• From 1.8.0
• Key value included if and only if the value is present
• Reduces size of index
• Limited to a single field
db.blogs.ensureIndex({loc: 1}, {sparse: true})
// loc key stored for Ben & Sarah only
db.blogs.save({author: "Jim"})
db.blogs.save({author: "Ben", loc: null})
db.blogs.save({author: "Sarah", loc: "CA"})
Wednesday, 22 February 12

20. Geospatial
• Geo hash stored in B-Tree
• First two values indexed
db.blogs.save({
loc: { long: 40.739037, lat: 40.739037 }
});
db.blogs.save({
loc: [40.739037, 40.739037]
});
db.blogs.ensureIndex({"loc": "2d"})
Wednesday, 22 February 12

21. Replication
Wednesday, 22 February 12

22. Types of outage
• Planned
• Hardware upgrade
• O/S or file-system tuning
• Relocation of data to new file-system / storage
• Software upgrade
• Unplanned
• Hardware failure
• Data center failure
• Region outage
• Human error
• Application corruption
Wednesday, 22 February 12

23. Replica Set features
• A cluster of N servers
• Any (one) node can be primary
• Consensus election of primary
• Automatic failover
• Automatic recovery
• All writes to primary
• Reads can be to primary (default) or a secondary
Wednesday, 22 February 12

24. How MongoDB Replication works
Member
1 Member
3
Member
2
•Set is made up of 2 or more nodes
Wednesday, 22 February 12

25. How MongoDB Replication works
Member
1 Member
3
Member
2
PRIMARY
•Election establishes the PRIMARY
•Data replication from PRIMARY to SECONDARY
Wednesday, 22 February 12

26. How MongoDB Replication works
negotiate
new master
Member
1 Member
3
Member
2
DOWN
•PRIMARY may fail
•Automatic election of new PRIMARY if majority
exists
Wednesday, 22 February 12

27. How MongoDB Replication works
Member
1 Member
3
PRIMARY
Member
2
DOWN
•New PRIMARY elected
•Replication Set re-established
Wednesday, 22 February 12

28. How MongoDB Replication works
Member
1 Member
3
PRIMARY
Member 2
RECOVERING
•Automatic recovery
Wednesday, 22 February 12

29. How MongoDB Replication works
Member
1 Member
3
PRIMARY
Member
2
•Replication Set re-established
Wednesday, 22 February 12

30. Sharding
Wednesday, 22 February 12

31. http://community.qlikview.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/
theqlikviewblog/Cutting-Grass-with-Scissors-_2D00_-2.jpg
Wednesday, 22 February 12

32. http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg
Wednesday, 22 February 12

33. MongoDB Scaling - Single Node
read
node_a1
write
Wednesday, 22 February 12

34. Write scaling - add Shards
read
shard1 shard2
node_c1 node_c2
node_b1 node_b2
node_a1 node_a2
write
Wednesday, 22 February 12

35. Write scaling - add Shards
read
shard1 shard2 shard3
node_c1 node_c2 node_c3
node_b1 node_b2 node_b3
node_a1 node_a2 node_a3
write
Wednesday, 22 February 12

36. MongoDB Sharding
• Automatic partitioning and management
• Range based
• Convert to sharded system with no downtime
• Fully consistent
Wednesday, 22 February 12

37. How MongoDB Sharding works
> db.posts.save( {age:40} )
-∞ +∞
-∞ 40 41 +∞
•Data in inserted
•Ranges are split into more “chunks”
Wednesday, 22 February 12

38. How MongoDB Sharding works
> db.posts.save( {age:40} )
> db.posts.save( {age:50} )
-∞ +∞
-∞ 40 41 +∞
41 50 51 +∞
•More Data in inserted
•Ranges are split into more“chunks”
Wednesday, 22 February 12

39. How MongoDB Sharding works
> db.posts.save( {age:40} )
> db.posts.save( {age:50} )
> db.posts.save( {age:60} )
-∞ +∞
-∞ 40 41 +∞
41 50 51 +∞
51 60 61 +∞
Wednesday, 22 February 12

40. Balancing
mongos
config
balancer
config
Chunks!
config
1 2 3 4 13 14 15 16 25 26 27 28 37 38 39 40
5 6 7 8 17 18 19 20 29 30 31 32 41 42 43 44
9 10 11 12 21 22 23 24 33 34 35 36 45 46 47 48
Shard 1 Shard 2 Shard 3 Shard 4
Wednesday, 22 February 12

41. Balancing
mongos
config
balancer
config
Imbalance
Imbalance config
1 2 3 4
5 6 7 8
9 10 11 12 21 22 23 24 33 34 35 36 45 46 47 48
Shard 1 Shard 2 Shard 3 Shard 4
Wednesday, 22 February 12

42. Balancing
mongos
config
balancer
config
Move chunk 1 to config
Shard 2
1 2 3 4
5 6 7 8
9 10 11 12 21 22 23 24 33 34 35 36 45 46 47 48
Shard 1 Shard 2 Shard 3 Shard 4
Wednesday, 22 February 12

43. Balancing
mongos
config
balancer
config
config
1 2 3 4
5 6 7 8
9 10 11 12 21 22 23 24 33 34 35 36 45 46 47 48
Shard 1 Shard 2 Shard 3 Shard 4
Wednesday, 22 February 12

44. Balancing
mongos
config
balancer
config
config
2 3 4
5 6 7 8 1
9 10 11 12 21 22 23 24 33 34 35 36 45 46 47 48
Shard 1 Shard 2 Shard 3 Shard 4
Wednesday, 22 February 12

45. Balancing
mongos
config
balancer
config
Chunk 1 now lives on
Shard 2
config
2 3 4
5 6 7 8 1
9 10 11 12 21 22 23 24 33 34 35 36 45 46 47 48
Shard 1 Shard 2 Shard 3 Shard 4
Wednesday, 22 February 12