This presentation covers two important topics for scalable mongodb deployments. First, we will discus how MongoDB stores data so you can figure out how much RAM you need in each of your servers. Second, we'll look at replica sets and how to design a highly available topology for your replica set.

1. MongoDB deployment tipsJared Rosoff - @forjared

2. Agenda Sizing Machines Understanding working set Sizing RAM Sizing Disk Configuring Replica Sets Understanding failover Avoiding single points of failure Minimizing recovery time

3. Overview Secondary Secondary Secondary Config Config Config Router Router Shard 1 Shard 2 Shard 3 Primary Primary Primary Secondary Secondary Secondary

4. Servers and Hardware Secondary Secondary Secondary Config Config Config Router Router Shard 1 Shard 2 Shard 3 Primary Primary Primary Secondary Secondary Secondary

5. Sizing ram and disk

6. Collection 1 Index 1

7. Collection 1 Virtual Address Space 1 Index 1

8. Collection 1 Virtual Address Space 1 This is your virtual memory size (mapped) Index 1

9. Collection 1 Virtual Address Space 1 Physical RAM Index 1

10. Collection 1 Virtual Address Space 1 Physical RAM Index 1 This is your resident memory size

11. Disk Collection 1 Virtual Address Space 1 Physical RAM Index 1

12. Disk Collection 1 Virtual Address Space 1 Physical RAM Index 1 Virtual Address Space 2

13. Disk Collection 1 Virtual Address Space 1 Physical RAM Index 1 100 ns = 10,000 ns =

14. Disk configurations Single Disk ~200 seeks / second

15. Disk configurations Single Disk RAID 0 ~200 seeks / second ~200 seeks / second ~200 seeks / second ~200 seeks / second

16. Disk configurations Single Disk RAID 0 ~200 seeks / second RAID 10 ~200 seeks / second ~200 seeks / second ~200 seeks / second ~400 seeks / second ~400 seeks / second ~400 seeks / second

17. SSDs ?? Seek time of 0.1ms vs 5ms (200 seeks / sec => 10000 seeks / sec) But expensive

18. Tips for sizing hardware Know how important page faults are If you want low latency, avoid page faults Size memory appropriately To avoid page faults, fit everything in RAM Collection Data + Index Data Provision disk appropriately RAID10 is recommended SSD’s are fast, if you can afford them

19. Replica Sets Secondary Secondary Secondary Config Config Config Router Router Shard 1 Shard 2 Shard 3 Primary Primary Primary Secondary Secondary Secondary

20. Understanding automatic failover

21. Primary Election As long as a partition can see a majority (>50%) of the cluster, then it will elect a primary. Secondary Primary Secondary

22. Simple Failure 66% of cluster visible. Primary is elected Primary Failed Node Secondary

23. Simple Failure 33% of cluster visible. Read only mode. Failed Node Failed Node Secondary

24. Network Partition Secondary Primary Secondary

25. Network Partition 66% of cluster visible. Primary is elected Secondary Failed Node Primary Primary Secondary Secondary

26. Secondary Network Partition Primary Failed Node Secondary Secondary Failed Node 33% of cluster visible. Read only mode.

27. Even Cluster Size Secondary Primary Secondary Secondary

28. Even Cluster Size 50% of cluster visible. Read only mode. Secondary Failed Node Primary Secondary Failed Node Secondary Secondary Secondary

29. Even Cluster Size Secondary Failed Node Primary Secondary Failed Node Secondary Secondary Secondary 50% of cluster visible. Read only mode.

30. Avoiding single points of failure

31. Avoid Single points of failure

32. Avoid Single points of failure Secondary Primary Secondary Top of rack switch Rack falls over

33. Better Secondary Primary Secondary Loss of internet Building burns down

34. Better yet San Francisco Secondary Primary Dallas Secondary

35. Priorities San Francisco Priority 1 Priority 1 Secondary Priority 0 Primary Dallas Secondary Disaster recover data center. Will never become primary automatically.

36. Even Better San Francisco New York Secondary Primary Dallas Secondary

37. Fast recovery

38. 2 Replicas + Arbiter?? Arbiter Is this a good idea? Primary Secondary

39. 2 Replicas + Arbiter?? 1 Arbiter Primary Secondary

40. 2 Replicas + Arbiter?? 1 2 Arbiter Arbiter Primary Primary Secondary Secondary

41. 2 Replicas + Arbiter?? 3 1 2 Full Sync Arbiter Arbiter Arbiter Primary Primary Primary Secondary Secondary Secondary Secondary Uh oh. Full Sync is going to use a lot of resources on the primary. So I may have downtime or degraded performance

42. With 3 replicas 1 Primary Secondary Secondary

43. With 3 replicas 1 2 Primary Primary Secondary Secondary Secondary Secondary

44. With 3 replicas 3 1 2 Primary Primary Primary Full Sync Secondary Secondary Secondary Secondary Secondary Secondary Secondary Sync can happen from secondary, which will not impact traffic on Primary.

45. Tips for choosing replica set topology Avoid single points of failure Separate racks Separate data centers Avoid long recovery downtime Use journaling Use 3+ replicas Keep your actives close Use priority to control where failovers happen

46. Summary Sizing a machine Know your working set size Size RAM appropriately Provision sufficient disks Designing a replica set Know how failover happens Design for failure Design for fast recover