1. Virtualizing MongoDB: Cloud,
EC2, GCE or Dedicated?
David Mytton
MongoDB World - July 2014
blog.serverdensity.com

2. David Mytton

3. Server monitoring, cloud management,
dashboards and alerting
serverdensity.com

4. Server Density Architecture

5. Server Density Architecture
● ~100 servers - Ubuntu 12.04

6. Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated

7. Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 30TB/m incoming data

8. Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 30TB/m incoming data
● Nginx, Python, MongoDB

9. Server Density Architecture
● ~100 servers - Ubuntu 12.04
● 50:50 virtual/dedicated
● 30TB/m incoming data
● Nginx, Python, MongoDB
● Softlayer > Google Cloud

10. Two choices for deployment

11. Two choices for deployment
● Virtualized
● Bare metal

12. Advantages of virtualization
● Easy to manage

13. Advantages of virtualization
● Easy to manage
● Fast boot

14. Advantages of virtualization
● Easy to manage
● Fast boot
● Easier to resize/migrate

15. Advantages of virtualization
● Easy to manage
● Fast boot
● Easier to resize/migrate
● Templating/snapshots

16. Advantages of virtualization
● Easy to manage
● Fast boot
● Easier to resize/migrate
● Templating/snapshots
● Containment

17. Disadvantages of virtualization
● Another layer

18. Disadvantages of virtualization
● Another layer
● Hypervisor overhead

19. Disadvantages of virtualization
● Another layer
● Hypervisor overhead
● Host contention

20. Disadvantages of virtualization
● Another layer
● Hypervisor overhead
● Host contention
● i/o performance

21. Advantages of bare metal
● Dedicated resources

22. Advantages of bare metal
● Dedicated resources
● Direct access to hardware

23. Advantages of bare metal
● Dedicated resources
● Direct access to hardware
● Customisable specs

24. Advantages of bare metal
● Dedicated resources
● Direct access to hardware
● Customisable specs
● Performance

25. Disadvantages of bare metal
● Build/deploy time

26. Disadvantages of bare metal
● Build/deploy time
● More difficult to resize

27. Disadvantages of bare metal
● Build/deploy time
● More difficult to resize
● Difficult to migrate/snapshot

28. Disadvantages of bare metal
● Build/deploy time
● More difficult to resize
● Capex/lifetime
● Difficult to migrate/snapshot

29. Why cloud?
● Flexible

30. Why cloud?
● Flexible
● Unlimited resources

31. Why cloud?
● Flexible
● Unlimited resources
● Cheap to get started

32. Why cloud?
● Flexible
● Unlimited resources
● Cheap to get started
● Other products

33. Why colo?

34. Why colo?
● Vastly cheaper
● Complete control

35. Performance problems?

36. Performance problems?
Easy answer: move to bare metal!

37. Optimising virtualization
● Dedicated CPUs

38. Optimising virtualization
● Dedicated CPUs
● High speed networking

39. Optimising virtualization
● Dedicated CPUs
● High speed networking
● Disk i/o (MB/s & IOPS)

40. Optimising EC2

41. Optimising EC2
● Newer, 64bit instance types

42. Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances

43. Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
● High memory (CPU not important)

44. Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
● High memory (CPU not important)
● Local storage ephemeral

45. Optimising EC2
● Newer, 64bit instance types
● Use network / EBS optimised instances
● High memory (CPU not important)
● Local storage ephemeral
● Volumes for log, journal and data

46. Optimising EC2
● PIOPS EBS volumes

47. Optimising EC2
● PIOPS EBS volumes
● RAID for redundancy & performance

48. Optimising EC2
● PIOPS EBS volumes
● RAID for redundancy & performance
● blockdev --setra 32

49. Common EC2 Problems

50. Common EC2 Problems
● Networking

51. Common EC2 Problems
● Networking
● Older hardware

52. Common EC2 Problems
● Networking
● Older hardware
● Complex pricing (PIOPS)

53. Why EC2?

54. Why EC2?
● Most popular

55. Why EC2?
● Most popular
● Range of instance types

56. Why EC2?
● Most popular
● Range of instance types
● Many regions

57. Optimising GCE
● High memory instances

58. Optimising GCE
● High memory instances
● Volume size = IOPS available

59. Optimising GCE
● High memory instances
● Volume size = IOPS available
● VMs have limits too

60. Optimising GCE
● High memory instances
● Volume size = IOPS available
● VMs have limits too
● Single volume or wasted space

61. Optimising GCE
● High memory instances
● Volume size = IOPS available
● VMs have limits too
● Single volume or wasted space
● Network / RAID already handled

62. Common GCE problems
● Volume too small

63. Common GCE problems
● Volume too small
● RAIDing unnecessary

64. Common GCE problems
● Volume too small
● RAIDing unnecessary
● Hitting limits?

65. Why GCE?
● Best network performance

66. Why GCE?
● Best network performance
● Easy to size disks / pricing

67. Why GCE?
● Best network performance
● Easy to size disks / pricing
● Global snapshots

68. Why GCE?
● Best network performance
● Easy to size disks / pricing
● Global snapshots
● Live migrate

69. Networking performance
AWS
GCE
bit.ly/googlevsamazon

70. Tweak: separate disks

71. Tweak: separate disks
● Avoids i/o contention

72. Tweak: separate disks
● Optimise IOPS/size/price for
journal, dbpath, logs
● Avoids i/o contention

73. Tweak: separate disks
● Optimise IOPS/size/price for
journal, dbpath, logs
● --directoryperdb - even
separate out each database
● Avoids i/o contention

74. --directoryperdb warning

75. --directoryperdb warning
● Snapshotting more complex

76. --directoryperdb warning
● Snapshotting more complex
● Requires fsync lock +
snapshot all volumes

77. Summary
● VMs for flexibility

78. Summary
● VMs for flexibility
● … but need tweaks

79. Summary
● VMs for flexibility
● Bare metal/colo for price/performance
● … but need tweaks

80. Summary
● VMs for flexibility
● Bare metal/colo for price/performance
● … but ops overhead
● … but need tweaks

81. どもありがとうございます
＠davidmytton
david@serverdensity.com
blog.serverdensity.com