Partner Webinar: MongoDB and Softlayer on Bare Metal: Stability, Performance, and Control
•Als PPTX, PDF herunterladen•
3 gefällt mir•1,368 views
There are no shortage of options when it comes to infrastructure and where someone deploys a MongoDB based application. We will discuss the advantages of deploying MongoDB on bare metal, presenting actual performance tests of MongoDB on SoftLayer’s on-demand dedicated server cloud infrastructure. We’ll address common concerns like the time investment of deploying complex MongoDB environments, and demonstrate the ease and control provided by the SoftLayer MongoDB Solution Designer.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (7)
Ähnlich wie Partner Webinar: MongoDB and Softlayer on Bare Metal: Stability, Performance, and Control
Ähnlich wie Partner Webinar: MongoDB and Softlayer on Bare Metal: Stability, Performance, and Control (20)
Mehr von MongoDB
Mehr von MongoDB (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Partner Webinar: MongoDB and Softlayer on Bare Metal: Stability, Performance, and Control
- 1. Performance, Stability & Control Harold Hannon Sr. Software Architect, SoftLayer
- 3. 13 data centers 16 network POPs 20Gb fiber interconnects Global Footprint
- 4. • Performance • Stability • Control On the agenda today
- 5. The 3 V’s
- 6. Test Cases • Small MongoDB Bare Metal Cloud vs Public Cloud Instance • Medium MongoDB Bare Metal Cloud vs Public Cloud Instance SSD and 15K SAS • Large MongoDB Bare Metal Cloud vs Public Cloud Instance SSD and 15K SAS
- 7. Private Network JMETER SERVER JMETER SERVER JMETER SERVER JMETER SERVER RMI Jmeter Master Client RDP Tester’s Local Machine Test Environment
- 8. Beware The “Best Case Test Case” 1 8 5 8 1 7 . 6 1 9 0 5 2 5 . 4 1 8 7 8 8 2 . 2 1 9 1 1 0 1 . 8 1 8 4 4 0 8 . 8 1 8 8 1 3 5 . 4 1 8 7 0 8 0 . 6 1 8 6 3 4 3 . 4 1 9 1 8 9 9 . 6 1 8 7 7 3 6 . 6 1 8 8 9 7 8 . 8 1 8 7 4 4 0 1 8 6 9 5 0 . 4 1 8 7 6 2 3 1 8 7 7 8 3 . 8 1 8 7 7 7 5 . 8 1 9 2 8 0 6 . 8 1 8 6 6 4 3 . 2
- 9. Do It Yourself • Data Set Sizing • Document/Object Sizes • Platform • Controlled client or AFAIC • Concurrency • Local or Remote Client • Read/Write Tests
- 10. JS Benchmarking Harness • Available on 10Gen website • Baseline performance tool • benchrun executable • Create javascript test case • Run from mongo command
- 11. db.foo.drop(); db.foo.insert( { _id : 1 } ) ops = [{op: "findOne", ns: "test.foo", query: {_id: 1}}, {op: "update", ns: "test.foo", query: {_id: 1}, update: {$inc: {x: 1}}}] for ( var x = 1; x <= 128; x *= 2) { res = benchRun( { parallel : x , seconds : 5 , ops : ops } ); print( "threads: " + x + "t queries/sec: " + res.query ); } Quick Example
- 12. host The hostname of the machine mongod is running on (defaults to localhost). username The username to use when authenticating to mongod (only use if running with auth). password The password to use when authenticating to mongod (only use if running with auth). db The database to authenticate to (only necessary if running with auth). ops A list of objects describing the operations to run (documented below). parallel The number of threads to run (defaults to single thread). seconds The amount of time to run the tests for (defaults to one second). Options
- 13. ns The namespace of the collection you are running the operation on, should be of the form "db.collection". op The type of operation can be "findOne", "insert", "update", "remove", "createIndex", "dropIndex" or "command". query The query object to use when querying or updating documents. update The update object (same as 2nd argument of update() function). doc The document to insert into the database (only for insert and remove). safe boolean specifying whether to use safe writes (only for update and insert). Options
- 14. { "#RAND_INT" : [ min , max , <multiplier> ] } [ 0 , 10 , 4 ] would produce random numbers between 0 and 10 and then multiply by 4. { "#RAND_STRING" : [ length ] } [ 3 ] would produce a string of 3 random characters. var complexDoc3 = { info: "#RAND_STRING": [30] } } var complexDoc3 = { info: { inner_field: { "#RAND_STRING": [30] } } } Dynamic Values
- 15. Lots of them here: https://github.com/mongodb/mongo/tree/master/jstests Example Scripts
- 16. Read Only Test • Random document size < 4k (mostly 1k) • 6GB Working Data Set Size • Random read only • 10 second per query set execution • Exponentially increasing concurrent clients from 1-128 • 48 Hour Test Run • RAID10 4 SSD drives • Local Client • “Pre-warmed cache”
- 17. The Results Concurrent Clients Avg Read OPS/Sec 1 38288.527 2 72103.35796 4 127451.8867 8 180798.4396 16 191817.3361 32 186429.4517 64 187011.7824 128 188187.0704
- 18. Performance
- 19. Small Test Small MongoDB Server Single 4-core Intel 1270 CPU 64-bit CentOS 8GB RAM 2 x 500GB SATAII – RAID1 1Gb Network Virtual Provider Instance 4 Virtual Compute Units 64-bit CentOS 7.5GB RAM 2 x 500GB Network Storage – RAID1 1Gb Network Tests Performed Small Data Set (8GB of .5mb documents) 200 iterations of 6:1 query-to-update operations Concurrent client connections exponentially increased from 1 to 32 Test duration spanned 48 hours
- 20. Small Test Read Operations per Second by Concurrent Client 0 200 400 600 800 1000 1200 1400 1600 1 Client 2 Clients 4 Clients 8 Clients 16 Clients 32 Clients Virtual Provider AVG SoftLayer AVG Virtual Provider Peak SoftLayer Peak
- 21. Write Operations per Second by Concurrent Client Small Test 0 200 400 600 800 1000 1200 1400 1600 1 Client 2 Clients 4 Clients 8 Clients 16 Clients 32 Clients Virtual Provider AVG SoftLayer AVG Virtual Provider Peak SoftLayer Peak
- 22. Medium Test Medium MongoDB Server Dual 6-core Intel 5670 CPUs 64-bit CentOS 36GB RAM 2 x 64GB SSD – RAID1 (Journal Mount) 4 x 300GB 15K SAS – RAID10 (Data Mount) 1Gb Network – Bonded Virtual Provider Instance 26 Virtual Compute Units 64-bit CentOS 30GB RAM 2 x 64GB Network Storage – RAID1 (Journal Mount) 4 x 300GB Network Storage – RAID10 (Data Mount) 1Gb Network Tests Performed Small Data Set (32GB of .5mb documents) 200 iterations of 6:1 query-to-update operations Concurrent client connections exponentially increased from 1 to 128 Test duration spanned 48 hours
- 23. Medium Test 15k SAS Read Operations per Second by Concurrent Client 0 1000 2000 3000 4000 5000 6000 7000 1 Client 2 Clients 4 Clients 8 Clients 16 Clients 32 Clients 64 Clients 128 Clients Virtual Provider AVG SoftLayer AVG Virtual Provider Peak SoftLayer Peak
- 24. Medium Test 15k SAS Write Operations per Second by Concurrent Client 0 1000 2000 3000 4000 5000 6000 7000 1 Client 2 Clients 4 Clients 8 Clients 16 Clients 32 Clients 64 Clients 128 Clients Virtual Provider AVG SoftLayer AVG Virtual Provider Peak SoftLayer Peak
- 25. Medium Test SSD Read Operations per Second by Concurrent Client 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 1 Client 2 Clients 4 Clients 8 Clients 16 Clients 32 Clients 64 Clients 128 Clients Virtual Provider AVG SoftLayer AVG Virtual Provider Peak SoftLayer Peak
- 26. Medium Test SSD Write Operations per Second by Concurrent Client 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 1 Client 2 Clients 4 Clients 8 Clients 16 Clients 32 Clients 64 Clients 128 Clients Virtual Provider AVG SoftLayer AVG Virtual Provider Peak SoftLayer Peak
- 27. Large Test Large MongoDB Server Dual 8-core Intel E5-2620 CPUs 64-bit CentOS 128GB RAM 2 x 64GB SSD – RAID1 (Journal Mount) 6 x 600GB 15K SAS – RAID10 (Data Mount) 1Gb Network – Bonded Virtual Provider Instance 26 Virtual Compute Units 64-bit CentOS 64GB RAM (Maximum available on this provider) 2 x 64GB Network Storage – RAID1 (Journal Mount) 6 x 600GB Network Storage – RAID10 (Data Mount) 1Gb Network Tests Performed Small Data Set (64GB of .5mb documents) 200 iterations of 6:1 query-to-update operations Concurrent client connections exponentially increased from 1 to 128 Test duration spanned 48 hours
- 28. Large Test 15k SAS Read Operations per Second by Concurrent Client 0 1000 2000 3000 4000 5000 6000 7000 1 Client 2 Clients 4 Clients 8 Clients 16 Clients 32 Clients 64 Clients 128 Clients Virtual Provider AVG SoftLayer AVG Virtual Provider Peak SoftLayer Peak
- 29. Large Test 15k SAS Write Operations per Second by Concurrent Client 0 1000 2000 3000 4000 5000 6000 7000 1 Client 2 Clients 4 Clients 8 Clients 16 Clients 32 Clients 64 Clients 128 Clients Virtual Provider AVG SoftLayer AVG Virtual Provider Peak SoftLayer Peak
- 30. Large Test SSD Read Operations per Second by Concurrent Client 0 1000 2000 3000 4000 5000 6000 1 Client 2 Clients 4 Clients 8 Clients 16 Clients 32 Clients 64 Clients 128 Clients Virtual Provider AVG SoftLayer AVG Virtual Provider Peak SoftLayer Peak
- 31. Large Test SSD Write Operations per Second by Concurrent Client 0 1000 2000 3000 4000 5000 6000 1 Client 2 Clients 4 Clients 8 Clients 16 Clients 32 Clients 64 Clients 128 Clients Virtual Provider AVG SoftLayer AVG Virtual Provider Peak SoftLayer Peak
- 32. Superior Performance 70% 133% 297% 111% 446% Small SATA II Medium 15k SAS Medium SSD Large 15k SAS Large SSD
- 33. Stability
- 34. Small Test Small Bare Metal Cloud Instance 64-bit CentOS 8GB RAM 2 x 500GB SATAII – RAID1 1Gb Network Public Cloud Instance 4 Virtual Compute Units 64-bit CentOS 7.5GB RAM 2 x 500GB Network Storage – RAID1 1Gb Network Tests Performed Data Set (8GB of .5mb documents) 200 iterations of 6:1 query-to-update operations Concurrent client connections exponentially increased from 1 to 32 Test duration spanned 48 hours
- 35. 122 193 201 271 480 835 0 200 400 600 800 1000 1200 1400 1 2 4 8 16 32 Ops/Second Concurrent Clients Small Public Cloud
- 36. 237 337 413 524 597 1112 0 200 400 600 800 1000 1200 1400 1600 1 2 4 8 16 32 Ops/Second Concurrent Clients Small Bare Metal
- 37. Medium Test Bare Metal Cloud Instance Dual 6-core Intel 5670 CPUs 64-bit CentOS 36GB RAM 2 x 64GB SSD – RAID1 (Journal Mount) 4 x 300GB 15K SAS – RAID10 (Data Mount) 4 x 400GB SSD– RAID10 (Data Mount) 1Gb Network – Bonded Public Cloud Instance 26 Virtual Compute Units 64-bit CentOS 30GB RAM 2 x 64GB Network Storage – RAID1 (Journal Mount) 4 x 300GB Network Storage – RAID10 (Data Mount) 4 x 400GB Network Storage – RAID10 (Data Mount) 1Gb Network Tests Performed Data Set (32GB of .5mb documents) 200 iterations of 6:1 query-to-update operations Concurrent client connections exponentially increased from 1 to 128 Test duration spanned 48 hours
- 38. 219 326 477 716 1298 1554 1483 1594 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 1 2 4 8 16 32 64 128 Ops/Second Concurrent Clients Medium Public Cloud
- 39. 542 818 1042 1260 1643 3392 4120 5443 0 1000 2000 3000 4000 5000 6000 7000 8000 1 2 4 8 16 32 64 128 Ops/Second Concurrent Clients 15k SAS Medium Bare Metal
- 40. 1389 2115 2637 2995 3047 3161 3742 3846 0 500 1000 1500 2000 2500 3000 3500 4000 4500 1 2 4 8 16 32 64 128 Ops/Second Concurrent Clients SSD Medium Bare Metal
- 41. Large Test Bare Metal Cloud Instance Dual 8-core Intel E5-2620 CPUs 64-bit CentOS 128GB RAM 2 x 64GB SSD – RAID1 (Journal Mount) 6 x 600GB 15K SAS – RAID10 (Data Mount) 6 x 400GB SSD – RAID10 (Data Mount) 1Gb Network – Bonded Public Cloud Instance 26 Virtual Compute Units 64-bit CentOS 64GB RAM (Maximum available on this provider) 2 x 64GB Network Storage – RAID1 (Journal Mount) 6 x 400GB Network Storage – RAID10 (Data Mount) 6 x 600GB Network Storage – RAID10 (Data Mount) 1Gb Network Tests Performed Data Set (64GB of .5mb documents) 200 iterations of 6:1 query-to-update operations Concurrent client connections exponentially increased from 1 to 128 Test duration spanned 48 hours
- 42. 105 409 943 636 1252 1733 1902 2044 0 1000 2000 3000 4000 5000 6000 1 2 4 8 16 32 64 128 Ops/Second Concurrent Clients Large Public Cloud
- 43. 412 686 946 1123 1373 2353 5097 5572 0 1000 2000 3000 4000 5000 6000 7000 1 2 4 8 16 32 64 128 Ops/Second Concurrent Clients 15k SAS Large Bare Metal
- 44. 1898 2919 3672 4351 3961 3629 3737 3864 0 1000 2000 3000 4000 5000 6000 1 2 4 8 16 32 64 128 Ops/Second Concurrent Clients SSD Large Bare Metal
- 45. Consistent Performance 36% 43% 93% 9% 8% 9% Small Medium Large Virtual Instance Bare Metal Instances
- 46. Control
- 47. The Dream
- 48. Virtual Instance Striped Network Attached Virtual Volumes The Reality
- 49. Cluster Virtual Instance Striped Network Attached Virtual Volumes Virtual Instance Striped Network Attached Virtual Volumes Virtual Instance Striped Network Attached Virtual Volumes Deployment Complexity
- 50. The Solution Designer Deployment Serenity
- 51. Big Data Solutions • Bare Metal Servers for Better Performance Your big data solution can now have the power of bare metal and the ease of the cloud. Our bare-metal servers are all available in real-time with no downtime via our portal or API, so you don’t have to sacrifice agility for performance. • MongoDB Best Practices The configurations have been optimized based on 10gen and SoftLayer’s insight, expertise and experience and provide an idea database environment. • Full Access and Control You get the total access and control of your complete MongoDB solution, including total utilization of your hardware, total say in where your servers and replica sets are deployed, and the ability to further customize it to any specification or requirement. • Easy Design and Provisioning Our configuration tool makes it easy to design and deploy complete customized MongoDB architectures.
- 52. “We have over two terabytes of raw event data coming in every day ... Struq has been able to process over 95 percent of requests in fewer than 30 milliseconds” - Aaron McKee CTO, Struq Customer Feedback
- 53. • Bare Metal Cloud can be leveraged to simplify deployments • Bare Metal has a significant performance superiority/consistency over Public Cloud • Public Cloud is best suited for Dev/POC or when running data sets in memory only Summary
- 54. www.softlayer.com blog @ http://sftlyr.com/bdperf More Information
Hinweis der Redaktion
- Softlayer?
- So here is our one and only obligatory analyst slide I promiseThink in terms of the 3 V’s Gartner definedThere are lots of 4th V’s (Value, Veracity, etc.) But really these apply to all data right? These 3 are at the coreAlso for our discussion today we are mostly going to be focused on Volume and Velocity (Variety is a given for us)These are important to consider when we start talking about how we want to deploy our solutionHow much and How fast is our data going to come at us?
- Numbers with no context are not very useful
- This is the ACTUAL test for that crazy number from before. Notice it has been heavily designed to produce a falsely high number. Not very useful.
- These were the results
- Numbers speak for themselvesYou take the overall average consistent performance + the consistencyCoupled with the ease of deployment
- The numbers are average read operations per second with writes occurring as well. The vertical white lines represent variance in that data. This slide and the other public cloud ones show that the variance in the data is HUGE. This means the platform is unstable under load and cannot give you a reliable predictable deployment
- Numbers speak for themselvesYou take the overall average consistent performance + the consistencyCoupled with the ease of deployment
- When we talk about a public cloud deployment everyone has this dream of just right clicking and “adding new” and everything is perfect
- Although at first things seem simple scaling on multi tenant (especially with NAS) gets trickyIn this case this is a SINGLE instance of a Mongo Node (This is one node, most deployments are going to have 3 or more of these)In order to achieve desired performance you have to raid network volumes and attach them to virtual instancesThis still doesn’t solve shared I/O deviation issues it just smears them so they may not spike as drastically
- It gets even crazier when you do highly available deploymentsStriped volumes (sometimes up to 10) attachedSo you can see that as you scale on a NAS Virtual environment You start to see when you look at this picture that your simple Virtualized environment has suddenly started to get very complexIf you are an engineer that believes in keeping things simple to avoid issues, this sort of thing keeps you up at nightBoth complexity and cost can start to spiral beyond what you may have anticipated
- The goal is to capture the ease of virtual deploymentConfigure complex cluster environmentsAllow for rapid deployment
- Highlight the 95% as further evidence of our extreme superiority in consistent performance.
- Thank you for your time, I hope you found this helpful QuestionsBlog