Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Copyright Valiton GmbH 1
Moving from AWS to
Atlas
Our Motivation and Journey from
Self-Hosted to Managed MongoDB
26th
June...
Copyright Valiton GmbH 2
Personal Background
2011
2009
Web Developer at food IT Consulting
2013
Co-Founder of allergenis U...
Copyright Valiton GmbH 3
Valiton as a Subsidiary of Hubert Burda Media
Copyright Valiton GmbH 4
Project Background
...
Decreasing
Ad Revenue
New Markets
New
Business Models
Newsletter E-Commerc...
Copyright Valiton GmbH 5
New Business Models
Newsletter
Web-Tracking
User-Properties
E-Commerce
Audience Groups
Exclusive ...
Copyright Valiton GmbH 6
System Overview
Privacy Agreement, Contact Agreement and Deletion Process
userdata
LeadGen
raffle...
Copyright Valiton GmbH 7
AWS MongoDB Architecture (1)
read write
Cluster B
Secondary Secondary
Amazon EC2 Amazon EC2
Amazo...
Copyright Valiton GmbH 8
AWS MongoDB Architecture (2)
Elastic Load
Balancer
Cluster
Secondary Secondary
Amazon EC2 Amazon ...
Copyright Valiton GmbH 9
Challenges with Self-Hosted Architecture
Security Monitoring Maintenance
Copyright Valiton GmbH 10
Challenges with Self-Hosted Architecture
Security Monitoring Maintenance
Copyright Valiton GmbH 11
Security – Authentication
Elastic Load
Balancer
Cluster
Secondary Secondary
Amazon EC2 Amazon EC...
Copyright Valiton GmbH 12
Security – Jump Host
EBS-Volume
EBS-Volume
Elastic Load
Balancer
Cluster
Secondary Secondary
Ama...
Copyright Valiton GmbH 13
Challenges with Self-Hosted Architecture
Security Monitoring Maintenance
Copyright Valiton GmbH 14
Monitoring CPUonly Memory Disk
no Process Database
Copyright Valiton GmbH 15
Challenges with Self-Hosted Architecture
Security Monitoring Maintenance
Copyright Valiton GmbH 16
Maintenance – Mongo Upgrade
UPGRADE
High Effort
3.0
Copyright Valiton GmbH 17
Maintenance - Scaling
Seasonal traffic
Overprovisioning
Copyright Valiton GmbH 18
Maintenance – Agile Development
DevOps
Scrum-Master
Developers
Fix MongoDB
Problem
Try to fulfil...
Copyright Valiton GmbH 19
Solution: Moving to MongoDB Atlas
Contracts
Migration Preparation
Migration
Advantages:
• Focus ...
Copyright Valiton GmbH 20
Migration Preparation – MongoDB Upgrade
3.0
3.2
UPGRADE
UPGRADE
3.2
UPGRADE
3.4
3.6
Followed Mon...
Copyright Valiton GmbH 21
Migration Preparation – Add Authentication
Elastic Load
Balancer
Primary
Cluster
Secondary Secon...
Copyright Valiton GmbH 22
Migration – MongoDB Atlas Live Migration
Elastic Load
Balancer
Cluster
Secondary Secondary
Amazo...
Copyright Valiton GmbH 23
Performance Boost via Performance Advisor
The number of database objects
requested for a query h...
Copyright Valiton GmbH 24
Look back: Challenges with Self-Hosted Architecture
Security Monitoring Maintenance
Authenticati...
Copyright Valiton GmbH 25
How to merge AWS and Atlas logs?
Problem: No Grafana integration yet available
Copyright Valiton GmbH 26
Solution: Develop and Open-Source Atlas Grafana
Plugin
Get the plugin:
https://github.com/valito...
Copyright Valiton GmbH 27
Copyright Valiton GmbH 28
Copyright Valiton GmbH 29
What‘s next?
● Participation at MongoDB University
● Use Functions added in MongoDB > 3.0
● Data...
Copyright Valiton GmbH 30
Q & A
Security Monitoring Maintenance
Authentication
IP Whitelisting
In parallel AWS and Atlas V...
Nächste SlideShare
Wird geladen in …5
×

Moving from AWS to Atlas - Our Motivation and Journey from Self-Hosted to Managed MongoDB

37 Aufrufe

Veröffentlicht am

Valiton / Hubert Burda Media shares insights about the challenges they faced with self-hosted MongoDB clusters on AWS and how they solved it with the migration to MongoDB Atlas

Veröffentlicht in: Software
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Moving from AWS to Atlas - Our Motivation and Journey from Self-Hosted to Managed MongoDB

  1. 1. Copyright Valiton GmbH 1 Moving from AWS to Atlas Our Motivation and Journey from Self-Hosted to Managed MongoDB 26th June 2019
  2. 2. Copyright Valiton GmbH 2 Personal Background 2011 2009 Web Developer at food IT Consulting 2013 Co-Founder of allergenis UG 2016 now Freelancer Software-Engineer at Valiton GmbH Christoph Caprano Christoph.Caprano@valiton.com
  3. 3. Copyright Valiton GmbH 3 Valiton as a Subsidiary of Hubert Burda Media
  4. 4. Copyright Valiton GmbH 4 Project Background ... Decreasing Ad Revenue New Markets New Business Models Newsletter E-CommerceTracking
  5. 5. Copyright Valiton GmbH 5 New Business Models Newsletter Web-Tracking User-Properties E-Commerce Audience Groups Exclusive Shooting Workshop … Exclusive wine offers
  6. 6. Copyright Valiton GmbH 6 System Overview Privacy Agreement, Contact Agreement and Deletion Process userdata LeadGen raffle participations
  7. 7. Copyright Valiton GmbH 7 AWS MongoDB Architecture (1) read write Cluster B Secondary Secondary Amazon EC2 Amazon EC2 Amazon EC2 read write Cluster A Secondary Secondary Amazon EC2 Amazon EC2 Amazon EC2 Replica-Set Replica-Set PrimaryPrimary
  8. 8. Copyright Valiton GmbH 8 AWS MongoDB Architecture (2) Elastic Load Balancer Cluster Secondary Secondary Amazon EC2 Amazon EC2 Amazon EC2 EBS-Volume EBS-VolumeEBS-Volume Primary
  9. 9. Copyright Valiton GmbH 9 Challenges with Self-Hosted Architecture Security Monitoring Maintenance
  10. 10. Copyright Valiton GmbH 10 Challenges with Self-Hosted Architecture Security Monitoring Maintenance
  11. 11. Copyright Valiton GmbH 11 Security – Authentication Elastic Load Balancer Cluster Secondary Secondary Amazon EC2 Amazon EC2 Amazon EC2 EBS-Volume EBS-VolumeEBS-Volume Primary
  12. 12. Copyright Valiton GmbH 12 Security – Jump Host EBS-Volume EBS-Volume Elastic Load Balancer Cluster Secondary Secondary Amazon EC2 Amazon EC2 Amazon EC2 EBS-Volume Jump Host VPN Amazon EC2 Primary
  13. 13. Copyright Valiton GmbH 13 Challenges with Self-Hosted Architecture Security Monitoring Maintenance
  14. 14. Copyright Valiton GmbH 14 Monitoring CPUonly Memory Disk no Process Database
  15. 15. Copyright Valiton GmbH 15 Challenges with Self-Hosted Architecture Security Monitoring Maintenance
  16. 16. Copyright Valiton GmbH 16 Maintenance – Mongo Upgrade UPGRADE High Effort 3.0
  17. 17. Copyright Valiton GmbH 17 Maintenance - Scaling Seasonal traffic Overprovisioning
  18. 18. Copyright Valiton GmbH 18 Maintenance – Agile Development DevOps Scrum-Master Developers Fix MongoDB Problem Try to fulfil sprint goal
  19. 19. Copyright Valiton GmbH 19 Solution: Moving to MongoDB Atlas Contracts Migration Preparation Migration Advantages: • Focus on Software Development • Possibility of Cloud-Agnostic Required Steps:
  20. 20. Copyright Valiton GmbH 20 Migration Preparation – MongoDB Upgrade 3.0 3.2 UPGRADE UPGRADE 3.2 UPGRADE 3.4 3.6 Followed MongoDB Migration Guide
  21. 21. Copyright Valiton GmbH 21 Migration Preparation – Add Authentication Elastic Load Balancer Primary Cluster Secondary Secondary Amazon EC2 Amazon EC2 Amazon EC2 EBS-Volume EBS-VolumeEBS-Volume
  22. 22. Copyright Valiton GmbH 22 Migration – MongoDB Atlas Live Migration Elastic Load Balancer Cluster Secondary Secondary Amazon EC2 Amazon EC2 Amazon EC2 EBS-Volume EBS-VolumeEBS-Volume x Live Migration Atlas Cluster Primary
  23. 23. Copyright Valiton GmbH 23 Performance Boost via Performance Advisor The number of database objects requested for a query has been drastically reduced. This has caused the API to respond twice as fast on average (~15 ms instead of ~30 ms). This reduces the average query execution time from up to 7 ms to significantly less than 1 ms. Query Targeting Query Executor AVG API Response Time
  24. 24. Copyright Valiton GmbH 24 Look back: Challenges with Self-Hosted Architecture Security Monitoring Maintenance Authentication IP Whitelisting In parallel AWS and Atlas Version Upgrade: SaaS Scaling: SaaS
  25. 25. Copyright Valiton GmbH 25 How to merge AWS and Atlas logs? Problem: No Grafana integration yet available
  26. 26. Copyright Valiton GmbH 26 Solution: Develop and Open-Source Atlas Grafana Plugin Get the plugin: https://github.com/valiton/grafana-mongodb-atlas-datasource
  27. 27. Copyright Valiton GmbH 27
  28. 28. Copyright Valiton GmbH 28
  29. 29. Copyright Valiton GmbH 29 What‘s next? ● Participation at MongoDB University ● Use Functions added in MongoDB > 3.0 ● Data and Service Redesign https://www.mongodb.com/blog/post/building-with-patterns-a-summary ● Implementation of Aggregation Pipelines
  30. 30. Copyright Valiton GmbH 30 Q & A Security Monitoring Maintenance Authentication IP Whitelisting In parallel AWS and Atlas Version Upgrade: SaaS Scaling: SaaS

×