Into the cloud
•Als PPTX, PDF herunterladen•
0 gefällt mir•545 views
Presentation about how and why we moved our Continuous Delivery implementation from onpremis datacenter to the AWS Cloud
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (10)
Ă„hnlich wie Into the cloud
Ă„hnlich wie Into the cloud (20)
KĂĽrzlich hochgeladen
KĂĽrzlich hochgeladen (20)
Into the cloud
- 1. Into the Cloud Making our Continuous Delivery fly.
- 2. Tomas Riha Architect @ VGT/WirelessCar Passionate about creativity, change and improvement Horrible at following instructions and performing repetitive tasks MAJOR Project Liability mail: tomas.riha@wirelesscar.com twitter: @TomasRihaSE blog: continuous-delivery-and-more.blogspot.com
- 3. Four Years ago New Telematics Delivery Platform Sent us on road to Micro Services Continuous Delivery
- 4. Infrastructure at that time Low level of virtualization “Two servers and a Oracle DB” per delivery Split Operations and Development Infrastructure and Application Architecture separated Long process to get a server
- 6. Reality
- 7. Continuous Delivery Build Release Deploy Test Build Release Deploy Test Build Release Deploy Test Build Release Deploy Test Build Release Deploy Test Assemble Release Deploy Test Service Pipes 100s of tests run in parallel testing Solution Pipes 10s Use Case Requirements Verified
- 8. Here is your test server Build Release Deploy Test DB Test Server
- 9. Static virtual instances Master Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Custom DeliveryEngine GUI Master Mind Gerrit
- 10. Capacity Planning & Peaks Master Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker ?
- 11. Pampering the Snowflakes Each static server needs maintenance. Server Maintenance one of the top task the team does Each server provisioned by different version of our chef scripts, where baseline is a moving target
- 12. Continuous Delivery as a Service Ok so this Continuous Delivery thing is nice.... ....what does it take to deliver it to the entire organization as a Service? 400+ developers
- 13. Continuous Delivery as a Service Blue skies, Sunshine and ohh yes a CLOUD PROVIDER!! Amazon Web Services
- 14. Load Tests We have used AWS a bit for Load Tests
- 15. Go! Continuous Delivery in the AWS Cloud Demo Environments in the AWS Cloud
- 16. Build in Cloud host on premises and at partner AWS Volvo Hosting Partner Hosting Partner Compatibility in Environments!
- 17. What we got Automated provisioning on our Continuous Delivery infrastructure using chef Automated provisioning of our Jenkins jobs into build pipes Rudimentary Image Baking Capabilities from our AWS Load Tests Basic understanding of how our services behave in a cloud environment
- 18. What we need to do Build a Virtual Private Cloud with networks Connect VPC to Volvo Network Migrate about ~800 Jenkins jobs 60 Jenkins master/slaves 1 Nexus server, 2 test databases, 1 Sonar, 1 Elk 2 custom applications, 1 MongoDB
- 19. Our approach Change as little as possible during migration Automate everything Establish responsibility between Operations and Development
- 21. Defining Application Topology Load Balancer Server Instance Server Instance Data Storage In Amazon Web Services â—Ź Elastic Load Balancer â—‹ Security Groups â—Ź Auto Scaling Group â—‹ Launch Configuration â—‹ Amazon Machine Image â—‹ Security Groups â—Ź Data Storages â—‹ Simple Storage Service â—‹ Relational Data Service â– Oracle â– Postgres â—‹ Custom Databases â– MongoDB â—‹ Security Groups
- 22. Defining Application Environment Load Balancer Server Instance Server Instance Data Storage In Amazon Web Services â—Ź Virtual Private Cloud â—‹ Subnets â—Ź Auto Scaling Group â—‹ Scaling Rules â– Up/Down â– Max/min â—‹ Instance Size â– Mem/Cpu â—Ź Data Storages â—‹ Size â– Mem/Cpu/Storage
- 23. Infrastructure as Code Dev Test Load Test UAT INT Test Provisioning Interface Vagrant Impl Cloud Provider Impl PDF Document Impl Topology Spec Env Spec Legacy PROD Environment Definition Networking Loadbalancing Servers Databases Location Integration Scaling
- 24. Infrastructure as Code Test Load Test Provisioning Interface AWS Cloudformation Impl Topology Spec Env Spec Demo Delivery Engine Tool
- 25. Baking Machine Images Git Repo Base Packer Scripts Bakery Instance Git Repo Delta Packer Scripts AMI Alpha RHEL AMI Bakery Instance AMI Delta
- 26. Baking Machine Images RHEL AMI VGT Base Java Base Jenkins Base Nexus Graphite ELK Jenkins Master Jenkins SlaveApplications Applications Applications
- 27. Chef Issues Never share same scripts for baking image that go from known state to a delta with scripts that manage snowflakes!
- 28. Phase 1 Politics, Cross Organizational handovers & PoC Way too much time Phase 2 Build & Automate Setup Build & Tear Up the VPC a few 100 times 1.5 months ~ 2 engineers Phase 3 Migration Start moving over the repositories. Add .delivery-engine.yml files to repositories Disable build pipes in old environment 1 month ~ 3 engineers The Migration
- 29. Daily Cost tracking during migration!! Did we do something really stupid yesterday? Bosses love you when you report cost and think about optimizing cost Cost Tracking
- 31. Since the migration started we have not launched one single instance through the AWS Console. Console only used to terminate instances, never to create instances. Done well
- 32. Delivery Engine after Migration More or less the same setup Different distribution of jobs Auto Scaling Workers Orchestration Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Orchestration Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Custom DeliveryEngine GUI Master Mind Gerrit
- 33. Immediate Benefits Maintenance down at least 80% at least due to worker recycling No more capacity planning Orchestration Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Orchestration Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Custom DeliveryEngine GUI Master Mind Gerrit
- 34. Not High Availability Setup is still stateful Orchestration Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Orchestration Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Worker Custom DeliveryEngine GUI Master Mind Gerrit
- 35. 250 Git repositories building, testing, releasing Components, Applications, Solutions For closer to 150 developers Usage
- 36. Growth during migration wasn't expected but was about 50 repos Over 100 Git repos added after migration New teams constantly onboarding to Delivery Engine Expecting 200 plus developers by end of year Growth
- 37. Grow number of users and content not the size of our Team Size! We do it through a Event Driven, Stateless, Application developed using ALL the Continuous Delivery principles. Using Continuous Delivery to Deliver Continuous Delivery. We need to ensure scalability and availability
- 38. Stateless Build Environment Pipes as Code Unit test pipe task Integration test pipes Job Pipe as Code Client
- 39. Stateless Build Environment Build Data and reports away from Build Environment Job Pipe as Code Client
- 40. Stateless Build Environment No Dependency between GIT Repository and Build Job Only one job exists Job Pipe as Code Client Queue Gerrit Listener
- 41. Job Stateless Build Environment Aws Tools Pipe as Code Client Test Environment Queue Gerrit Listener Split Test Environment from Build Environment
- 42. HA Build environment Test Environment Rabbit MQ Cluster Coordinator Jenkins Factory Foreman Jenkins Workers Jenkins SlavesWorkers Jenkins SlavesWorkers Jenkins Slaves Gerrit Factory Foreman Jenkins Workers Jenkins SlavesWorkers Jenkins SlavesWorkers Jenkins Slaves Test Environment Test Environment Test Environment ELK Graphite S3 Master Mind Custom DeliveryEngine GUI Slack
- 43. From idea to rolled out in about 2 months, 1 man month of work Without AWS not possible HA Rewrite
- 44. + Lead time for Infrastructural change drastically reduced Ability to build HA and scalability on a level previously not possible No more capacity planning Demo Environment using same tooling and fully up and running QA and Integration testing environments coming up as well DevOps Amazon Web Services works GREAT! - Different tooling for creating aws infrastructure to creating in house infrastructure Retrospective
- 45. Questions? here: @TheBar mail: tomas.riha@wirelesscar.com twitter: @TomasRihaSE blog: continuous-delivery-and-more.blogspot.com Thats it!