DevOps für mittlere Unternehmen und Großunternehmen - AWS Cloud Web Day für Mittelstand und Großunternehmen
•Als PPTX, PDF herunterladen•
0 gefällt mir•575 views
Vortrag "DevOps für mittlere Unternehmen und Großunternehmen" von Oliver Arafat beim AWS Cloud Web Day für Mittelstand und Großunternehmen. Alle Videos und Präsentationen finden Sie hier: http://amzn.to/1VUJZsT
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (18)
Ähnlich wie DevOps für mittlere Unternehmen und Großunternehmen - AWS Cloud Web Day für Mittelstand und Großunternehmen
Ähnlich wie DevOps für mittlere Unternehmen und Großunternehmen - AWS Cloud Web Day für Mittelstand und Großunternehmen (20)
Mehr von AWS Germany
Mehr von AWS Germany (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
DevOps für mittlere Unternehmen und Großunternehmen - AWS Cloud Web Day für Mittelstand und Großunternehmen
- 1. ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved DevOps in the Enterprise Oliver Arafat Technical Evangelist Amazon Web Services
- 2. Agenda • The Amazon DevOps story • Building Blocks for a Delivery Pipeline on AWS • Three Tenets for DevOps in the Enterpris
- 3. Prelude
- 4. What is DevOps? DevOps = efficiencies that speed up this lifecycle developers customers releasetestbuild plan monitor delivery pipeline feedback loop Software development lifecycle
- 5. ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved The Amazon DevOps story ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
- 8. • Service-Oriented Architecture (SOA) • Single-purpose • Connect only through APIs • “Microservices”
- 9. • Two-pizza teams • Full ownership • Full accountability • Aligned incentives • “DevOps”
- 10. Missing tools developers delivery pipelineservices ???
- 11. • Self-service • Technology-agnostic • Encourage best practices • Single-purpose services
- 12. • Deployment service • No downtime deployments • Health checking • Versioned artifacts and rollbacks
- 13. • Continuous delivery • From check-in to production • Automated actions and transitions • >90% of teams Pipelines
- 14. Microservice development lifecycle developers delivery pipelinesservices releasetestbuild releasetestbuild releasetestbuild releasetestbuild releasetestbuild releasetestbuild
- 15. = 50 million deployments a year Thousands of teams × Microservice architecture × Continuous delivery × Multiple environments
- 16. How can others do this?
- 17. ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Building Blocks for a Delivery Pipeline on AWS ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
- 18. Setting up a delivery pipeline Testing Staging Production deploy deploy deploy Source Build release AWS CodeDeploy AWS CodePipeline AWS CodeCommit Automate your software deployments Automate your release process
- 19. AWS CodeDeploy • Easy and reliable deployments • Scale with ease • Deploy to any server Test CodeDeployv1, v2, v3 Production Dev application revisions deployment groups
- 20. AWS CodePipeline • Connect to best-of-breed tools • Accelerate your release process • Consistently verify each release Build 1) Build 2) Unit test 1) Deploy 2) UI test Source Beta Production 1) Deploy 2) Perf test Gamma 1) Deploy canary 2) Deploy region 1 3) Deploy region 2 1) Pull
- 21. AWS CodeCommit • Use standard Git tools • Scalability, availability, and durability of Amazon S3 • Encryption at rest with customer-specific keys git pull/push CodeCommit Git objects in Amazon S3 Git index in Amazon DynamoDB Encryption key in AWS KMS SSH or HTTPS
- 23. ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Three Tenets for DevOps in the Enterprise ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
- 25. Be customer service-oriented throughout your organization. Customer-service-centricity improves the IT brand Customer-service-centricity is good for your career
- 26. Automate everything. • Efficiency • Elasticity • Portability • Security • Auditability • Recoverability “As machines become more and more efficient and perfect, so it will become clear that imperfection is the greatness of man.” — Ernst Fischer
- 27. Run what you build. • Design for Production • Greater employee autonomy • Greater transparency • More automation • Better operational quality • More satisfied customers
- 28. Book recommendation Gene Kim, Kevin Behr, and George Spafford The Phoenix Project Taschenbuch: 382 Seiten Verlag: Astronaut Projects; Auflage: Revised. (16. Oktober 2014) Sprache: Englisch
- 29. Thank you! Oliver Arafat Technical Evangelist Amazon Web Services arafato@amazon.de @OliverArafat
Hinweis der Redaktion
- - welcome and introduction - today I'm going to talk about DevOps at Amazon - it's a bit unusual for Amazon to be revealing so much about internal tools and processes, but for this case case, we're making an exception As software teams transition to cloud-based architectures and adopt more agile processes, the tools they need to support their development cycles will change. In this session, we'll take you through the transition that Amazon made to a service-oriented architecture over a decade ago. We will share the lessons we learned, the processes we adopted, and the tools we built to increase both our agility and reliability. We will also introduce you to AWS CodeCommit, AWS CodePipeline, and AWS CodeDeploy, three new services born out of Amazon's internal DevOps experience.
- - I'm going to break the talk into 2 parts - first, I'll start with the backstory about how Amazon got into DevOps, and the changes that we made to get proficient at building new services - after covering our history, I'm going to talk about the present - I'll introduce new AWS services that give you the same type of tools that we use at Amazon to set up your own DevOps process Agenda In this talk, I'm going to walk you through some the changes that Amazon has made over the years, and tell you about the developer tools we built to support those architectural and organizational changes. - the Amazon DevOps Story - CodeDeploy - CodePipeline - CodeCommit
- - and the reason leads back to our customers - at AWS, we're regularly talking with customers to learn about their needs and shape our products - and a few years ago, we started to notice a trend in those conversations - in addition to, or sometimes instead of, asking us about EC2, DynamoDB, and our other services, they asked about our internal development processes, org structures, and tooling - they wanted to know how we built new services and delivered new features so quickly - after hearing this again, and again, and again, we knew there was a big opportunity here - that's what I want to address in the talk today At Amazon, we are customer obsessed and work with customers very closely to help guide our products and investments. A couple years ago, we starting hearing customers ask us about the tools we use inside of Amazon—these customers were moving to the cloud, adopting agile process, SOA or microservices, and trying to figure out what tools to use. we were famously a devops company, released a ridiculous amount of software, and they started asking how we do it.
- - I'm going to start off by addressing the elephant in the room, and that is that I hate the term DevOps - it's a fuzzy term that people have used in many different ways - I was hesitant to use it in my talk, but I have to admit, it's the term that comes closest to describing this new style of software delivery - what I want to do now is align on what we mean when we say DevOps - rather that try to define it directly, I'm going to put it in the context of something most all of us are familiar with - the software development lifecycle - here's the general development lifecycle for an application or service - every new feature or bug fix goes through this process - developer writes code, code is built and unit tested, app is deployed to a testing environment for deeper testing, finally given a thumbs up and deployed to production where customers can use it - after that happens, the company can collect feedback from customers, make decisions, and continue to iterate and improve the product - there are a few important things to note here - the speed of completing this loop determines your business agility: to go from an idea, to a delivered feature, to learning about it and coming up with the next idea - the faster you can complete that loop, the faster you can innovate - if you can only complete this cycle once a month, you will be outmaneuvered by competitors that can do this every day - another point is that you're only adding value when you're writing code for new changes - the effort you spend in this middle section is lost time - don't get me wrong, you need to ensure high quality releases, but the less your team spends releasing software, the more time they can be writing code - to me, that's the essence of DevOps—to make this process as efficient as possible, and speed up the learning cycle
- - now to make this more concrete, let's look at the story of Amazon's transformation to DevOps - like most companies, we did not start out this way
- - if we go back to 2001, the amazon.com website was an architectural monolith - it had multiple tiers, but they consisted of tightly-coupled components - this monolith-first architecture is not uncommon - like many fast growing startups, you make tradeoffs early that optimize for short term speed but can cause longer term issues - this happened to Amazon, and as we scaled the website and the team, we started to get bogged down - to visualize why it was getting bogged down, let's look at the development lifecycle A dozen years ago, Amazon.com was a monolithic app, very centralized, (central deployment team) Breaking apart this monolithic app required two fundamental changes: (1) an architectural change and (2) an organizational change.
- - when you're working with a monolithic app, you have many developers all pushing changes through a shared release pipeline - this causes frictions at many points of the lifecycle - upfront during development, engineers need to coordinate their changes to make sure they're not stepping on each others' toes - you probably also need a "merge Friday" or maybe a "merge week" where all the developers have to compile their changes and resolve any conflicts for the next release - then when you're pushing the changes through the delivery pipeline, you need to push the entire app - you need to build everything, test everything, and deploy everything - even if you're just making a one-line change in a tiny piece of code you own, you still need to go through this heavyweight process and wait to catch the next train leaving the station - for a fast growth company trying to innovate and compete, this overhead and sluggishness was unacceptable - the monolith became too big to scale efficiently so we made a couple of big changes - one was architectural, and the other was organizational - updating one small piece required coordinating with multiple teams, integration testing with the rest of the website, and re-deploying everything - it was taking way too long for changes to get queued into releases and deployed to production - in addition to time delays, this also added organizational overhead - we had a hierarchical organizational structure to manage the cross-team coordination that was needed - we even had a central team whose sole job it was to deploy this monolithic app into production
- - we took the monolith and broke it apart into a service oriented architecture - factored the app into small, focused, single-purpose services, which we call "primitives" - for example, we had a primitive for displaying the buy button on a product page, and we had one for calculating taxes - every primitive was packaged as a standalone web service, and got an HTTP interface - these building blocks only communicated to each other through the web service interfaces - to give you an idea of the scope of these small services, I've included this graphic - this is the constellation of services that deliver the Amazon.com website back in 2009, 6 years ago - this term didn't exist back then, but today you'd call this a microservice architecture Moving to a SOA - everything gets a service interface - breaking monolithic app into small services (primitives) - picture of what amazon.com looked like after this transformation (actual depiction of our service architecture) - today, I guess it’s popular to call this “microservices”
- - in conjunction with breaking apart the architecture, we also broke apart the organization - we split up the hierarchical org into small teams - we called them 2-pizza teams, because if they got larger than you could feed with 2 pizzas, we'd break them up - in reality, the target number is about 8 people per team, so I personally think the 2 pizza goal is maybe a little too frugal - another important change that went along with this is cultural - when we split up the org, we gave the teams full autonomy - they became small startups that owned every aspect of their service - they worked directly with their customers (internal or external), set their roadmap, designed their features, wrote the code, ran the tests, deployed to production, and operated it - if there was pain anywhere in the process they felt it - operational issue in the middle of the night, the team was paged - lack of tests breaking customers, the team got a bunch of support tickets - that motivation ensured the team focused on all aspects of the software lifecycle, broke down any barriers between the phases, and made the process flow as efficiently as possible - we didn't have this term at the time, but this was the start of our "DevOps" culture Moving to agile (organizational change to two pizza teams) - speed & agility - ownership - autonomy & accountability - today, I guess it’s popular to call this “devops”
- - these two changes decoupled the teams and made it very easy for them to make decisions and write new code - but we had a tooling gap - the shared release pipeline was no longer workable, so teams were now responsible for releasing the services themselves - for a little while, these individual service teams started building their own tools, but those soon coalesced into a shared effort Awkward state. Painful, but necessary to make the transformation. - after making the architectural and organization changes, the small teams were able to quickly make decisions and produce new software - but they quickly realized that they needed new tools, because the old tools and processes no longer worked - gone was the single deployment team that everyone relied on - that wouldn't scale to be able to serve thousands of different teams with different services and technologies
- - Amazon started a shared team, which provided new tooling to the decentralized service teams - these new tools had some special characteristics - the tools had to be self-service, because there's no other way to be able to scale to that many customers - the tools had to be technology agnostic, because the teams chose many different types of platforms and programming languages for their services - the tools had to encourage best practices, while we allow autonomy, we also want to support shared learning across the teams so everyone can improve - and of course, in the service-oriented mindset, the tools were delivered as primitive services Needed new tools requirements: - decentralized ownership (self-service) - promote best practices without being restrictive (guardrails) - technology agnostic - support agile SDLC
- - one of the first primitives to emerge was Apollo, a name that we clearly borrowed from Nasa - Apollo is the deployment engine for Amazon, everything from the retail site to AWS services - it's how we roll out software changes across our servers - we first launched Apollo over a dozen years ago - in that time we've been continually learning about how to manage deployments and baking that knowledge back into the service - one capability was zero downtime deployments - there's no way we would allow taking the retail site down just to push a software change - Apollo supports rolling out a software change without taking down an application - we also can't let a deployment bug take down the app, so Apollo tracks deployment health and stops bad deployments Apollo - dozen years ago Deploy without downtime Health checking Centralized dashboard
- - another primitive that emerged was Pipelines, our continuous delivery service - started after we did a study of how long a software change took to go from a developer check in to running in production - I'm not going to share any numbers, but let's say that it was embarrassing how long that took - we found that it wasn't the builds, tests, or deployments that were taking so long, but rather the human processes that tied them all together - one person would notify another person that a task was ready, eventually they'd see the request and batch it with others, finally they'd start a job and let it run, they'd come back later to see if it completed successfully or needed to be re-run, then they'd finally route the task onto another group for the next job - this process added in a ton of human delay, and for a company with an insane focus on efficiency, this was unacceptable - since we're automating our fulfillment centers, we thought we should automate our software delivery - we created Pipelines to automate that end-to-end release process, from code check-in to build to test to production - this tool is used pervasively across Amazon, by well over 90% of the teams - Pipelines is our internal continuous deployment tool About 6-7 years ago, we looked at how long it was taking teams inside of amazon to go from source code checkin to have the code running in production, and we discovered that it took an embarrassing long time (several weeks). What we discovered was that it wasn’t that the tools were taking that long. It was all the manual human processes in between (approval, feedback, notifications). We were wasting a lot of time because humans were advancing software through the different phases of the development process (source, build, test, deploy). so we built “pipelines” (and this is before the industry even had a term for it). we built in all the traditional aspects of CI/CD (internal build and test), but we extended that to also include what is sometimes referred to as “release automation” (defining the different stages of your application—alpha, beta, gamma, prod -- and orchestrating the release of software through that—and even beyond that—to different AZs and different regions. What is now called “continuous deployment” - Today, most teams at amazon use pipelines (>90% of teams)
- - with these new tools, we completed the puzzle - the teams were decoupled and they had the tools necessary to efficiently release on their own
- - what does success look like - there are a lot of ways that you can measure the process, and no one way is perfect - but here's one data point - when you have thousands of independent teams - producing highly-factored microservices - that are deployed across multiple dev, test, and production environments - in a continuous delivery process - you get a lot of deployments - at Amazon in 2014, we ran over 50M deployments - that's an average of 1.5 deployments every second So, what happens when you have thousands of development teams… each running a microservices or SOA architecture… each with multiple environments (alpha, beta, gamma, prod… something multiple regions)… each doing continuous deployments? You get *a lot* of deployments. Last year, Apollo handles 50 million deployments—that’s 1.5 deployments per second!
- - now let's go back to our customers once again - after they ask us how we do it, they usually follow with how can they do it too - that's the focus for the remainder of this talk - I'm going to talk about the new AWS Code services and how you can use these to set up your own DevOps pipelines But AWS customers want the same tooling to release as frequently and reliably as Amazon - the new Code services will enable customers to do just that - we'll walk through how you can use these tools to wire together your own DevOps pipeline
- - now to make this more concrete, let's look at the story of Amazon's transformation to DevOps - like most companies, we did not start out this way
- - the first service I'd like to introduce is CodeDeploy - CodeDeploy is the externalization of our internal Apollo service, and it enables you to deploy just like Amazon - you specify what version of your application to install on what group of servers, and CodeDeploy coordinates that rollout for you - it has the same rolling update feature to deploy without downtime - it has the health tracking feature to stop bad deployments before they take down your application - all you do is define how to install your application on a single machine, and CodeDeploy can scale that across a fleet of hundreds of servers - when we launched CodeDeploy, it only supported deployments to Amazon EC2 instances - but earlier this year, we released support for on premises deployments - this allows you to deploy to servers in your private data center, as well as VMs in other clouds - as long as the machine can run our agent and make calls to our public service endpoint, you can deploy to it - this means you can have a single tool to centralize the deployment for all of your applications to all of your different environments
- - the next service is CodePipeline, which was inspired by our internal Pipelines service - it allows you to completely model out your custom software release process - you specify how you want your new code changes built and unit tested, how they should be deployed to pre-production test environments, how they should be validated with functional and performance tests, and ultimately how they should roll out to production - you have complete control over the end-to-end workflow and how each step is performed - you can connect each step to an AWS service like CodeDeploy, your own custom server like Jenkins, or even an integrated partner tool like GitHub - it's completely extensible and allows anyone to plug in - one of the great things about CodePipeline is how it pulls in our large ecosystem of great developer tool partners - you'll see in the upcoming demo how easy it is to discover and connect to these partner services, and include them as a step in your own release process - after you set up your automated release process, then you're free to push changes as often a you like - CodePipeline will automatically marshal your code changes through your validation steps as quickly as possible, while at the same time maintaining a consistent bar for quality control
- - the final service is CodeCommit, where we implemented the Git protocol on top of Amazon S3 storage - this means from the front-end, it behaves like any other Git source control system - you continue to use the same Git tools and issue the same Git commands that you do today - on the backend though, we've taken a whole new approach - rather than use a file-system based architecture, we use the cloud-scale storage of Amazon S3 and DynamoDB - this brings all of their benefits of replicated storage, plus some interesting bonus features - CodeCommit automatically encrypts all repositories using customer-specific keys - this means that every customer will have their repositories encrypted differently in S3
- - I demonstrated using a few partner solutions with our AWS Code services - Here's the full list of partners who have integrated their tools with CodePipeline and CodeDeploy - And this list is growing as we welcome more integrations into our tools suite - Many of these partners have booths in the expo hall - I encourage everyone to explore their solutions to see how they might benefit your cloud development projects - We even have some partners here at this talk, so when we take Q&A (up front|| out in the hallway) after the talk, you'll be able to ask questions of them as well
- - now to make this more concrete, let's look at the story of Amazon's transformation to DevOps - like most companies, we did not start out this way