DevOps and the Death & Rebirth of Childhood Innocence
•Als PPTX, PDF herunterladen•
1 gefällt mir•387 views
The document discusses the evolution of software development from a fun childhood activity to a complex enterprise process due to increased regulations and procedures. It argues this led to failures like the Equifax data breach. The solution proposed is adopting modern DevOps practices like continuous integration/delivery, immutable infrastructure, and automated testing to make deploying code simple and safe again. The talk advocates configuring applications to run securely in containers managed by Kubernetes for easy, isolated testing of all code branches.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie DevOps and the Death & Rebirth of Childhood Innocence
Ähnlich wie DevOps and the Death & Rebirth of Childhood Innocence (20)
Mehr von Robert Douglass
Mehr von Robert Douglass (8)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
DevOps and the Death & Rebirth of Childhood Innocence
- 1. Platform.sh, Moscow, November 15, 2018 DevOps & the Death and Rebirth of Childhood Innocence Robert Douglass, Chief DevRel Officer, Platform.sh @robertDouglass
- 2. Rob’s First Law of Coder Dynamics
- 3. Problem
- 4. Solution?
- 6. Solution?
- 12. In the beginning… computers were toys.
- 13. We played with them.
- 21. In the beginning… deploying code was fun.
- 26. All of the keywords in C64 BASIC
- 36. Problem
- 37. Solution?
- 45. Goal
- 59. Source: Carlos León, yesterday’s talk
- 60. Source: Carlos León, yesterday’s talk
- 61. "Why should you (a DevOps person) be the one who fixes someone else's code running in production?" Carlos León, Adopting DevOps: Are You Still on Time?
- 65. Part III: The Death of Childhood Innocence
- 67. Equifax is disaster of such epic proportions that their brand now looks like this:
- 68. Equifax is a clusterfuck of such epic propotions it is not an easy subject
- 69. We will look at just two of the epic moments of this saga. 1. The infamous CVE-2017-5638
- 72. At $4,000,000,000 this person is worth way more than Steve Austin who is currently worth $29,791,399* *adjusted for inflation since 1974.
- 73. What was the salary of the person who has the manual/menial job of “update this package” ?
- 74. “Equifax CEO Richard Smith Who Oversaw Breach to Collect $90 Million” http://fortune.com/2017/09/26/equifax-ceo-richard-smith-net-worth/
- 75. Who done it? Was this a fault of Gary, the developer who didn’t apply the patch? Was this the fault of his manager Diane? I posit this was the fault of faulty thinking about software.
- 76. Software is a function of time.
- 77. ● Over time two things happen to software: ○ Creating new stuff ○ Repairing broken stuff Software is a function of time.
- 78. Creating new stuff is voluntary. You do it on your own rhythm. The better automation you have the faster and more productive you will be. The better your tests are … the less you will suffer from quality degradation and rot. Software is a function of time.
- 79. Repairing broken stuff is not on your own rhythm. The fix for CVE-2017-5638 should have been deployed an hour after it was out. Software is a function of time.
- 80. We will look at just two of the epic moments of this saga. 1. The infamous CVE-2017-5638 2. The brilliant, flawless response on the part of Equifax
- 83. No. It wasn’t this. That would be semi-competent.
- 89. Is it Diane or Gary’s fault again? No. It is about snowflakes. When infrastructure is done by hand you need a “change request form”.
- 90. There is no way in hell a “mature enterprise” will have procedures that are lightweight enough to roll- out a full new project in a day. If you need to go through IT and Security. If you need to fill a form.
- 91. In an emergency someone will “power through”. And when that happens, you can get Equifaxed.
- 92. And when you get Equifaxed….
- 94. Part IV: The Rebirth of Childhood Innocence
- 96. 2018-11-14 “Adopt a Dev-Centric Position” Gil Zellner, From Ops to Dev and Back Again
- 97. ✤deploy by git push or git merge. This triggers the entire process. Note that this is NOT “ssh into the server and git pull”
- 98. “Make release management a functional practice instead of a technical one” Rene van Osnabrugge, Continuous Delivery 3.0 – the Next Step
- 99. ✤Your system (code / applications / services) has a build process. Embrace that fact and standardize it for every member of the team.
- 100. ✤ Alway test against real data. Use copy-on-write technologies (eg. CEPH) to make production data available for every test environment.
- 101. ✤Every build should have a testable URL that you can send to stakeholders.
- 102. “Create feedback loops to improve communication, Involve stakeholders and management” Mieke Deenen, Get Started with DevOps for Government
- 103. ✤ Built code should be put onto a read-only file system. Your operating system should be put onto a read-only file system. The only writable volumes should be protected from program execution. Many applications violate this but there are workarounds.
- 104. ✤ Read-only file systems lead to disposable containers. Containers should be built, used, then thrown away. Don’t update containers. Don’t maintain containers. Rebuild them and replace them.
- 105. “Think immutable - build your infrastructures from code” Rene van Osnabrugge, Continuous Delivery 3.0 – the Next Step
- 106. ✤ Developers should have some (but not total) control over the services and environment. For example: runtime versions configuration of services that affect the application nginx redirects / rewrites extensions, libraries, and dependencies
- 107. ✤Variables that change per-environment should be injected into the running application.
- 108. “If you're running your application in a Docker container, you want to configure that from the outside.” Patrick Baumgartner, Demystifying Spring Boot Magic
- 109. ✤Sensitive data (keys, tokens) should also be injected into the running application.
- 110. "You don't want sensitive information in your configuration files" Benny Michielsen, Secure Multi-tenant Apps with Azure and VSTS
- 111. ✤Deploy and test every git branch in isolation.
- 112. ✤ Deploy all applications and services in the same way. Your Node.js app shouldn’t have a fundamentally different deployment process than your Python app.
- 113. ✤ Make it easy - automatic! - to practice database migrations. Test them on every deploy.
- 114. ✤ Deployments should be optimized for protecting data consistency, but this may come at a price: downtime.
- 115. ✤Feature toggles are a hack. Don’t do them. Unless there’s no other way.
- 116. ✤ If you have to do database migrations that take a longer time or will lead to downtime, you should have seen Michiel Rook’s session yesterday. Michiel Rook Database Schema Migrations with Zero Downtime
- 117. ✤ If you have to do your own container orchestration, use Kubernetes.
- 118. ✤ We are betting on Kubernetes for everything. Kubernetes is the defacto standard now. Carlos Sanchez (Cloudbees) Using Kubernetes for Continuous Integration and Continuous Delivery
- 122. @platformsh CONTINUOUS DEPLOYMENT CLOUD HOSTING
- 123. Moscow, November 15, 2018 DevOps & the Death and Rebirth of Childhood Innocence Robert Douglass, Chief DevRel Officer, Platform.sh @robertDouglass
Hinweis der Redaktion
- Before we really begin, I need to introduce you to Rob’s First Law of Coder Dynamics:
- It says that When solving a problem
- and following a logic path towards a solution
- the larger the distance between the current topic and the original topic
- the more likely you are to install software you don't need.
- For example: Programmer: I think I'll write an app that helps me identify vintage autos! Thought path: Autos -> image recognition -> training algorithms -> python libraries -> C bindings -> static binaries
- BOOM! Time to download Call of Duty Black Ops 4!
- This is, by the way, very closely related to Rob’s Second Law of Coder Dynamics which says that for every second you spend waiting for a page to load, the likelihood of spontaneously consuming 1,000 calories in the form of Coke and Pringles increases by 1000%.
- through play, we learned to program. as programmers, the feeling of power and control excited us.
- when we told our computer to do something, it obeyed we imagined all of the fantastic programs we would build.
- these became our goals. We are very goal oriented people. We learned to problem solve.
- We were children, innocent children, fascinated with the endless possibilities our incredible toys afforded us.
- Our Childhood Innocence allowed us to play without boundaries, and thus create the magnificent world we enjoy around us today, full of incredible sites, apps, and tools, enabling communication and sharing, improving lives around the world.
- This is a story of how we have lost that Childhood Innocence. How it crashes, burns, and falls apart, leaving us bitter, confused, and unable to focus on the things we want to build.
- You would subscribe to a magazine with computer programs printed in it, and every month you'd spend time typing the new programs into your computer. When I started, as I didn't have enough money to buy a disk drive, I would use the program as long as the computer was turned on, and it would disappear from RAM when I switched it off. Later, if I wanted to run the program again, it was a cost-benefit-analysis situation. Was the program good enough to merit typing it all in again? Usually if the answer was "yes", it was because I had come up with some ideas on how to change, it, extend it, hack it. Then I'd gladly type it all in again.
- And, if you don’t believe me, you can walk into the room back there and visit the vintage computer collection, and see for yourself that even in 1986, that’s how programs were being distributed.
- Later, with a disk drive that I purchased with birthday and Christmas money, I'd go over to my friend's house, with the drive, and make copies of programs on disks. Then I could run those programs at home, too.
- These programs were scripts and they just ran. The computer "interpreted" them. They used languages like BASIC. Or Perl. Or later, PHP.
- When you ran into the limits of scripting languages, you moved on to compiled languages. These were more rigorous, serious languages, like C, C++, or later, Java. Compiling code was cool. But it was also the first moment when we began to trade our innocence for power.
- If you didn't have a computer or game console, you'd go over to your friend's house and play with theirs. They were the cool kids. If you had an Atari 2600 and your friend had an Atari 5200, your friend was the cooler one, and you'd spend time at their house.
- We hooked our toys up to networks, first with modems, later with DSL, wifi, and then 4G. Maybe your first computer was a smartphone. If so, you’re young and beautiful and probably have no idea what I'm talking about. Enjoy your innocence =)
- The network let you skip the magazine subscription, typing, and compiling altogether. You could connect to other computers and just copy the programs or files. Suddenly the world seemed both endlessly vast, and yet a lot smaller than it used to.
- Now the cool kids were the ones who ran computers on the network- the servers. They'd run bulletin boards, or forums, or chat rooms.
- When the internet happened, everything started to move to the network. We didn't need to store things on our own computer if we knew it was being stored on someone else's computer, and if we knew we could find it.
- Programming changed too. Now you could write programs that depended on other programs. This made our programs more powerful. We could benefit from the good ideas and hard work of others. But while these dependencies bring benefits, they signify another major step in the tragic death of our childhood innocence: managing dependencies.
- I got a glimpse of that the first time I ran Maven. I felt like I had just downloaded the whole internet! So many jars! It was wonderful.
- Dependencies are great, but they bring many dangers. They require strict version management. Dependencies that you blindly download might have security problems, or bugs, and you might not be in a position to fix those or deal with them. Dependencies become abandoned. Their APIs change. Dependencies might have their own dependencies. And those might have further dependencies. And amazingly, in some cases, if you're not careful, they might have circular dependencies with version mismatches, so you might end up with multiple versions of the same package.
- By the way, I really love this exhibition next door. If you haven’t seen it, make sure and take a look. The history of computing is really fascinating and I’m happy to have seen a part of it happening.
- What was our goal as beginning programmers?
- what problem were we trying to solve?
- Originally I just wanted to write games. I wanted to be Sid Meier. If I had been a smarter 10 year old I could have met him, since he was my neighbor. Sadly I never became a games programmer.
- Later, I wanted to build all sorts of websites. I wanted to build a news aggregator.
- I wanted to build a CMS for musicians.
- I wanted to build a learning management system.
- I wanted to build a video rating site.
- I wanted to build an app that recognizes stolen violins.
- I wanted to build a specialized dating app. Those were my goals.
- In all of the cases, the goal was an app or a website that did a specific thing. My interest was in solving real world problems, and creating specific digital experiences. Those goals would be the center red dot in my diagram.
- In my innocence, I thought that building those apps would involve lots of thinking about the actual functionality, user experience, and business models around those ideas. In reality, it was more like: -> I need a rating widget -> npm needs to be updated -> the resulting new version of my XML parser dies on a manifest in a library I didn't know I had -> my two factor authentication on Github has expired (click!) PRINGLES!!!! DOWNLOAD PHP STORM!!!!
- 1000 CALORIES! NOW!!!!
- In each case, the innocent idea got worn down more and more by the need to maintain the tooling required to do development. Innocence didn't die there, but it was weakened. A little like Gandolf in Moria.
- Remember the cool kids who could run their own servers? Let's call them Operations. Their world has also evolved. They innocently started running Bulletin Boards, forums, or sites, which allowed them to communicate with other humans and have social contact without ever having to leave their bedroom, let alone turn on the lights, put on clothing, or brush their teeth
- They too lost their innocence. Operations exploded as the internet began to scale. Setting up one server was fun. Maintaining a fleet of aging servers while people still asked you to fix their printers was not so much fun.
- Fortunately for the Operations people, virtualization came and everything be done with nice clouds. Much of the complexity and all of the cables have been abstracted conveniently away by services such as AWS and Google Cloud.
- The server admin used to use tools like: VirtualBox, VMWare, Vagrant, CHRoot, CPanel, Capistrano, Puppet, Chef, Ansible and so forth.
- By the way, do you know how you turn on the fan in your laptop from the command line? Vagrant up.
- Now the operations people use things like containers, and container orchestration. More buzzwords, more power, more complexity. You can hear ops people talking about things like: kubelets, pods, replication controllers, configmaps, persistent volume claims, swarms, and trust.
- To keep up, and to be able to work together, The Developer is thinking about things like: Docker compose, build scripts, deployment pipelines, local environments, QA environments.
- The core problem is that, Operations have gotten really really good with all the powerful tools they have at creating huge and very complicated infrastructures which meet the demands of modern computing workloads.
- And the Developers have gotten really really good with all the powerful tools they have at creating huge and very complicated application architectures which meet the demands of modern applications.
- And that’s how we ended up at the Wall of Confusion between Developers and Operations, each losing their innocence by creating more and more complexity. But they depend on each other, and both are needed to run web applications.
- Which is why we have DevOps. DevOps is meant to help these poor Developers and poor Operations people work together, and maybe, if done really well, restore some of the Childhood innocence that makes our work fun and meaningful.
- But Operations people shouldn’t need to know how the programs run. They shouldn’t have to worry about the happiness and childhood innocence of the developers.
- And application developers are dealing with enough complexity that they shouldn’t have to deal with kubelets, pods, replication controllers, configmaps, persistent volume claims, swarms, and trust. Right???
- When application developers have to spend that much effort on DevOps issues, it’s time to admit there’s a problem. Someone has to stage an intervention. Hello, my name is Robert. I'm developer and I do DevOps. It's been 31 days since I've logged in as root.
- While awesome, the new breed of Devops tools are for application developers, just a distraction on the logic path of problem solving that you should be doing for your application. And you know what happens when you get distracted from your goal. Pringles.
- 800 million individual consumers US$3.1 billion in annual revenue They look at your ability to pay, run credit checks on you, etc. The data they collect on you can decide whether you can take a loan, buy a house, get a job, even sign up for an internet contract.
- Equifax is disaster of such epic proportions that their brand now looks like this:
- Why is their brand so bad? Aside from the fact that Equifax was the subject of more than 57,000 consumer complaints to the Consumer Financial Protection Bureau, with most complaints relating to incomplete, inaccurate, outdated, or misattributed information held by the company. In September 2017, Equifax announced a cyber-security breach where criminals accessed approximately 147 million consumers' personal data, including their full names, Social Security numbers, birth dates, addresses, and driver license numbers. Equifax also confirmed at least 209,000 consumers' credit card credentials were taken in the attack.
- Apache Struts allows remote attackers to execute arbitrary commands
- Let’s call him Gary.
- 1974
- How much do you think the CEO who oversaw the company at the time of the breach got when he resigned? I bet we all wish we had a job like that.
- Creating new stuff is voluntary. You do it on your own rhythm. The better automation you have the faster and more productive you will be. The better your tests are … the less you will suffer from quality degradation and rot.
- 2 The brilliant, flawless response on the part of Equifax
- The site the company set up for consumers to check if they were affected by the breach asked for six digits of the visitor’s Social Security number, then told people their data was exposed no matter what they entered. The website itself was also vulnerable to hacking, not that it really mattered all that much seeing as Equifax sent people to a phishing site set up to look just like the real one.
- Is it Diane or Gary’s fault again? No. It is about snowflakes. When infrastructure is done by hand you need a “change request form”.
- There is no way in hell a “mature enterprise” will have procedures that are lightweight enough to roll-out a full new project in a day. If you need to go through IT and Security. If you need to fill a form.
- At the moment you have been equifaxed, you have experienced the absolute horrifying death of Childhood Innocence. No more play. No more discovery. No more wonder. No lightness. Only darkness. Every character that you code will feel like bitter, hard work and suffering.
- Now we've seen how the Childhood Innocence of a developer dies. Is that the end of the story? Are we destined to fight with overwhelming complexity, distracting us from focusing on our passions and ideas? will the bad guys always be able to break in and steal our data?
- The answer to regaining childhood innocence was given by Gill Zellner yesterday when he said “Adopt a Dev-Centric Position”. What follows is a set of guidelines that I recommend for doing that - for using DevOps ideas and tools with the goal of having a Dev Centric Position.
- deploy by git push or git merge. This triggers the entire process. Note that this is NOT “ssh into the server and git pull” Note that uploading a zip file to AWS Lamda violates this.
- “Make release management a functional practice instead of a technical one” Rene van Osnabrugge, Continuous Delivery 3.0 – the Next Step
- Your system (code / applications / services) has a build process. Embrace that fact and standardize it for every member of the team. Whether they are front end or back end or dev ops shouldn’t matter, they should all build the same way.
- Alway test against real data. Use copy-on-write technologies (eg. CEPH) to make production data available for every test environment.
- Every build should have a testable URL that you can send to stakeholders.
- “Create feedback loops to improve communication, Involve stakeholders and management” Mieke Deenen, Get Started with DevOps for Government
- Built code should be put onto a read-only file system. Your operating system should be put onto a read-only file system. The only writable volumes should be protected from program execution. Many applications violate this but there are workarounds.
- Read-only file systems lead to disposable containers. Containers should be built, used, then thrown away. Don’t update containers. Don’t maintain containers. Rebuild them and replace them.
- “Think immutable - build your infrastructures from code” Rene van Osnabrugge, Continuous Delivery 3.0 – the Next Step
- Developers should have some (but not total) control over the services and environment. For example: runtime versions configuration of services that affect the application nginx redirects / rewrites extensions, libraries, and dependencies
- Variables that change per-environment should be injected into the running application.
- “If you're running your application in a Docker container, you want to configure that from the outside.” Patrick Baumgartner, Demystifying Spring Boot Magic
- Sensitive data (keys, tokens) should also be injected into the running application.
- "You don't want sensitive information in your configuration files" Benny Michielsen, Secure Multi-tenant Apps with Azure and VSTS
- Deploy and test every git branch in isolation.
- Deploy all applications and services in the same way. Your Node.js app shouldn’t have a fundamentally different deployment process than your Python app.
- Make it easy - automatic! - to practice database migrations. Test them on every deploy.
- Deployments should be optimized for protecting data consistency, but this may come at a price: downtime. Therefore it’s also important to have a risky deploy mode that quickly swaps out application code but does nothing else.
- Feature toggles are a hack. Don’t do them. Unless there’s no other way. Deploy the code you want to run. Test new features in test environments that are fast to deploy and exact copies of production. Make sure your application can be deployed quickly.
- If you have to do database migrations that take a longer time or will lead to downtime, you should have seen Michiel Rook’s session yesterday. Michiel Rook Database Schema Migrations with Zero Downtime
- Deployments should be optimized for protecting data consistency, but this may come at a price: downtime. Therefore it’s also important to have a risky deploy mode that quickly swaps out application code but does nothing else.
- We are betting on Kubernetes for everything. Kubernetes is the defacto standard now. Carlos Sanchez (Cloudbees) Using Kubernetes for Continuous Integration and Continuous Delivery
- Saving Childhood Innocence then means removing the complexity and distraction that stands betwen you and your programming goals. Saving Childhood Innocence means not going outside of your Circle of Concentration when following the idea threads and problem solving.
- Ideally, as an Application developer, you should be able to describe what you need for your application to run. You should be able to say, for example [click] "I need PHP 7.2, [click] and I need 50G of disk space, [click] and PostgreSQL, ElasaticSearch, and RabbitMQ. [click] and, oh, and make sure my PHP dependencies get resolved, composer and npm both run, and all my caches clear every time I merge code."
- And then, Our Developers and our Operations teams will live together in perfect harmony, building wonderful applications, improving the world we live in, and enjoying their Childhood Innocence.
- I am grateful to the company I work for, Platform.sh, for supporting me being here today, and for teaching me most of the concepts that I bring to you today.
- This presentation included a lot of nostalgia for the early days of personal computing as I experienced it, growing up in the USA in the 80’s and 90’s. I’d love to hear from some of you about your experiences growing up with computers in different countries.