Deploy Faster Without Failing Faster - Metrics-Driven - Dynatrace User Groups - June 2015
•Als PPTX, PDF herunterladen•
4 gefällt mir•1,161 views
Do it like the "DevOps Unicorns" Etsy, Facebook and Co: Deploy more frequently. But how and why? Challenges? Deploying Software Faster without Failing Faster is possible through Metrics driven Engineering. Identify problems early on using a "Shift-Left in Quality". This requires a Level-Up of Dev, Test, Ops, Biz See some of the metrics that I think you need to look at and how to upgrade your engineering team to produce better quality right from the start
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (20)
Ähnlich wie Deploy Faster Without Failing Faster - Metrics-Driven - Dynatrace User Groups - June 2015
Ähnlich wie Deploy Faster Without Failing Faster - Metrics-Driven - Dynatrace User Groups - June 2015 (20)
Mehr von Andreas Grabner
Mehr von Andreas Grabner (14)
Kürzlich hochgeladen
Kürzlich hochgeladen (19)
Deploy Faster Without Failing Faster - Metrics-Driven - Dynatrace User Groups - June 2015
- 1. @Dynatrace - More on http://blog.dynatrace.com - Dynatrace Free Trial: http://bit.ly/dttrial (R)Evolutionize APM Hosted by: Andreas Grabner - @grabnerandi DevOps
- 5. Waterfall -> Agile: 3 Years 1 Deployment per Month “EVERYONE can do Continuous Delivery” “EVERY MANUAL TESTER DOES AUTOMATION!”
- 9. SO – Why did they do it?
- 17. Uber: 1 million Shared taxis per day Didi: 5 million Sources: KPCB, Uber. Date: May 2014 (Didi), Feb 2015 (Uber)
- 18. Rapidly Changing Markets, Requirements & User Expectations
- 19. GOAL!
- 20. Utmost goal: minimize cycle time feature cycle time time Customer / Market Users
- 21. Utmost goal: minimize cycle time feature cycle time time minimize Users
- 22. Utmost goal: minimize cycle time feature cycle time time This is where you create value! minimize You
- 23. ROI?
- 24. High Performers Are More Agile 30x 8,000x more frequent deployments faster lead times than their peers Source: Puppet Labs 2013 State Of DevOps: http://puppetlabs.com/2013-state-of-devops-infographic
- 25. High Performers Are More Reliable 2x 12x the change success rate faster mean time to recover (MTTR) Source: Puppet Labs 2013 State Of DevOps: http://puppetlabs.com/2013-state-of-devops-infographic
- 26. Challenges?
- 27. Deploy Faster!!
- 28. Fail Faster!!?
- 29. Right Focus?!
- 31. Availability dropped to 0%
- 36. Technical Debt!
- 37. 80% $60B
- 38. Insufficient Focus on Quality
- 39. The “War Room” Facebook – December 2012
- 41. 20% 80%
- 43. Virtual War Room
- 44. Virtual NO War Room
- 45. • “All-In” Agile / “All-In” DevOps • PurePath: Level-Up Dev, Test, Test Automation, Ops, Biz • High Level of Automation • REST: Automated Deployment, Metrics, Data Analysis • Small Iterative Releases • UEM & BTs: We understand Builds, App Versions, … • Metrics-based Feedback Loops • PurePath: Meaningful metrics to make good decisions for everyone • Collaboration X-Teams • Dynatrace Sessions: PurePath, PureStack, PureLytics • Eliminate Unplanned Work • Test Automation: Automate finding problems early – saving time later DevOps 2.0: Built-In Continuous Quality!
- 46. “All In” Agile / DevOps
- 47. All-In Agile/DevOps Browser Diagnostics Response Size, # Images JBoss, Tomcat, JConsole IIS, ASP.NET, PerfLib Oracle, SQL Garbage Collection Thread Dumps
- 48. Architecture # of Services # of Databases # of Servers # of Calls between Components
- 49. Performance Page Load Time Render Time SQL Query Time Service Call Time
- 50. Scalability Memory Usage per User # Connections Cache Utilization Load Distribution Component Roundtrips
- 54. Level-Up: Share and Collaborate
- 57. Putting it into Continuous Deployment 12 0 120ms 3 1 68ms Build 20 testPurchase OK testSearch OK Build 17 testPurchase OK testSearch OK Build 18 testPurchase FAILED testSearch OK Build 19 testPurchase OK testSearch OK Build # Test Case Status # SQL # Excep CPU 12 0 120ms 3 1 68ms 12 5 60ms 3 1 68ms 75 0 230ms 3 1 68ms Test & Monitoring Framework Results Architectural Data We identified a regresesion Problem solved Exceptions probably reason for failed tests Problem fixed but now we have an architectural regression Problem fixed but now we have an architectural regressionNow we have the functional and architectural confidence Let’s look behind the scenes
- 58. #1: Analyzing each Test #2: Metrics for each Test #3: Detecting Regression based on Measure
- 61. Release Acceptance Testing Unit Testing Performance Testing Monitor Tests Analyze Results Quality Gate in your Build Tool Every 11.6 seconds
- 65. Example from Web Diagnostics 282! Objects on that page9.68MB Page Size 8.8s Page Load Time Most objects are images delivered from your main domain Very long Connect time (1.8s) to your CDN
- 66. Online Performance Clinics Every Other week @ bit.ly/onlineperfclinic
- 67. Your Benefits • Free Performance Review • Extended Dynatrace License “Share Your PurePath” bit.ly/sharepurepath My Benefits • More blog material for next year • Gratification that I could help you
- 68. 68 @Dynatrace Questions and/or Demo Slides: slideshare.net/grabnerandi Get Tools: bit.ly/dttrial YouTube Tutorials: bit.ly/dttutorials Contact Me: agrabner@dynatrace.com Follow Me: @grabnerandi Read More: blog.dynatrace.com
- 69. 69 @Dynatrace Andreas Grabner Dynatrace Developer Advocate @grabnerandi http://blog.dynatrace.com
Hinweis der Redaktion
- We managed to misuse the word DevOps – ALL of us in the industry
- Leading to a DevOps Hangover
- But it exists – and companies at Velocity presented their latest success stories – let me give you a quick highlight tour from Velocity 2015 in Santa Clara
- The world around us is changing
- We are consuming software services all the time through different devices
- And its not just us – but the next generation of users is already there: less patient – more demanding!
- We also see a huge shift on eCommerce
- And a lot of new market disruptors
- And there are markets we do not even tab into yet: more people living within that circle than outside – and they are just getting started!
- We always think we are already spearheading – but – if you compare the two most popular ride share services in the US and China we can see that they already do much more than we do
- There is rapid change in requirements and user expectations for us as service providers
- Cycle time is the most relevant metric in the software delivery process. “How long would it take your organization to deploy a change that involves just one single line of code?” Mary Poppendieck
- Cycle time is the most relevant metric in the software delivery process. “How long would it take your organization to deploy a change that involves just one single line of code?” Mary Poppendieck
- Cycle time is the most relevant metric in the software delivery process. “How long would it take your organization to deploy a change that involves just one single line of code?” Mary Poppendieck
- So – our goal is to deploy new features faster to get it in front of our paying end users or employees
- For many companies that tried this it may also meant that they fail faster
- Its also very important to keep the focus right – building and fixing those things that matter. If you car burns down it is not time to take a selfie!
- If you deploy and availabilty goes to 0 you know it is time to act and not take a selfie!
- This PurePath tells me I have to react immediately – happened to us last week after we made a small deployment change in the way we send emails. It had huge impacts. A single metric “Count of Exceptions” showed us that we had this problem!
- Be aware that not ever feature is yet ready for prime time. Be aware that not all features are used that way you think they will
- Make sure you monitor user behavior
- And of course: Focus on Performance – we know the studies!!
- If we don’t continuously improve and also take the risk of taking old code out we just add technical debt. We add one feature after the next not knowing how it impacts current operation and what it will mean for engineers one or two releases down the road …
- Based on a recent study: 80% of Dev Team overall is spent in Bugfixing instead of building new cool features $60B annual costs of bad software instead of investing it in new cool features to spearhead competition
- We need to leave that status quo. And there are two numbers that tell us that it is not as hard to do as it may seem
- Based on my experience 80% of the problems are only caused by 20% problem patterns. And focusing on 20% of potential problems that take away 80% of the pain is a very good starting point
- Sounds super nice on paper – so – how do we get there?
- The solution is not a virtual war room as some other tool vendors may make you believe
- We want to get rid of the war room. WE know – its not 100% possible – but we can do much better than we do now
- Here is what it takes!
- Level-Up Skills by talking and exchanging ideas for your piers: developers, ops, business It is important that both sides start understanding the challenges of the other side. It is important that they speak the same language, e.g: what does this metric mean to you? How do you measure it? Sit down and level-up skills for everybody and agree on a common set of tools and metrics But what types of metrics?
- Metrics around Architecture, e.g: how many web service calls does it take to implement this new feature? How many AJAX calls do we make when people logon to our site? Is that smart?
- How fast is that piece of code? Is it efficient in its usage of CPU, Memory, Disk and Network?
- Is the application also going to scale? Which components perform better or worse with increasing load? Where is the breaking point? Where is the API that is the issue? Whats the architectural decision behind the app not scaling?
- Make Ctrl-Shift-I your friend
- Transaction Flow can be read and understood by everyone – and we can easily find problems!
- Make sure you do end-to-end checks. Let you testers capture all technical details of the test case. No need to reproduce any problem later on the dev workstation.
- Share your data: Dynatrace makes that easy with Sharing a Session File
- Dynatrace Free Trial is the key to a general Level Up. Sign up for it @ http://bit.ly/dttrial
- Once we figured out how to get these measures it is time to automate the capturing but also automate quality alerting in case these metrics are showing us that we ran into one of these well known use cases.
- Here is how we do this. In addition to looking at functional and unit test results which only tell us how functionality is we also look into these backed metrics for every test. With that we can immediately identify whether code changes result in any performance, scalability or architectural regressions. Knowing this allows us to stop that build early
- This is how this can look like in a real life example. Analyzing Key Performance, Scalability and Architectural Metrics for every single test
- Build by Build view in Dynatrace
- Or integrated into Jenkins, Bamboo, …
- Now as we know which metrics we need to look at and how to automate the capturing and detect regressions from build to build we simply add it to the continuous delivery pipeline by letting these metrics act as quality gateways. We do not let a build move forward if we already know that it has a well known problem. Here are all the benefits Only good code reaches production We eliminate time spent in later stages if we already identify problems earlier We all level up our skills and become a better team We produce better software faster -> we don’t crash the car
- I do this stuff every day – I am happy to help you as well
- Send me your data and I do a free performance review through my “Share Your PurePath” – http://bit.ly/sharepurepath - program
- Watch me online how I analyze performance or watch the recordings on my YouTube Channel