A DevOps Journey - CDK Global
•Als PPTX, PDF herunterladen•
1 gefällt mir•1,224 views
Brian Garofola's presentation at "DevOps: The Key to Delivery Applications and Services Faster and Better" InformationWeek event in Chicago on December 3, 2014. CDK Global is the largest global provider of integrated information technology and digital marketing solutions to the automotive retail industry. Brian Garofola, Engineering Director at CDK Global, talks about the journey towards increasing the flow of value to customers by constantly improving the people, processes, and tools in the delivery pipeline.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (7)
Ähnlich wie A DevOps Journey - CDK Global
Ähnlich wie A DevOps Journey - CDK Global (20)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
A DevOps Journey - CDK Global
- 1. A DevOps Success Story Journey Brian Garofola Engineering Director, CDK Global
- 2. 9,000 total associates 40 years in business 26,000 customers worldwide $2 billion annual revenue 1,400 associates in R&D Headquartered in Hoffman Estates, IL CDK Global is the largest global provider of integrated information technology and digital marketing solutions to the automotive retail industry.
- 3. Enter Presentation Name via Insert Tab > Header & Footer Page 3 So what is DevOps?
- 4. DevOps is having everyone pulling in the same direction by sharing common goals and priorities, sharing responsibility for successes and failures, and constantly improving communication, processes, and tools to increase the flow of value to customers
- 5. A Textbook DevOps Problem • 12 - 18 month release cycles • Manual and error prone deployments • Snowflake infrastructure • ~45 days from request to working environment • Traditional QA team performing manual tests • Different reporting lines to the executive level for R&D and IT • Over 100 different products on a variety of platforms Enter Presentation Name via Insert Tab > Header & Footer Page 5
- 7. • Quarterly meeting with leaders from R&D and IT • Rate ourselves across 20 operational perspectives • Identify high risk areas to focus on • Track our progress and read out to executives Page 7 Operational heat map FY’14 Q2 FY’14 Q1 FY’14 Q1 FY’14 Q2
- 8. Take control so that you can start improving
- 9. • Critical stakeholders talking daily • Delivery was consistent • Peer-level accountability kept velocity up • Morale spike immediately upon seeing rapid progress Enter Presentation Name via Insert Tab > Header & Footer Page 9 Operationalization scrums R&D NOC Product Systems DBA
- 10. You don’t need a revolution; you need to introduce an evolutionary capability
- 11. Principles of Continuous Delivery • If something is painful or difficult, do it more often • Automate everything • Everyone is responsible for releasing and operating • Quality must be built in Enter Presentation Name via Insert Tab > Header & Footer Page 11 Cultivate a new mindset A product owner could request that the current development version of the application be delivered to production at a moment's notice — and nobody would bat an eyelid, let alone panic. -Martin Fowler
- 12. Celebrate the small wins; they add up to accomplish big things
- 13. Enter Presentation Name via Insert Tab > Header & Footer Page 13 Introducing tools
- 14. Tools can drive culture, which, in turn, can drive better tools
- 15. The ALM Team • Catalog of standard offerings • Pulled by users, on-demand and self service • Implementation details not required for normal use • ALM team spends their time building services and coaches users What’s in the catalog? What’s coming? Enter Presentation Name via Insert Tab > Header & Footer Page 15 • Project management • SCM • Build • Deploy • Code analysis • Functional test • Performance test • Security scan • Infrastructure provisioning • Monitoring
- 16. Create an internal service provider to offer cross-cutting concerns as-a-service
- 17. • Continue to break down silos • Increase self-service operations to deliver full environments as a service • Increase empathy that R&D has for IT by sharing accountability for systems configuration management • Expanding the use of operationalization scrums and creating a definition of done for infrastructure Enter Presentation Name via Insert Tab > Header & Footer Page 17 What next?
- 18. • Take control so that you can start improving • You don’t need a revolution; you need to introduce an evolutionary capability • Celebrate the small wins; they add up to accomplish big things • Tools can drive culture, which, in turn, can drive better tools • Create an internal service provider to offer cross-cutting concerns as-a-service Enter Presentation Name via Insert Tab > Header & Footer Page 18 In Summary
- 19. Thank you! Brian.Garofola@cdk.com @BrianGarofola
Hinweis der Redaktion
- -thank InformationWeek, Dr. Dobbs, and CA -the agenda has a title of “Ensuring Predictability and Quality in Unpredictable Business Environments: A DevOps Success Story” I won’t say we’re a success story so I’m going to tell you about our journey
- About CDK -you’ve probably never heard of CDK Global and that’s because up until a couple of months ago, a company by that name didn’t exist -we were part of ADP, who you most likely know as the company that prints your pay checks. We were a division of ADP called Dealer Services, providing technology to the automotive industry -on October 1st, we spun off as an independent, publicly traded company trading on the NASDAQ as CDK -offer technology solutions for consumer research and marketing, dealership visits and sales, negotiation and F&I, and service and parts -ranges from digital advertising to accounting and payroll to telephony systems -if you’ve ever bought a car, chances are you interacted with our systems either from your home while you researched vehicles online or in the dealership when you did your financing About ME -I’ve been in the technology industry for about 14 years -I attended college Marist College in Poughkeepsie, NY where I studied a wide variety of technologies from programming with Java and C++, to virtualization and databases -out of school I worked for a couple of years with a small company as a software engineer working on IP video products -a friend and I decided we should work for ourselves because how hard could it be running a company – it turns out that as much as customers love your products and services, they don’t like to pay you very quickly – lesson learned -one of our biggest customers was a large dealership group, Van Tuyl Auto Group, who was recently in the news since they were acquired by Warren Buffett -this was my segue into the automotive technology space and a few years later, we shut our shop down, sold out to ADP and were part of the Dealer Services family -I bounced around various roles in engineering until landing in my current role about a year and a half back -throughout my tenure, I witnessed many challenges with respect to getting product from an idea into the customers hands and started focusing a lot of effort on solving this so called ‘DevOps’ problem
- -before we get rolling, I’d like to try to level set on a definition DevOps to be used for this presentation -I had my own definition but did some research on how the industry experts were attempting to define it -of course, we all agree with DevOps Borats’ definition [read it] -[read the others] -but ultimately I landed on this [next slide]
- -the key words in here are sharing and common -I spent some time at DevOps days Chicago and many of the talks kept coming back to the essence of DevOps – empathy, not sympathy, empathy -yes, I feel bad for the IT person on pager duty getting woken up 3 times in the night, but I can’t have empathy for him until I’ve been on pager duty myself -then we can start to better understand the problems so we can try to fix them
- -release cycles – major releases were once per year with some patches released during the year – every release was chaos lasting multiple days -DESKING’s OLD RELEASE PROCESS: the deployment process was slow and error prone – one team cherry picked files they knew they changed out of source control, zipped those up, and shipped them since they no source of truth for what should be in production anymore -servers were lovingly handcrafted by artisans, each one with it’s own unique touch -getting those servers provisioned into a working environment took well over a month -the development team “shipped to QA” and it literally meant they sent the code to another team to perform manual testing -R&D reports up to the CTO and IT reports up to the CIO and they had different goals, incentives, and priorities -we have a lot of products on a lot of platforms from .NET, Java, and PHP to Oracle, SQL Server, PostgreSQL, and MySQL -WE ARE A TEXT BOOK DEVOPS problem
- -there is only one answer – start where you are -we booked a room for 2 hours – me, my boss, and the 2 key IT leaders and talked about operational perspectives were important to us – this includes things like an appropriate performance test environments, configuration management, automated deployment and release processes, documented run books and monitoring, disaster recovery, sox compliance and other security perspectives -we then rated our products against each of these perspectives – we were brutally honest – and there was a lot of red!! -we showed it to our CIO and CTO and after their outrage they us the 3 questions from the executive playbook – you know those questions right? How did we get here? What are we doing about it? How do we keep it from happening again? -we picked a select few items to address over the next quarter and then got back together again to see where we stood -each quarter we were improving but the most important thing is that we took control of the situation and started improving while we shared the accountability -which leads me to my first piece of advice
- -so we built on our newly found partnership but found that while we had alignment at the leadership level, the key players who we performing the work were still working in silos -DAMON EDWARDS quote: and there is no greater enemy to throughput and quality than silos. Almost every “devops” problem I’ve seen can be traced back to silos -so what could we do? -we kept thinking, “if only there was a construct that promoted cross-functional teams working towards a shared goal” -at this point, R&D was well on it’s way down the road of Agile but the concepts had not been leveraged in IT so we decided to give it a shot -a cross-functional team meeting daily, tackling action items, and being accountable to each other
- -at this point we were getting better at collaborating, but much of our focus was on IT -as an R&D leader, I knew we needed into improve our flow of business value so I started evangelizing the principles of continuous delivery -LEAD MANAGER STORY-we started small, introducing unit testing, TDD, and continuous builds and for some teams that grew to continuous quality inspection, automated acceptance tests, automated deployments, and more -it’s easy to look at the task as daunting, but we had a team start with a single unit test in an application that was over 75k lines of code, slowly but surely coverage increased while new features were also being created -that team now has over 50% code coverage and is able to release to production every other week – all seeded from 1 passing unit test -DESKING STORY -refer back to cherry picking releases -1 release from June 2012 – June 2013 -3 releases from June 2013 – Dec 2013 -10 releases so far in 2014 -over 2x volume on Black Friday 2014 with no hiccups whereas Black Friday 2014 the system crashed multiple times -all through lots of iterative changes along with a new mindset – a DBA tuning an index here, an engineer optimizing some code there, increased test coverage, etc -so my advice…
- -I heard a great quote at DevOpsDay Chicago, I believe it’s credited to Nathan Harvey, “the only devops tool is the person who calls themselves the director of devops” -But obviously tools have value if wielded correctly -the story I just told about the team that increased code coverage was able to do that using a tool called SonarQube which provides continuous quality inspection – every day they looked at the tool and saw coverage was going up and to the right, motivating them to do more -another team never considered automated deployment until we introduced CA Release Automation to them – after automating their application deployment, they became one of the first groups to adopt LiquiBase for database change management -tools can sometimes be seen as a 4-letter word in the DevOps community, but they play a vital role in it
- -one of the dangers of tools is tool sprawl, and we had it -Jenkins servers under desk, TFS and ClearCase, a little GitHub enterprise here, some Subversion sprinkled there -=