Mark Mzyk
Engineering Manager with Chef
Find more by Mark Mzyk: https://speakerdeck.com/mmzyk
All Things Open
October 26-27, 2016
Raleigh, North Carolina
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Â
DevOps for Managers
1. DevOps For Managers*
*And Managers at â€
Get a feel for the room:
How many managers?
How many ICs (individual contributors)?
How many leads (straddle IC and management)?
This is take 1 of this talk - there are many possible variations.
2. Who Am I?
âą Mark Mzyk
âą Engineering Manager
at Chef
âą Organizer:
Triangle DevOps
DevOpsDays Raleigh
Why listen to me?
Engineering Manager for a year at Chef
Dev for 4 previous years at Chef, 9 years in the industry total
5 years organizing Triangle DevOps
Helped organized and emceed 1st DevOpsDays Raleigh
3. âVinny, My CousinVinny
âWhat is a DevOp?â
Lead oïŹ talking about DevOps
Then present examples of what we do at Chef from a managerâs perspective that ïŹt into DevOps -
which is a higher level view of engineering than an IC (individual contributor) might view things
4. Development +
Operations =
DevOps
Itâs simple - combine development and operations and you have DevOps, right?
Leads to the idea of the DevOps team - this isnât bad, but itâs not the view I prefer
5. âDevOps is a cultural movement that changes how
individuals think about their work, values the diversity of
work done, supports intentional processes that accelerate
the rate by which businesses realize value, and measures
the effect of social and technical change. It is a way of
thinking and a way of working that enables individuals and
organizations to develop and maintain sustainable work
practices. It is a cultural framework for sharing stories and
developing empathy, enabling people and teams to practice
their crafts in effective and lasting ways.â
- Chapter 2,What is Devops?, Effective DevOps
Itâs not so simple.
DeïŹnition from EïŹective DevOps.
Also wonât ïŹnd a spelled out deïŹnition in The DevOps Handbook (that I could ïŹnd on skimming)
Itâs complicated, itâs culture - and this is why we therefore often focus on the tools
Tools are visible and interplay with culture, but they are not culture
(Also, vendors can sell tools - I work for one)
7. We watch someone else do it
We say what we hope is the right
incantation
If all goes right âŠ
8. We get dragons! (Hopefully not as scary though).
Do we even know why we got a dragon?
And most of the time, we donât get a dragon - sometimes it blows up in our face, but often it is just a dud.
9. Blueprint for DevOps
EïŹective Devops and The DevOps Handbook are two books that I think lay out the best blueprint for achieving a learning focused, DevOps culture.
Just because they give you a blueprint doesnât mean youâll be able to follow it exactly or that itâll be easy.
11. Context Matters
Your company culture is diïŹerent from Chefâs.
What I describe might be helpful, but youâll have to ïŹgure out how it applies to you.
What we think of as successful DevOps companies - NetïŹix, Etsy, etc.
They donât know what works either - but they have a learning culture, keep learning.
The world changes, so we all have to keep learning
12. Kaizen - Continuous Improvement
Kaikaku - Radical Change
Types of Change
From Toyota, Lean Thinking
Most people have heard of kaizen, less so kaikaku
We often hear and think of doing kaizen, but sometimes you need kaikaku
Letâs talk about an example of kaikaku at Chef.
13. Conwayâs Law
Any organization that designs a
system (deïŹned broadly) will
produce a design whose structure
is a copy of the organization's
communication structure.
http://melconway.com/Home/Conways_Law.html
15. Fixed Teams
Chef Server
Analytics
Delivery
Example of how teams at Chef used to be - ïŹxed to a product.
Chef then tended to create features for each product - whether it was needed or not.
Chef shipped a lot of software - but it wasnât moving the needle on the business.
16. Flexible Teams
Chef Server Automate Habitat
Kaikaku - Switched to ïŹexible teams (feature teams)
Enables us to focus on the products and features that need focus
Might have products that arenât being actively worked for a time
Aligned Business and Product - shipped software that clearly was having an impact
(Also, product names have shifted/changes as business model evolves)
17. Issues
âą In a model with rotating teams, how does an
engineer build expertise?
âą Emphasizing product alignment led to
deferment to product, loss of some of
engineeringâs voice
Teams rotated often - but that meant engineers sometimes lacked stability, felt they couldnât go deep in an area before shifting away
With the emphasis on feature teams, shipping became focus and emphasis shifted away some from quality, wasnât clear when or how technical debt should be
addressed
Also - attrition did happen. Sometimes you have to be okay that people will leave over change.
18. Kaizen
âą Shift focus from implementing a feature to
achieving an outcome
âą Let teams live longer
These are kaizen steps currently in progress
Teams focus on an outcome (Automate Adoption) instead of a feature (GitHub integration)
Team can integrate in the problem space until a suïŹcient solution is found
Let the teams live longer, but still be willing to switch up things when a team isnât working or someone needs a change
Do not intent to go back to ïŹxed teams
19. EmbraceVariability
a.k.a
Learn to Live with
Ambiguity
Your world wonât stay the same - so you have to learn how to live with variability and ambiguity
This will vary based on your circumstances - startups will tend toward more variability, enterprises less so (most of the time)
20. â Rands (Michael Lopp)
âProcess is documentation
of culture and values.â
From: http://randsinrepose.com/archives/the-process-myth/
At some point youâll realize youâll need process
Process helps control variability
Aside: We dislike process when weâve lost sight of the value it was put in place for
If you canât remember the why for a process, remove it or change it
21. No Process
Clearly DeïŹned Process
This is the path it will probably take to ïŹnd the right process. Donât be afraid to change.
Everyone lives in a diïŹerent comfort zone -
Some people operate easily with no process and ïŹnd a way forward.
Others need a clearly deïŹned process to feel comfortable.
Learn where your peers and reports live.
If you are at one extreme, know the other might be your blind spot.
How can you leverage your peers who are in a diïŹerent place so you end up in a good place as an organization?
22. DeïŹne your processes like dirt paths - see what works, try out diïŹerent things, change while they are easy - let them wind a bit.
Only when they have been successful for a while should you path them and make then more solid
Picture Credits:
âPathâ by Tim Green https://ïŹic.kr/p/6TM1w4 https://creativecommons.org/licenses/by/2.0/ No changes made
âPathâ by Allen Watkin https://ïŹic.kr/p/4bmtAD https://creativecommons.org/licenses/by-sa/2.0/ No changes made
23. Where to ask for help?
This is a story of winding process at Chef
When a customer facing person needs help from engineering where do they ask?
Observed that they often asked for help in engineering channels, but might go unanswered or sit for a while. What if they needed immediate help?
Set up #eng-escalations channel for immediate help during business hours.
Engineering managers and principal engineers watch room for any activity, mount immediate response
For after hours, have a deïŹne pager duty escalation to page an engineering manager - this was easier to put in place than the #eng-escalations channel, because we had
experience here
24. HBR Article - https://hbr.org/1991/05/teaching-smart-people-how-to-learn
To achieve a learning and DevOps culture you will have to combat this
In both yourself and those you manage and work with
Weâre all smart - single-loop learning (problem solving) comes easy to us
Double-loop learning (reïŹection on ourselves) is hard
Read this article - at least twice.
Read it once, reïŹect on it, then come back and read it again days later.
25. Make Failure Safe
If your people are afraid to fail, your org wonât learn, it wonât improve.
The only way to avoid failure is keep the status quo - but this will result in the long term failure of the business as it doesnât respond to the change around it.
Without this, nothing else in this presentation will matter.
26. Googleâs research into what makes a good team: #1 item - Psychological safety
Without that, nothing else matters. It underpins all else.
If you donât have safety, you donât speak up, you lose trust
https://rework.withgoogle.com/blog/ïŹve-keys-to-a-successful-google-team/
27. It takes many actions to
build trust, but only one lie
to destroy it.
We know this to be true.
This is why establishing safety is so hard - itâs a continual task.
This is the challenge of being a manager and establishing a DevOps culture.
Your words and actions are endlessly interpreted by everyone around you - and they donât have the same context you do.
28. Your Actions Establish or
Destroy Safety
Bottom line. How you approach the world sets this up.
29. Blameless Post Mortem
One way to establish safety is blameless post mortem
Recognize we operate in a system and assume positive intent - everyone does the best they can with the information they have - so where did the system fail and how
do we improve?
This is a screenshot of the Chef post mortem template
John Allspaw as spoken and written at length on blameless post mortem
https://codeascraft.com/2012/05/22/blameless-postmortems/
30. Small Actions Matter
âą One on Ones
âą How you respond to requests
âą How you treat outcomes on your team
These are where a true learning culture is built.
Most of these will be seen by one person, or only a few people, but it shapes their perceptions of what you think is important - and if they have safety
It is an ongoing conversation that evolves over time.
Tell the Engineer and Elm story if there is time - unexpected request, but meet halfway, explored it
32. ThankYou
Keep the conversation going,
through conversation we learn and grow.
Twitter: @mzyk83
Email: mm@chef.io
Slacks: @mm
is hiring
Some Slacks I hang out in:
Triangle Devs, Randâs Leadership, Chefâs Community, eng-managers, DevOpsDays organizerâs