Despite the best of intentions, we sometimes find ourselves working on a team of size one. Groups shrink for many reasons: attrition, mergers and acquisitions, transfers, and financial distress. It's never comfortable being a Single Point of Failure, but how can you survive this state of non-redundancy? Are there any benefits to being a team of "me, myself, and I", or is it all a pit of despair? What kind of red flags should you be on the lookout for? And, most importantly, what compelling leverage can you try to use to encourage team growth back to a reasonable size?
At DevOps Days Silicon Valley 2015, I shared a series of unfortunate events that led to my current status: the Human SPoF; I also discussed some of the tactics I've used to survive. Automation, tools, and code-as-infrastructure are a force multiplier applied correctly, allowing one engineer to do the work of many. However, these wonders come with a price tag. I also covered some strategies to grow a team, and ways to maintain sanity while keeping the lights blinking and the disks spinning in a 24x7 real-time environment with over 2000 servers.
5. EHLO jon
• Principal DevOps Engineer, Oracle Data Cloud
• Connoisseur of Fountain Pens
• Amateur Miata Mechanic
6. Which way to the datacenter?!
Image by Dito Milan, GotBlueMilk Photography
7. EHLO jon
• Principal DevOps Engineer, Oracle Data Cloud
• Connoisseur of Fountain Pens
• Amateur Miata Mechanic
• Briefly a load balancer
8. EHLO jon
• Principal DevOps Engineer, Oracle Data Cloud
• Connoisseur of Fountain Pens
• Amateur Miata Mechanic
• Briefly a load balancer
• More on that later
9. EHLO jon
• Principal DevOps Engineer, Oracle Data Cloud
• Connoisseur of Fountain Pens
• Amateur Miata Mechanic
• Briefly a load balancer
• More on that later
• Single Point of Failure
10. Poll: How many of you are now,
or have previously, worked in
DevOps groups of size==1?
(Specifically where the group was, at one time, size > 1,
and shrank to 1)
11. Once upon a time…
“Now, children, come on over here. I'm going to tell you a bedtime story.
Are you sitting comfortably? Then I'll begin.”
— John Wood, War Games (1983)
12. Once upon a time…
• 11/2012 - 3 engineers, 450 hosts
13. Once upon a time…
• 11/2012 - 3 engineers, 450 hosts
• 06/2013 - 2 engineers, 800 hosts
14. Once upon a time…
• 11/2012 - 3 engineers, 450 hosts
• 06/2013 - 2 engineers, 800 hosts
• 9/2013 - 1 eng, 1000 hosts
23. "Single Point of Failure" by Charles Féval - http://commons.wikimedia.org/wiki/File:SPOF.png.
Licensed under CC BY-SA 2.5 via Commons - https://commons.wikimedia.org/wiki/File:Single_Point_of_Failure.png
26. Punctum Singulari Deficiendi
• We’re always on the lookout for SPoF’s
• Hardware
• Software
• We often forget the wetware!
• Domain knowledge, environmental quirks
44. Downsides
• Just because you are physically able to do it by
yourself, doesn’t mean you should.
• Tools and Automation have created breathing
room, not an excuse to not hire more people
56. Growing Your Team
• Ask for headcount. Come prepared with data.
• Ticket counts, emergencies, interrupts.
• Estimated vs actual time to complete projects
57.
58. Growing Your Team
• Ask for headcount. Come prepared with data.
• Ticket counts, emergencies, interrupts.
• Estimated vs actual time to complete projects
• Delegation and Enablement
• MUST be people you trust
59. Growing Your Team
• Ask for headcount. Come prepared with data.
• Ticket counts, emergencies, interrupts.
• Estimated vs actual time to complete projects
• Delegation and Enablement
• MUST be people you trust
• Enlistment and Poaching