The presentation describes how bitbrains is transforming from a traditional organisation towards a DevOps organisation while keeping the business running at the same time.
This presentation was given at the "Devops and the enterprise" meetup in Amsterdam on Jan 8th, 2014.
Corporate Profile 47Billion Information Technology
From Dev and Ops to DevOps - reconfiguring the plane in flight.
1. From Dev and Ops to DevOps
“reconfiguring the plane while flying”
Mike Wessling
Lead Nerd @Bitbrains
2. Who is Bitbrains
• Bitbrains provides managed hosting for medium
to large enterprises on our own shared platform.
– Mainly for financials
– Often HPC platforms and other odd ducks.
• From Day Zero to the Bitter End™
– We get involved as early as possible in projects.
– Prefer iterative design approach.
• Start building early
• Think, build, test, review, repeat.
• 35 bitbrainers.
3. Old Organization
Fairly traditional along life cycle steps:
• Project engineering (customer+platform)
–
–
–
–
–
Presales consultancy
Proof-of-Concepts
Design
Implementation
Big Changes
• Operations (customer+platform)
–
–
–
–
Maintaining status Quo
Changes
Maintenance
User requests
• Separated by the big Ops Hand-over
– 47 checks long. Some checkboxes are big..
4. Why Change?
•
External influences:
– Customers using agile project approaches.
•
•
–
–
–
–
•
Even the big ones.
Traditional Waterfall doesn’t deliver.
Customers want “try before you buy”,
Customers want “pay as you go”.
Applications getting bigger and more complex.
Growth!!
Internal influences:
–
–
–
–
The organization didn’t reflect how things worked,
Projects rarely end…
And never get handed-over completely
Customer like the TLC they get in the project phase.
•
•
•
Responsive and flexible.
SLAs and processes are less important
Willing to pay for it.
– 1 project = 1 engineer is vulnerable
•
Especially when running at 100%+ capacity
5. Result: Out of control in 2013
• Still delivering happy customers but
– Project engineers overstretched
• Handovers suffered.
• No time to make life easier.
– Customers stuck in project phase
– Operations struggling with
•
•
•
•
Partially handed-over customers,
Complex customers without history and experience
Customers not used to more rigorous processes
Out of date documentation.
– Internal improvements not happing
– Platform support stuck between engineering groups.
• No time to grow
• Organization can’t scale any further this way.
• Engineers working on 20 or more projects
6. So Now What??
•
•
•
•
•
•
Need to scale the teams
Need to reduce pressure
Need to get the process back in control
Need to create space to grow.
Need to get rid of the handovers
Need to spend time on internal improvements
7. Lets try this DevOps Thingy
• Seems to solve key problems
– No to hand-overs
– Deals with the ‘Never ending’ projects
– Shared responsibility between Project and Ops.
• We are already doing it
– But Implicit, unstructured & out-of-process
• In our DNA – it is how we started
– So how to scale this….
8. DevOps? Ops ok But Dev?
In our enterprisey world:
• Read “Dev” as the team
”Specifying, designing and
building the service.”
– Not just the “code-monkeys”
– “Coding the stack”
• Read “Ops” as the team “Running
and delivering the service”
9. 1st step: Creating Time and Space
• Nothing is going to change otherwise
– Customer projects are always priority
– Customer questions and requests are second
Approach:
• Take work of the plates
–
–
–
–
Outsource well defined projects.
Verify/renegotiate target dates.. How hard are they?
Teach Sales not to say yes to all dates.
Cancel/Delay projects if needed.
10. Enter KanBan
• Start with project engineering
– Mature self-aware group
– Role model for other engineers
• Start with a good coach (!)
– External/neutral
– Group will rebel or demotivated at some point
• Requires somebody who stands above the process
• KanBan will expose problems
– Be prepared to deal.
• Look for any benefit
– Being able to share the work across the engineers for example
• DON’T GET HUNG UP ON TOOLS..
– Not important.
– Distract
– Force a way of working
11. Result
• Project Engineers happier
– In control of work
– Less context switching
– Able to share the workload
• New engineer productive after 3 days.
• Project Manager happier
– Better overview
– No need to ask all the time
– Aware of the scary pile of work in the queues
• Operations eager to switch to KanBan
12. Next: Operations
• Bigger challenge
– Vast majority of work are tickets
– Any projects are individuals doing their best
– More junior group
• First Results:
– Insight:
• many tickets were parked
– Returned to queue.
– Bad stats – Service management unhappy
• Lots of little side projects
13. Operations continued
• Insight:
– Need to add long term and short term engineers
• Short term to clear the heap.
• Long term to stay on top and have room.
• Creation of Platform team
– Different beast with different focus
– Big projects and operational role. (very DevOpsy)
14. towards: Dev+Ops = DevOps
• Working towards close alignment of processes
– Same Queues
– Same Definitions of Work
• Already work can travel across teams
• Coordinated Set of priorities across teams
– Service management plays key role
– Currently 1 week window, extending to 2 and 4
weeks.
15. The Hand-over Elephant
• The big divider between Dev and Ops
• Big To-Do list.
– Big block of work after the interesting bits are
finished
• Used by Project engineering as buffer time.
• If Ops accept they are stuck with the customer
16. The Hand-over needs TO GO
But how?
• Engineers took as step back and looked.
• Hand-over must be continuous process.
– Stuff gets added and changed all the time
– Different bits of the environment are in different
stages
• Integrated in the definitions of work.
17. Switch to DevOps Mode
• Flip the engineering teams from a Life Cycle
organization to DevOps teams
– Won’t be using DevOps as name
– Teams are end-to-end responsible for a set of
environments grouped by common factor and
single board.
– Should be a small natural event.
– Number of teams: depends