TJ Randall, VP of Customer Success at XebiaLabs, gives his presentation on how to express the cost of your application delivery at the DevOps Leadership Summit in Boston MA.

1. My Dad Won’t Buy Me DevOps
T.j. Randall
VP of Customer Success

2. 2
• How to express cost of your application delivery
• DevOps Project Risks
Agenda
Existing Delivery
Model
New Delivery Model

3. 3
The success of implementing DevOps
(or any other organizational initiative)
comes down to the ability to express
value of the change for other teams.
The agenda today plans to help you:
• Calculate value
• Identify risk to your investments
• Information to accelerate your work
Argument
People
Process
Tools

4. 4
• Agile / CI / CD changes are
occurring across organizations
• Unfortunately, these changes
occur:
• Within individual silos
• Each team focusing on what they
need
• Everyone is solving their
problems in their own silo
• Operations struggling to unify
into a consistent pattern
Silos, Silos Everywhere
Web
OFFSHORE
NEW
MOBILE
APP
Windows

5. 5
In order to implement a solution,
you have to understand how
much your application delivery is
really costing you.
Cost of Application Delivery

6. 6
None of these individual cost centers
can accurately calculate the cost to
get a code change from developer
into Production
Most people point to specific cost
metrics:
• Cost of DEV staff
• Cost of Operations
• Cost of Project
• Cost of outsourcing
Application Delivery Cost

7. 7
How can you expect to draw out
the cost of implementing
Continuous Delivery (CD) in a IT
Spending chart??
• Hardware
• Software
• Staff
• But that’s only part of the
picture….
Where is DevOps In Here?
TechCrunch 2011 IT Spend

8. 8
• People and Release Centric
• Application, Environment and
Release Centric
Both approaches help to describe
the people, process and
environments involved with
application delivery from
Development to Production.
Two Suggested Approaches for Calculating Cost
DEV QA STAGE PROD
Think about ALL releases over entire DEV to
PROD pipeline

9. 9
Consider:
• What defines a release?
• Is it a Saturday night outage, with all
apps included? Or is it a per-app
event?
• Think about everyone who has a
role in getting your application
running in PROD.
• DEV, QA, InfoSec, OPS, DBAs, etc
“How many applications do you
have and how many active
releases are associated with an
app at any given time?”
Prep Work
Note: For this cost exercise, we’ll use XebiaLabs tools to express the “how”

10. First Up: People and Release Centric
For this exercise, let’s think about your overall release process. This
means that we’ll think about all applications as a large group.
We’ll estimate
• Application(s)
• Releases
• Staff, and the amount of time they spend in releases
10

11. 11
Active This Year Next Year
Applications 10 20
Releases
Planned 52 90
Emergency 20 2
Failed 25 1
How many applications do you have
and how many active releases are
associated with an app at any given
time?
Consider: What defines a release?
• How do apps translate to releases?
• For example, one app can have multiple versions
which translates to many releases or one billing
app could have 15 components which can
translate to 15 releases.
Apps and Releases

12. 12
Releases (in hrs)
Planned Emergency
128 25
What is your average time per release
(time post build process)?
Release Times
FTEs
Release Manager 4
Developer 8
Build Engineer 4
Operations 6
IT Manager 4
How many FTEs who work on releases?
Note: Releases are defined as work post build process.

13. 13
Time Spent on Releases
Role Example of Release Task
% Time on
Planned
Releases
% Time on
Emergency
Releases
% Time on
Failed
Releases
Release
Manager
Moving apps and releases through release pipeline, across environments, ensuring
consistency of environments, compliance and security reporting…
60 10 30
Developers
Moving apps and releases through the release pipeline, scripting and maintaining
release tasks, fixing errors/bottlenecks…
25 15 10
Build Engineer
Coordination with other roles to drive releases to next stage, fixes, rework,
software implementation…
65 10 25
Operations Meetings, control, provisioning environments, compliance, security… 75 10 15
IT Manager
Time spent meeting, approving environments, supplying software for provisioning,
compliance, security…
55 15 15

14. 14
Staff for Releases
Average fully loaded annual cost per role?
(Use estimate 260 days working days per year, not just hourly rate, but also benefits - typically 30%)
Release Manager $110k
Developers $100k
Build Engineer $95k
Operations $85k
IT Manager $160k

15. 15
Some Results

16. Environment and Release Centric Approach
Let’s change the optic.
We’re still thinking about your entire application delivery process:
• Application(s) – all the active applications
• The environments that the application travels through
• Adding automation to address deployment effort, manual configuration and
reduction in errors
Note: for this exercise, we’ll break up releases into “Major” and “Minor”
16

17. Your Current Process
17
Description Value
Cost of IT deployment staff $125
Cost of Business/Test staff $75
Number of deployments to DEV per year 100
Number of deployments to QA/STAGE/TRAINING/PRE-PROD environments per year 132
Estimated HOURS to deploy to one of these environments: QA/STAGE/TRAINING/PRE-PROD 3
Error rate to deploy (DEV -> PREPROD) 15%
hours to fix deployment errors in DEV-PREPROD (notify, analyze, fix code, redeploy) 3
Number of MAJOR releases in PROD 5
Number of MINOR releases in PROD 10
HOURS - MAJOR PROD (total man hours) 24
HOURS - MINOR PROD (total man hours) 12
Error rate to deploy (PROD) 10%
hours to fix deployment errors in PROD (notify, analyze, fix code, redeploy) 6
Cost of unexpected PROD outage $10,000

18. Application Costs Under Current Process
18
Description Value
Current Cost for Deployments (DEV) $3,125.00
Current Cost for Deployments (QA/STAGE/TRAINING/PRE-PROD)
(includes "Error rate to deploy (DEV -> PREPROD)" value for considering failed/re-deployments)
$614,790.00
Current Cost for Deployments (PROD)
(includes "Error rate to deploy (PROD)" value for considering failed/re-deployments)
$144,009.00
Current Cost for Deployments (DEV->PROD) + 2 unexpected outages $781,924.00

19. Add in Some Automation
19
Description Value Notes
Anticipated reduction deployment effort (time) 65%
Anticipated reduction in manual configuration (resource efforts) 85%
Anticipated reduction in errors 65%
Number of deployments to DEV per year 100 Assuming same # of deployments, but no staff needed due to automation
Number of deployments to QA/STAGE/TRAINING/PRE-PROD environments per year 12 Implementing better CI practices via XLr will ensure better/less releases
Estimated HOURS to deploy to one of these environments: QA/STAGE/TRAINING/PRE-PROD 1 Decrease from current due to reduction of deployment effort (time)
Error rate to deploy (DEV -> PREPROD) 3% Decrease from Anticipated reduction in errors
hours to fix deployment errors in DEV-PREPROD (notify, analyze, fix code, redeploy) 1 Decrease due to automated deployments, and better testing in release
Number of MAJOR releases in Production 5
Number represents features/enhancement/fixes - not tied to releases b/c of
issues
Number of MINOR releases in Production 10
HOURS - MAJOR PROD 4.8 Decrease from current due to reduction of deployment effort (time)
HOURS - MINOR PROD 2.4 Decrease from current due to reduction of deployment effort (time)
Error rate to deploy (PROD) 3% Decrease from Anticipated reduction in errors
hours to fix deployment errors in PROD (notify, analyze, fix code, redeploy) 3
Cost of unexpected PROD outage $10,000

20. Now that’s cost reduction!
20
Description Before Automation With Automation
With XLd and XLr - Cost for Deployments (DEV) $3,125.00 $329.00
With XLd and XLr - Cost for Deployments (QA/STAGE/TRAINING/PRE-PROD)
(Less people, less time and less errors)
$614,790.00 $9,888.00
With XLd and XLr - Cost for Deployments (PROD)
(Less people, less time and less errors)
$144,009.00 $23,101.58
With XLd and XLr - Cost for Deployments (DEV->PROD) + 0 unexpected outages $781,924.00 $33,318.58

21. 21
Great – you’ve identified it. What next?
Take a look at your app stacks, as well as the
path that your applications take from DEV
through PROD
• What are you going to measure?
• How are you going to calculate?
More important: how can you use these
cost computations to help you with your
DevOps transformation initiative?
What Next?

22. 22
Notes from the War Desk

23. 23
Most Customers look to invest in DevOps to:
• Release Faster & Do More with the Same Staff
• Keep Employees Happy
• Build More Stuff & Reserve Your People for Highest Priority Work
• Have Fewer Meetings
• Make Your Tools Work Together
• Audit Reports
• Reduce Failures
Typical Customer Goals

24. We All Want This
24

25. 25
But we get a lot of this….

26. Executives echo a similar sentiment
“We’re making changes, but they don’t seem to be working as we hoped.
What is the problem?”
So…. how do you know if there are problems with your transformation
initiative?
26

27. 27
Warning signs:
• DEV using tools for delivery of application
that aren’t used in PROD
• Automated in DEV, manual in PRE-PROD
• DEV isn’t providing what other teams need
• “Check the Jenkins log to see what’s in the release”
• DEV provides packaged code, with
instructions
• “Pick up the Zip file in this folder…”
How to resolve:
• Defined paths to PROD for DEV
teams to choose from
DEV | ops

28. 28
Warning signs:
• OPS implementing tools that DEV won’t
use.
• OPS has their own scripting to deploy
• “For this team we do ‘x’, for this team we do ‘y’”
• Manual-intensive process for meeting all
Operational requirements.
How to resolve:
• Meeting: Dev Manager, Ops
Manager, CIO
dev| OPS

29. DevOps Credo
“Better to have them inside the tent pissing out than outside pissing in”
-- Lyndon B. Johnson
29

30. Bring Everyone to the Table
30
How do we help customers solve this problem?
• Schedule a 2-3 hour mandatory meeting
• Everyone involved with delivery of application from DEV to PROD
• For each team, review:
• What they need
• What is blocking
You now have your backlog, to solve your application delivery problem

31. DevOps Advisory Board
31
Do you have a DAB team?
I see a lot of customers with “Architecture
Boards” for reviewing software only.
You need to formalize a DEV-to-PROD
team, focusing on:
• People
• Process
• Tools
Example: Developer wants to implement
new provisioning tool.