This document discusses the evolution of continuous integration (CI) and its relationship with Agile development practices and automation. It provides historical context on how CI tools emerged in the early 2000s and were initially used by small Agile teams. It then discusses how CI and Agile expanded into larger enterprise organizations between 2005-2010, bringing new challenges around governance, testing, and operations. Finally, it describes how CI and Agile have influenced the emergence of DevOps practices aimed at optimizing the entire software delivery pipeline from development to production.

1. From Continuous Integration to DevOps
The Co-Evolution of Agile & Automation
Sanjeev Sharma
IBM Worldwide Lead – DevOps Technical Sales
Executive IT Specialist, IBM Software Group
sanjeev.sharma@us.ibm.com
@sd_architect
http://bit.ly/sdarchitect
© 2013 IBM Corporation

2. Manifesto for Agile Software Development
We are uncovering better ways of developing software by doing it and helping
others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left
more.
© 2013 IBM Corporation

3. Continuous Integration (CI)
“Continuous Integration is a software development practice where members of a
team integrate their work frequently; usually each person integrates at least daily
- leading to multiple integrations per day. Each integration is verified by an
automated build (including test) to detect integration errors as quickly as
possible.”
- Martin Fowler
© 2013 IBM Corporation

4. A Historical Perspective
© 2013 IBM Corporation

5. CI is about discovering problems early
Practice: Manual Continuous Integration
– the practice of frequently integrating my work with the team
– At least daily commit
Tools: Automated Continuous Integration
– Auto Tests (including build) on Commit
– Send feedback to the team automatically
© 2013 IBM Corporation

6. CI improves productivity and quality
• 90% rise in LOC output/programmer
when performing builds at least daily
• 36% reduction in defect rate
when integration/regression testing at each code check-in
“Trade-offs between Productivity and Quality in Selecting Software Development Practices”, IEEE Software,
Sept-Oct 2003
© 2013 IBM Corporation

7. 2001: First “CI” tools
Cruisecontrol
Anthill User Manual
BUILDFORGE
© 2013 IBM Corporation

8. Agile & CI in 2001-2004
Agile (XP)
• Small teams
• Developer-centric
• High discipline
• Co-located
Continuous Integration
•
•
•
•
Build focused
Developer testing
Open source
Lava lamps
© 2013 IBM Corporation

9. 2005-2010: To the Enterprise!
© 2013 IBM Corporation

10. Enterprise Agile… Governance?
“…clients
told me of their plans to use Scrum on a $5 million project with
400 developers in three countries…
“Its not the engineering practices that will trip us up, continuous integration,
test first, refactoring – these things are understood. Its governance that’s
going to be the problem.”
major analyst firm
© 2013 IBM Corporation

11. Agile and CI Pressures on Test and Ops
• Ability to create software with business value quickens
demanding more releases
• CI means more builds are available to test
• New Questions:
– How do we test more rapidly and release more rapidly without
unacceptable risk?
– Does a Release process that worked when we had 18 month cycles
work for a 1 month cycle?
© 2013 IBM Corporation

12. Agile & CD in 2006-2011
Agile (Scrum)
• Small, medium & large
teams
• Cross functional
• Standardized
• Distributed
Continuous Integration
Continuous Delivery
• Self-service
• Builds, tests &
deployments
• Enterprise rollouts
• Shared infrastructure
© 2013 IBM Corporation

13. Agile has conquered App-Dev
© 2013 IBM Corporation

14. Challenges to accelerate software delivery
Customers
41%
experience development
delays
Business Owners
34%
experience deployment
delays
Development/Test
45%
experience
production
delays
14
© 2013 IBM Corporation
Operations/Production
4-6
weeks
to deliver
code
changes

15. Valuing “Working Software” means working in
Production
© 2013 IBM Corporation

16. DevOps: Agile Reaches Ops
*image fromIBM Corporation
© 2013 Dev2Ops.org

17. DevOps is…
• Agile & Lean applied to the whole software delivery chain,
not just developers
– BizDevQaSecReleaseOps
• Driven by efficiency and consistency
• Optimizing software delivery end-to-end
© 2013 IBM Corporation

18. DevOps is Disruptive
Dev
Ops
• Very High Tempo

Slower Tempo
• Can rebuild database / app
from scratch

Incremental updates to Database
and App
–
– No need for Rollbacks
• Audit is nice to have

Audit Critical
–
– Security, traceability,
separation of duties.
• New Environments are
common

Rollbacks are huge
Security, traceability, separation of
duties.
New environments are
rare
© 2013 IBM Corporation

19. Deming Cycle
•
•
•
William Deming – American statistician
Major influencer of Japanese manufacturing and
business
Famous for Plan-Do-Check-Act cycle (Deming
Cycle)
–
•
19
I like “Adjust” versus “Act”
PDCA cycles found in DevOps
© 2013 IBM Corporation

20. DevOps approach: Apply Lean principles to software innovation and
delivery to create a continuous feedback loop with customers
1
1. Get ideas into production fast
2. Get people to use it
3. Get feedback
2
Line-ofbusiness
3
Customer
Adopt DevOps approach to continuously
manage changes, obtain feedback and ,
deliver changes to users
Eliminate any
activity that is not
necessary for
learning what
customers want
© 2013 IBM Corporation

21. DevOps
Enterprise capability for continuous software delivery that enables clients
to seize market opportunities and reduce time to customer feedback
DevOps Lifecycle
Customers
Business
Owners
Development/
Test
Continuous Innovation, Feedback and Improvements
Accelerate Software Delivery
Balance speed, cost, quality and risk
Reduce time to customer feedback
21
21
© 2013 IBM Corporation
Operations/
Production

22. Shared tooling is hard
build
dev
test
system
test
UAT
sign-off
staging
prod
• Traditional organizational silos can inhibit the end-to-end
infrastructure
– Development-centric tools may be a poor fit for use in the data
center
– Operations-centric tools are rarely deployed upstream into
development
© 2013 IBM Corporation

23. In Production, we don’t deploy “a build”
Ok, 12% deploy one
build at a time
© 2013 IBM Corporation

24. Need: A better model for the pipeline for Prod
build
dev
test
system
test
UAT
© 2013 IBM Corporation
sign-off
staging
prod

25. Need: A better model for the pipeline for Prod
build
dev
test
system
test
UAT
© 2013 IBM Corporation
sign-off
staging
prod

26. Use integrated pipelines with Snapshots
build
dev
test
Web
Fetch from
SCM
dev
test
system
test
Build
dev
test
system
test
Mid. Code
system
test
UAT
sign-off
Mid. Config
Config
Extract
DB
Fetch from
SCM
dev
test
3
system
test
system
test
prod
Snapshot
UAT
2
dev
test
staging
3
1
© 2013 IBM Corporation
2
Stage
Signoff
Prod

27. Agile & Automation Today
Agile
(Scrumban + DevOps)
Automation
•
•
•
•
•
• Platform as a Service
• Provision, build, test, deploy,
monitor
• Enterprise
• Shared infrastructure
Small & large teams
Business to Ops
Standardized
Distributed
RM build to Prod
(Provision -> Monitor)
© 2013 IBM Corporation

28. Read the manifesto with “In Production”
We are uncovering better ways of developing software by doing it and helping
others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left
more.
© 2013 IBM Corporation

29. Delivery Pipeline capabilities - Tools
Incrementally adopt when/if needed
Rational Focal Point
Rational Requirements Composer
IBM UrbanCode Release
Chef
SmartCloud Orchestrator
IBM PureApplication System
IBM UrbanCode Deploy
Line of
Business
Jenkins
Rational Build Forge
Rational Team Concert
Rational Quality Manager
Rational Test Workbench
Rational Test Virtualization Server
SmartCloud Application
Performance
Management
Tealeaf
© 2013 IBM Corporation

30. 30
© 2013 IBM Corporation

31. 31
© 2013 IBM Corporation

32. Related Exhibit
The exhibition booth in relation to this session is:
01
04
14
13
第１会場
第 2 会 場 第３会場
基 調 講 演 会 場
12
ソリューション・ツアー
受付
11
総合受付
10
09
08
Information Wall
07 06 05 04
DevOps
ゾーン
03 02 01
© 2013 IBM Corporation
ものづくり
ゾーン

33. Acknowledgements and disclaimers
Availability: References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries
in which IBM operates.
The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided
for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any
participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided
AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise
related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating
any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license
agreement governing the use of IBM software.
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may
have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is
intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue
growth or other results.
© Copyright IBM Corporation 2013. All rights reserved.
– U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
IBM, the IBM logo, ibm.com, Rational, the Rational logo, Telelogic, the Telelogic logo, Green Hat, the Green Hat logo, and other IBM products
and services are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or
both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these
symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may
also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and
trademark information” at www.ibm.com/legal/copytrade.shtml
Other company, product, or service names may be trademarks or service marks of others.
33
© 2013 IBM Corporation