At the heart of traditional Continuous Delivery is the deployment pipeline. A build is generated, promoted through several testing environments and if it passes tests and is aligns with business needs is deployed to Production. This model struggles to account for complex systems where releases involve numerous inter-related builds and/or components that don't fit neatly into the model of "builds" such as incremental content migrations, configuration changes, database schema updates, or report / ETL migrations. This presentation examines the limitations of the build promotion model, architectural approaches for adapting applications to that model, and deployment approaches that realign the release pipeline around the migration of value, rather than the migration of builds. Watch the Webinar http://www.urbancode.com/html/resources/webinars/Adapting_Deployment_Pipelines_to_Complex_Applications.html/

1. Adapting Deployment Pipelines for
Complex Applications
Eric Minick,
IBM UrbanCode Technical Evangelist
eminick@us.ibm.com
@ericminick
© 2013 IBM Corporation
© 2013 IBM Corporation

2. Acknowledgements and Disclaimers:
Availability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all
countries in which IBM operates.
The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are
provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or
advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this
presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages
arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is
intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering
the terms and conditions of the applicable license agreement governing the use of IBM software.
All customer examples described are presented as illustrations of how those customers have used IBM products and the results they
may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these
materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific
sales, revenue growth or other results.
© Copyright IBM Corporation 2013. All rights reserved.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.
IBM, the IBM logo, ibm.com,WebSphere, Rational, and IBM Mobile Enterrise are trademarks or registered trademarks of International
Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked
on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law
trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law
trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at
www.ibm.com/legal/copytrade.shtml
Other company, product, or service names may be trademarks or service marks of others.
© 2013 IBM Corporation

3. Presenting Today
Eric is a DevOps Evangelist where he
helps customers get the most out of
their build, deploy and release
processes.
Eric Minick
eminick@us.ibm.com
@EricMinick
Today he works with customers and
industry leaders to figure out this
DevOps thing.

4. Agenda
 Introduction to Continuous Delivery
 Continuous Delivery & Complex Apps
 Adapting apps to CD
 Adapting CD to complex apps
 Q&A

5. What is Continuous Delivery?

6. Why?

7. “Normal” CD
build
dev
test
system
test
UAT
sign-off
staging
prod

8. The Build Pipeline
 Perform a build
–and execute unit tests
 Promote build to a test environment & test
–Repeat N times
 Release
build
dev
test
system
test
UAT
sign-off
staging
prod

9. Continuous Delivery is a DevOps Strategy
 Successful implementation requires assistance from
developers, operations, and others
 Cooperation and coordination between developers and
operations must improve

10. Agenda
 Introduction to Continuous Delivery
 Continuous Delivery & Complex Apps
 Adapting apps to CD
 Adapting CD to complex apps
 Q&A

11. The Hard Part is Coordination
Image from wisc.edu

12. Complex apps have related builds
 Builds of one part of the app depend on another
 A change in one “pipeline” could impact another pipeline
 Tests cross-cut builds pipelines

13. Multiple tier apps
 Different tiers, different teams, different builds
JPetStore Application
 How do we align the changes?
Apache
SIT
WEB
Tomcat
MID
DB
JPetStore
Content
PROD
WEB
JPetStore
WAR
JPetStore
DB
MID
DB

14. Modern architectures make it worse
Image from ischool.tv

15. Prod deployments aren’t of one build*
*Except these

16. Build pipelines in coupled systems
build
build
build
build
build
build
build
build
build
build
build
build
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod

17. Some pieces aren’t built
Databases
Content
Reports / ETL
dev
test
system
test
UAT
sign-off
staging
prod
build
dev
test
system
test
UAT
sign-off
staging
prod
build
Infrastructure
build
dev
test
system
test
UAT
sign-off
staging
prod
build
dev
test
system
test
UAT
sign-off
staging
prod
build
dev
test
system
test
UAT
sign-off
staging
prod
build
dev
test
system
test
UAT
sign-off
staging
prod

18. The Build Pipeline fails to:
X
Account for deployment time dependencies
X
Model things that aren’t built
X
Deal with incremental updates

19. Now what?
Work hard to salvage build pipelines
Or
Use a different model

20. Agenda
 Introduction to Continuous Delivery
 Continuous Delivery & Complex Apps
 Adapting apps to CD
 Adapting CD to complex apps
 Q&A

21. Adapting apps to CD: Principals
1. Everything is a “build”
2. Builds are promoted independent of other builds

22. So we need great architecture
photo credit: http://www.flickr.com/photos/ishmaelo/256431712/

23. Build of Build Pattern
Challenges: My “mega build” is big, and is always fully
deployed. My components don’t know if they went to Prod
Comp.
dev
Build
test
Comp.
dev
Build
test
Comp.
dev
Build
test
Mega
Build
system
test
UAT
sign-off
staging
prod

24. A Build of Builds
Comp.
Build
test
Comp.
dev
Build
.
dev
test
Comp.
dev
Build
test
Mega
Build
system
test
UAT
sign-off
staging
prod

25. Enforce Backwards Compatibility
 Build and immediately deploy to integration testing
 If integration tests fail, the build is rejected and the old build
of that component is redeployed to integration testing

26. Enforce Backwards Compatibility
 Build and immediately deploy to integration testing
 If integration tests fail, the build is rejected and the old build
of that component is redeployed to integration testing
 Challenges: Good integration tests, extra engineering to
support new and old versions, etc.

27. Database Expand / Contract
Goal: Backwards compatible, zero downtime
database deployments.
Never remove objects old / active users of the
database need. Only add new objects. Once all
clients are using the new objects, remove the
old.
See: http://exortech.com/blog/2009/02/01/weekly-release-blog-11-zerodowntime-database-deployment/

28. Database Expand / Contract
 Goal: Backwards compatible, zero downtime database
deployments.
 Never remove objects old / active users of the database
need. Only add new objects. Once all clients are using the
new objects, remove the old.
 Challenges: a significant and not always easy change to how
organizations develop DB updates.
See: http://exortech.com/blog/2009/02/01/weekly-release-blog-11-zerodowntime-database-deployment/

29. Agenda
 Introduction to Continuous Delivery
 Continuous Delivery & Complex Apps
 Adapting apps to CD
 Adapting CD to complex apps
 Q&A

30. Adapting CD to our Apps
 Account for deployment time dependencies
 Model things that aren’t built
 Deal with incremental updates

31. Use the “Build of Builds” model as a start
Comp.
dev
Build
test
Comp.
dev
Build
test
Comp.
dev
Build
test
Mega
Build
system
test
UAT
sign-off
staging
prod

32. Shift to “Release Sets” or “Snapshots”
 We don’t need a new build
– we need a name for a collection of builds.
 Delay the creation of these until integration tests pass, and
create based on the successful integration tests
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
Snapshots at “Application”
or “System” level.
UAT
Signoff
Stagin
g
Prod

33. Don’t require “build”
 Extracts from existing systems, artifact repos, or source
control are OK to get deployable version.
Build
dev
test
system
test
Snapshots at “Application”
or “System” level.
Config
Extract
dev
test
system
test
Fetch from
SCM
dev
test
system
test
UAT
Signoff
Stagin
g
Prod

34. Support multiple incremental moves
 Incremental requires:
–Multiple versions of a component in snapshots
–Awareness when tracking what is where
–Order awareness when performing rollbacks.
Creating a Snapshot
Component Versions / Builds
Snapshot
Web
1
2
3
3
Mid. Code
1
2
3
2
Mid. Config
1
2
3
3
DB
1
2
3
1
2

35. Pipeline with Snapshots
Web
Mid.
Code
Fetch from
SCM
Build
dev
test
system
test
dev
test
system
test
Snapshot
3
UAT
2
Mid. Config
DB
Config
Extract
dev
test
system
test
Fetch from
SCM
dev
test
system
test
3
1
2
Stage
Signoff
Prod

36. In story form
 A change to a component, creates a new version (often by
doing a build).

37. In story form
 A change to a component, creates a new version (often by
doing a build).
 The new version is vetted, and then tested in an integration
environment.

38. In story form
 A change to a component, creates a new version (often by
doing a build).
 The new version is vetted, and then tested in an integration
environment.
 When the integrated system passes tests, a snapshot of all
the component versions in the system is created.

39. In story form
 A change to a component, creates a new version (often by
doing a build).
 The new version is vetted, and then tested in an integration
environment.
 When the integrated system passes tests, a snapshot of all
the component versions in the system is created.
 Snapshot deployments don’t redeploy unchanged
components

40. In story form
 A change to a component, creates a new version (often by
doing a build).
 The new version is vetted, and then tested in an integration
environment.
 When the integrated system passes tests, a snapshot of all
the component versions in the system is created.
 Snapshot deployments don’t redeploy unchanged
components
 The snapshot is promoted & released

41. In Summary
 Today, continuous delivery on complex systems is hard to
coordinate.
 Two options
1.
Strict development standards to force our systems into the build
promotion model
2.
A shift towards snapshot deployments that accommodate projects
“as they are”

42. References
http://urbancode.com/resources
http://ibm.com/devops/
 Enterprise CD Maturity Model
 Death to Manual Deployments!
 Build & Deployment Automation for the Lean Economy
 ITIL Release Management and Automation
Urbancode.com/blogs/
Twitter.com/UrbanCode
Facbebook.com/UrbanCodeSoft
Slideshare.net/Urbancode

43. Yes, we sell products for this
 IBM UrbanCode Deploy
–Application Deployment Automation
 IBM UrbanCode Release
–Coordination across many applications

44. Questions?
eminick@us.ibm.com
@EricMinick

45. www.ibm.com/software/rational
© Copyright IBM Corporation 2012. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind,
express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have
the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM
software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities
referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature
availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines
Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
© 2013 IBM Corporation

46. 46