The build pipeline model of continuous delivery works great for simple projects, but can be challenging for applications with many pieces and parts. In this deck, we look at two approaches for reconciling CD and these applications. In one approach, we force the applications into a simple pipeline, in the other, the pipeline is reimagined.

1. Next Gen Continuous Delivery
From Build delivery to Value delivery
1

2. Eric’s Bio
I’m a Lead Consultant at Urbancode
where I helps customers get the
most out of their build, deploy and
release processes. I have 9 years of
automation experience throughout
Eric Minick the application life-cycle in roles as
eric@urbancode.com
a developer, test automation
engineer, and support engineer. I’ve
been at the forefront of CI & CD for
7+ years
2

3. Agenda
• Introduction to Continuous Delivery
• Continuous Delivery & Complex Apps
• Adapting apps to CD
• Adapting CD to complex apps
• Q&A
3

4. What is CD?
• An automated flow from build to “ready to
deploy to production”
• Push-button deployment to production
• The execution of many types of tests
• Cultural emphasis on constant shipability
4

5. “Normal” CD
• Expanding the CI emphasis on quality and
automation downstream
dev system
build UAT sign-off staging prod
test test
5

6. The Build Pipeline (Build > ? > Prod)
• Perform a build
– and execute unit tests
• Promote build to a test environment & test
– Repeat N times
• Release
dev system
build UAT sign-off staging prod
test test
6

7. We even have a maturity model for it
7

8. Why are we doing it?
• Small batch sizes:
– Reduce risk
– Lower the “waste” of unreleased features
• Natural extension of the “test your builds”
mentality of CI
• Address the deployment bottleneck in
Operations.
8

9. Continuous Delivery is a “DevOps” Strategy
• Successful implementation requires assistance
from developers, operations, and others
• Cooperation and coordination between
developers and operations must improve
9

10. Agenda
• Introduction to Continuous Delivery
• Continuous Delivery & Complex Apps
• Adapting apps to CD
• Adapting CD to complex apps
• Q&A
10

11. The Hard Part is Coordination
Image from wisc.edu
11

12. Complex apps have related builds
• Builds of one part of the app depend on
another
• A change in one “pipeline” could impact
another pipeline
• Tests cross-cut builds pipelines
12

13. Multiple tier apps
• Different tiers, different JPetStore Application
teams, different builds
Apache SIT
WEB
• How do align the Tomcat MID
changes? DB
JPetStore
Content PROD
WEB
JPetStore
WAR MID
DB
JPetStore
DB
13

14. Modern architectures make it worse
• Dozens of inter-related
components
• The promise of “change
only one piece” is rarely
realized in practice
Image from ischool.tv
14

15. Prod deployments aren’t of one build
15

16. Build pipelines in coupled systems
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
16

17. Some pieces aren’t built
• Databases
dev system
build UAT sign-off staging prod
test test
• Infrastructure build
dev
test
system
test
UAT sign-off staging prod
dev system
build UAT sign-off staging prod
test test
dev system
build UAT sign-off staging prod
test test
• Content build
dev
test
system
test
UAT sign-off staging prod
dev system
build UAT sign-off staging prod
test test
• Reports / ETL
17

18. CD’s Build Pipeline isn’t Perfect
• It fails to:
– Account for deployment time dependencies
– Model things that aren’t built
– Deal with incremental updates
18

19. Now what?
Work hard to salvage build pipelines
Or
Use a different model
19

20. Agenda
• Introduction to Continuous Delivery
• Continuous Delivery & Complex Apps
• Adapting apps to CD
• Adapting CD to complex apps
• Q&A
20

21. Adapting apps to CD: Principals
• Principals
– Everything is a “build”
– Builds are promoted independent of other builds
• Example techniques
– Release build of builds
– Enforce backwards compatibility
– Expand / Contact pattern for databases
21

22. A Build of Builds
• Create a mini-release process for each
component, and combine them into a big build.
• Mega build contains the binaries from the others
Comp. dev
Build test
Comp. dev Mega system
UAT sign-off staging prod
Build test Build test
Comp. dev
Build test
22

23. A Build of Builds
• Create a mini-release process for each
component, and combine them into a big build.
• Mega build contains the binaries from the others
Comp. dev
Build test
Comp. dev Mega system
UAT sign-off staging prod
Build test Build test
Comp. dev
Build test
• Challenges: My “mega build” is big, and is always
fully deployed. My components don’t know if
they went to Prod.
23

24. Enforce Backwards Compatibility
• Build and immediately deploy to integration
testing
• If integration tests fail, the build is rejected
and the old build of that component is
redeployed to integration testing
24

25. Enforce Backwards Compatibility
• Build and immediately deploy to integration
testing
• If integration tests fail, the build is rejected
and the old build of that component is
redeployed to integration testing
• Challenges: Good integration tests, extra
engineering to support new and old
versions, etc.
25

26. Database Expand / Contract
• Goal: Backwards compatible, zero downtime
database deployments.
• Never remove objects old / active users of the
database need. Only add new objects. Once all
clients are using the new objects, remove the old.
26 See: http://exortech.com/blog/2009/02/01/weekly-release-blog-11-zero-downtime-
database-deployment/

27. Database Expand / Contract
• Goal: Backwards compatible, zero downtime
database deployments.
• Never remove objects old / active users of the
database need. Only add new objects. Once all
clients are using the new objects, remove the old.
• Challenges: a significant and not always easy
change to how organizations develop DB updates.
27 See: http://exortech.com/blog/2009/02/01/weekly-release-blog-11-zero-downtime-
database-deployment/

28. Agenda
• Introduction to Continuous Delivery
• Continuous Delivery & Complex Apps
• Adapting apps to CD
• Adapting CD to complex apps
• Q&A
28

29. Adapting CD to our Apps
• Account for deployment time dependencies
• Model things that aren’t built
• Deal with incremental updates
29

30. Use the “Build of Builds” model as a start
Comp. dev
Build test
Comp. dev Mega system
UAT sign-off staging prod
Build test Build test
Comp. dev
Build test
30

31. Shift to “Release Sets” or “Snapshots”
• We don’t need a new build, we need a name
for a collection of builds.
• Delay the creation of these until integration
tests pass, and create based on the successful
integration tests
Snapshots at “Application” or
dev system
build
test test
“System” level.
dev system Sign-
build
test test
UAT Staging Prod
off
dev system
build
test test
31

32. Don’t require “build”
• Extracts from existing systems, artifact
repos, or source control are OK to get
deployable version.
dev system
Build
test test
Snapshots at “Application” or
“System” level.
Config dev system
Extract test test Sign-
UAT Staging Prod
off
Fetch from dev system
SCM test test
32

33. Support multiple incremental moves
• Incremental requires:
– Multiple versions of a component in snapshots
– Awareness when tracking what is where
– Order awareness when performing rollbacks.
33

34. Pipeline with Snapshots
Fetch from dev system
Web SCM test test
dev system
Mid. Code Build
test test
Sign-
UAT Stage Prod
off
Config dev system
Mid. Config Extract test test
Fetch from dev system
DB SCM test test
34

35. In story form
• A change to a component, creates a new version
(often by doing a build).
35

36. In story form
• A change to a component, creates a new version
(often by doing a build).
• The new version is vetted, and then tested in an
integration environment.
36

37. In story form
• A change to a component, creates a new version
(often by doing a build).
• The new version is vetted, and then tested in an
integration environment.
• When the integrated system passes tests, a
snapshot of all the component versions in the
system is created.
37

38. In story form
• A change to a component, creates a new version
(often by doing a build).
• The new version is vetted, and then tested in an
integration environment.
• When the integrated system passes tests, a
snapshot of all the component versions in the
system is created.
• Snapshot deployments don’t redeploy unchanged
components
38

39. In story form
• A change to a component, creates a new version
(often by doing a build).
• The new version is vetted, and then tested in an
integration environment.
• When the integrated system passes tests, a
snapshot of all the component versions in the
system is created.
• Snapshot deployments don’t redeploy unchanged
components
• The component version is promoted & released
39

40. In Summary
• Today, continuous delivery on complex
systems is hard to coordinate.
• Two options
1. Strict development standards to force our
systems into the build promotion model
2. A shift towards snapshot deployments that
accommodate projects “as they are”
40

41. References
http://urbancode.com/resources
• Enterprise CD Maturity Model
• Death to Manual Deployments!
• Build & Deployment Automation for the Lean
Economy
• ITIL Release Management and Automation
Urbancode.com/blogs/
Twitter.com/UrbanCodeSoft
Facbebook.com/UrbanCodeSoft
Slideshare.net/Urbancode
41

42. Yes, we sell products for this
• AnthillPro
– Build automation and build promotion
• uDeploy
– Deployment and release management
Upcoming AnthillPro + uDeploy demo:
http://web.urbancode.com/using-anthillpro-with-udeploy
42

43. Questions?
eric@urbancode.com
@EricMinick
43