At the heart of traditional Continuous Delivery is the deployment pipeline. A build is generated, promoted through several testing environments and if it passes tests and is aligns with business needs is deployed to Production. This model struggles to account for complex systems where releases involve numerous inter-related builds and/or components that don't fit neatly into the model of "builds" such as incremental content migrations, configuration changes, database schema updates, or report / ETL migrations. This presentation examines the limitations of the build promotion model, architectural approaches for adapting applications to that model, and deployment approaches that realign the release pipeline around the migration of value, rather than the migration of builds.
Watch the Webinar
http://www.urbancode.com/html/resources/webinars/Adapting_Deployment_Pipelines_to_Complex_Applications.html/
3. Presenting Today
Eric is a DevOps Evangelist where he
helps customers get the most out of
their build, deploy and release
processes.
Eric Minick
eminick@us.ibm.com
@EricMinick
Today he works with customers and
industry leaders to figure out this
DevOps thing.
4. Agenda
Introduction to Continuous Delivery
Continuous Delivery & Complex Apps
Adapting apps to CD
Adapting CD to complex apps
Q&A
8. The Build Pipeline
Perform a build
–and execute unit tests
Promote build to a test environment & test
–Repeat N times
Release
build
dev
test
system
test
UAT
sign-off
staging
prod
9. Continuous Delivery is a DevOps Strategy
Successful implementation requires assistance from
developers, operations, and others
Cooperation and coordination between developers and
operations must improve
10. Agenda
Introduction to Continuous Delivery
Continuous Delivery & Complex Apps
Adapting apps to CD
Adapting CD to complex apps
Q&A
12. Complex apps have related builds
Builds of one part of the app depend on another
A change in one “pipeline” could impact another pipeline
Tests cross-cut builds pipelines
13. Multiple tier apps
Different tiers, different teams, different builds
JPetStore Application
How do we align the changes?
Apache
SIT
WEB
Tomcat
MID
DB
JPetStore
Content
PROD
WEB
JPetStore
WAR
JPetStore
DB
MID
DB
16. Build pipelines in coupled systems
build
build
build
build
build
build
build
build
build
build
build
build
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
dev
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
system
test
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
UAT
sign-off
staging
prod
17. Some pieces aren’t built
Databases
Content
Reports / ETL
dev
test
system
test
UAT
sign-off
staging
prod
build
dev
test
system
test
UAT
sign-off
staging
prod
build
Infrastructure
build
dev
test
system
test
UAT
sign-off
staging
prod
build
dev
test
system
test
UAT
sign-off
staging
prod
build
dev
test
system
test
UAT
sign-off
staging
prod
build
dev
test
system
test
UAT
sign-off
staging
prod
18. The Build Pipeline fails to:
X
Account for deployment time dependencies
X
Model things that aren’t built
X
Deal with incremental updates
20. Agenda
Introduction to Continuous Delivery
Continuous Delivery & Complex Apps
Adapting apps to CD
Adapting CD to complex apps
Q&A
21. Adapting apps to CD: Principals
1. Everything is a “build”
2. Builds are promoted independent of other builds
22. So we need great architecture
photo credit: http://www.flickr.com/photos/ishmaelo/256431712/
23. Build of Build Pattern
Challenges: My “mega build” is big, and is always fully
deployed. My components don’t know if they went to Prod
Comp.
dev
Build
test
Comp.
dev
Build
test
Comp.
dev
Build
test
Mega
Build
system
test
UAT
sign-off
staging
prod
24. A Build of Builds
Comp.
Build
test
Comp.
dev
Build
.
dev
test
Comp.
dev
Build
test
Mega
Build
system
test
UAT
sign-off
staging
prod
25. Enforce Backwards Compatibility
Build and immediately deploy to integration testing
If integration tests fail, the build is rejected and the old build
of that component is redeployed to integration testing
26. Enforce Backwards Compatibility
Build and immediately deploy to integration testing
If integration tests fail, the build is rejected and the old build
of that component is redeployed to integration testing
Challenges: Good integration tests, extra engineering to
support new and old versions, etc.
27. Database Expand / Contract
Goal: Backwards compatible, zero downtime
database deployments.
Never remove objects old / active users of the
database need. Only add new objects. Once all
clients are using the new objects, remove the
old.
See: http://exortech.com/blog/2009/02/01/weekly-release-blog-11-zerodowntime-database-deployment/
28. Database Expand / Contract
Goal: Backwards compatible, zero downtime database
deployments.
Never remove objects old / active users of the database
need. Only add new objects. Once all clients are using the
new objects, remove the old.
Challenges: a significant and not always easy change to how
organizations develop DB updates.
See: http://exortech.com/blog/2009/02/01/weekly-release-blog-11-zerodowntime-database-deployment/
29. Agenda
Introduction to Continuous Delivery
Continuous Delivery & Complex Apps
Adapting apps to CD
Adapting CD to complex apps
Q&A
30. Adapting CD to our Apps
Account for deployment time dependencies
Model things that aren’t built
Deal with incremental updates
31. Use the “Build of Builds” model as a start
Comp.
dev
Build
test
Comp.
dev
Build
test
Comp.
dev
Build
test
Mega
Build
system
test
UAT
sign-off
staging
prod
32. Shift to “Release Sets” or “Snapshots”
We don’t need a new build
– we need a name for a collection of builds.
Delay the creation of these until integration tests pass, and
create based on the successful integration tests
build
dev
test
system
test
build
dev
test
system
test
build
dev
test
system
test
Snapshots at “Application”
or “System” level.
UAT
Signoff
Stagin
g
Prod
33. Don’t require “build”
Extracts from existing systems, artifact repos, or source
control are OK to get deployable version.
Build
dev
test
system
test
Snapshots at “Application”
or “System” level.
Config
Extract
dev
test
system
test
Fetch from
SCM
dev
test
system
test
UAT
Signoff
Stagin
g
Prod
34. Support multiple incremental moves
Incremental requires:
–Multiple versions of a component in snapshots
–Awareness when tracking what is where
–Order awareness when performing rollbacks.
Creating a Snapshot
Component Versions / Builds
Snapshot
Web
1
2
3
3
Mid. Code
1
2
3
2
Mid. Config
1
2
3
3
DB
1
2
3
1
2
35. Pipeline with Snapshots
Web
Mid.
Code
Fetch from
SCM
Build
dev
test
system
test
dev
test
system
test
Snapshot
3
UAT
2
Mid. Config
DB
Config
Extract
dev
test
system
test
Fetch from
SCM
dev
test
system
test
3
1
2
Stage
Signoff
Prod
36. In story form
A change to a component, creates a new version (often by
doing a build).
37. In story form
A change to a component, creates a new version (often by
doing a build).
The new version is vetted, and then tested in an integration
environment.
38. In story form
A change to a component, creates a new version (often by
doing a build).
The new version is vetted, and then tested in an integration
environment.
When the integrated system passes tests, a snapshot of all
the component versions in the system is created.
39. In story form
A change to a component, creates a new version (often by
doing a build).
The new version is vetted, and then tested in an integration
environment.
When the integrated system passes tests, a snapshot of all
the component versions in the system is created.
Snapshot deployments don’t redeploy unchanged
components
40. In story form
A change to a component, creates a new version (often by
doing a build).
The new version is vetted, and then tested in an integration
environment.
When the integrated system passes tests, a snapshot of all
the component versions in the system is created.
Snapshot deployments don’t redeploy unchanged
components
The snapshot is promoted & released
41. In Summary
Today, continuous delivery on complex systems is hard to
coordinate.
Two options
1.
Strict development standards to force our systems into the build
promotion model
2.
A shift towards snapshot deployments that accommodate projects
“as they are”
43. Yes, we sell products for this
IBM UrbanCode Deploy
–Application Deployment Automation
IBM UrbanCode Release
–Coordination across many applications
Small batch sizes:Reduce riskLower the “waste” of unreleased featuresNatural extension of the “test your builds” mentality of CIAddress the deployment bottleneck in Operations.