Slides from the facilitated group discussion at the September 2016 Sydney Atlassian DevOps meet up. Group event page: https://www.meetup.com/sydney-atlassian-devops-group/events/233965163/
The links from the final slide for additional reading include:
http://martinfowler.com/bliki/BlueGreenDeployment.html
https://www.thoughtworks.com/insights/blog/implementing-blue-green-deployments-aws
https://botleg.com/stories/blue-green-deployment-with-docker/
http://dan.bravender.net/2014/8/24/Simple_0-Downtime_Blue_Green_Deployments.html
https://marketplace.atlassian.com/plugins/org.gaptap.bamboo.cloudfoundry.cloudfoundry-plugin/server/overview
https://www.youtube.com/watch?v=aX54mhZbN58
http://fbrnc.net/blog/2016/05/green-blue-deployments-with-aws-lambda-and-cloudformation
3. Purpose of the session
During the July 2016 Sydney Atlassian DevOps meetup there was a request from group members
to discuss blue-green deployments using Bamboo and Bitbucket.
We’re here to discuss blue-green deployment, sharing knowledge on what can work well and
discuss strategies for automating blue-green deployments.
This session will focus on a subset of the Atlassian tools however, we will discuss other tools too.
The session should:
provide an overview of how to manage blue-green deployment with Bamboo
encourage you to get started with your own automated blue-green deployments
4. Reminder about this session
We may have people of all levels of experience in this session.
The initial slides should align us with a common understanding of
blue-green deployment so we can explore the topic as a group.
When we begin the wider discussion please feel free to share your
knowledge and experience with the group and ask questions too.
6. Blue-green deployment overview
Blue-green deployment is a strategy for reducing risk when making
changes which are due to be promoted into the production
environment.
Successful implementation should result in a reduction of risk and
friction for all involved in the deployment process.
So….let’s talk about what that looks like.
7. Overview
Prior to promoting changes to a production environment, a separate
“parallel” stack of resources is created.
This can allow near seamless switching between the stacks if
behavior is in line with expectations.
9. Ok, that sounds expensive….is it?
Potentially!
We’re talking about creating a clone of production environment resources.
It makes sense to assess whether it is the right approach for your SDLC.
Understand the cost of downtime as well as the cost of duplicating your stack.
Frictionless deployments sound great however “bill shock” is quite the opposite.
Let’s continue…
10. Overview
If behavior of the green stack is in line with expectations then the
green stack should be promoted as “blue” with close to zero
downtime.
If behavior is not in line with expectations then the switch should not
take place.
If a “rollback” is required, it should be timely and low risk.
11. Overview
If a green stack is promoted to blue in a cloud environment, the
previous blue stack should probably be decommissioned.
Otherwise there will be some very unhappy CFOs….
12. Importance of environment boundaries
Persistent data stores and message queues are likely to be considered
non-disposable and may require more complex strategy….
13. Importance of environment boundaries
Centralised data writing strategies may be required to reduce risk of corruption.
Let’s look at some common blue-green deployment patterns.
14. Blue-green deploy patterns
There is no one size fits all approach however it’s good to be aware
of 2 patterns:
1. DNS cutover
2. Direct interaction with load balancers
15. DNS cutover
Gradually filter user traffic from blue to green stack using DNS
settings configured with a short time to live (TTL).
Monitor and if necessary ”rollback” so all traffic is diverted to the
blue stack.
The key questions are, “how long does it take to switch over and
how long to rollback?”
16. Direct interaction with load balancers
Avoid potential issues with DNS TTL and propagation.
Potentially leverage an additional internal load balancer to allow
further testing.
Ok so now we are aligned on that, let’s recap what Bamboo is for.
18. Build, Test, Deploy
Build
Most front end application clients are composed from a variety of files and rely on assets such as
images, JavaScript, CSS.
A build process will typically involve creating an ”artefact”.
An “artefact” could be a .zip file containing an optimized version of the application.
It could also be one of more files which will be useful for a more complex process.
19. Build, Test, Deploy
Test
In this step the ”artefact” may be tested using one or more appropriate test methodologies.
If the tests pass then this should mean the artefact is “deployable”.
Note that there may be a number of additional steps which you may need to go through to
verify whether the artefact is actually “deployable” and once you are aware of those steps the
next step could be to automate them!
20. Build, Test, Deploy
Deploy
In this step the tested ”artefact” is deployed using an appropriate strategy.
If the deployment process fails for any reason, ideally it will be easy to “roll back” until another
attempt can be made.
If you are already familiar with these concepts then you should find it relatively easy to begin
working with Bamboo following the documentation.
24. Configuring tasks related to a build
*If checking out multiple repositories
use a specific “Checkout Directory”
* Why? Because it makes it easier to use inline scripts such as: cd example-application/stack/php-api/src/ && php composer.phar install –vvv
26. Almost anything is possible with time…
Consider leveraging build scripts create stacks plus artefacts for
deployment then use a mixture of additional deployment and build tasks.
Consider leveraging existing add-ons in the Bamboo ecosystem or making
calls from Bamboo scripts to external APIs.
There are plenty of different tools out there to choose from including
some which have specific blue-green workflows.
29. So where to from here?
Great question!
Research whether blue-green is right for your SDLC and if so, start experimenting…
Some useful links for additional reading, in particular, investigate how to use a “serverless” approach to trigger changes.
http://martinfowler.com/bliki/BlueGreenDeployment.html
https://www.thoughtworks.com/insights/blog/implementing-blue-green-deployments-aws
https://botleg.com/stories/blue-green-deployment-with-docker/
http://dan.bravender.net/2014/8/24/Simple_0-Downtime_Blue_Green_Deployments.html
https://marketplace.atlassian.com/plugins/org.gaptap.bamboo.cloudfoundry.cloudfoundry-plugin/server/overview
https://www.youtube.com/watch?v=aX54mhZbN58
http://fbrnc.net/blog/2016/05/green-blue-deployments-with-aws-lambda-and-cloudformation
30. Enjoy and share what you do….
I’m on Twitter: @dkcwd
LinkedIn: https://au.linkedin.com/in/daveclarkprofile