In this session you’ll learn best practices for managing your application lifecycle with these tools with a particular focus on development speed and release agility. Through interactive demonstrations, this session shows you how to get an application running using AWS Elastic Beanstalk, CloudFormation and CodeDeploy. You will also see how advanced techniques such as blue/green deployment, AMI baking, customer resources and in-place deployment reduce deployment friction and rapid change in your environment.

1. Managing Your Application Lifecycle on AWS
Continuous Integration and Deployment
Adrian White, Solutions Architect
Amazon Web Services

2. Business
101 Technical
201 Technical
301 Technical
401 Technical
Session Grading

3. What are we covering today?
• Consistency through the development, test and
release lifecycle
• Improve quality over time
• Increase velocity of application change
• AWS deployment and management approaches
• What does deployment look like in the future?

4. Task tracker
Continuous integration /
Automated testing /
Release management
Artifact repository
Source code
management
Deployment service
AWS OpsWorks
Elastic Beanstalk
AWS
CloudFormation
AWS
CodeDeploy
Local
development
Application lifecycle management workflow

5. AWS Code and Deployment Tools
MonitorProvisionDeployTestBuildCode
Elastic Beanstalk
OpsWorks
Cloud
Watch
Cloud
Formation
Code
Deploy
Code
Commit
Code
Pipeline

6. Local development challenges
• Source code management design
• “But it works on my machine”
• Portable development environments
• Distributed teams work on tasks in parallel

7. Container image factories
(for consistency)
…

8. Source code management features
• Fast and easy branching
• Pull requests for distributed development workflows
• Code review
• Audit, logging, security

9. Feature branching, Gitflow and Pull requests
Inspired by https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow/
Master Develop Release Feature Hotfix

10. CodeCommit
• Private Git on AWS
• Massive scaled version controlled projects
• High service availability and resiliency
• Encrypted at rest
• Pay as you go pricing
• Import from SVN, Git, Microsoft TFS
• Use IAM to control access to repositories

11. CodeCommit workflow
git clone
create repository Create repository
Receive clone request
Sync local / remote repos
modify local files
git add / commit / push
Receive push request
Update remote repo
git pull
Receive push request
Update remote repo
List repos, list branches
Display response
Receive requests
Send responses

12. Why use a release automation service?
Automate
workflow
Release
quickly
Ensure
quality

13. CodePipeline
• Customizable workflow engine
• Integrate with partner and custom systems
• Visual editor and status
Continuous delivery and release automation, just like Amazon
Build
1) Build
2) Unit test
1) Deploy
2) UI test
Source Beta Production
1) Deploy
2) Load test
Gamma
1) Deploy region1
2) Deploy region2
3) Deploy region3

14. How do you ship application changes?
• Deployment approaches
– In place vs discrete stacks
• Where is state in the system?
– Stateless vs stateful application components
• Frequency and speed of change

15. Why use a deployment service?
Automate
deployments
Manage
complexity
Avoid
downtime

16. Shipping artifacts to existing environments

17. Shipping artifacts to existing environments

18. Shipping artifacts to existing environments

19. Shipping artifacts to existing environments

20. Shipping artifacts to existing environments

21. Shipping artifacts to existing environments

22. CodeDeploy helps with this!

23. CodeDeploy
• Scale from 1 instance to thousands
• Deploy without downtime
• Centralize deployment control and monitoring
Staging
CodeDeployv1, v2, v3
Production
Dev
Coordinate automated deployments, just like Amazon
Application
revisions
Deployment groups

24. Step 1: Package your application (with an AppSpec
file)
version: 0.0
os: linux
files:
- source: chef/
destination: /etc/chef/codedeploy
- source: target/hello.war
destination: /var/lib/tomcat6/webapps
hooks:
ApplicationStop:
- location: deploy_hooks/stop-tomcat.sh
BeforeInstall:
- location: deploy_hooks/install-chef.sh
AfterInstall:
- location: deploy_hooks/librarian-install.sh
ApplicationStart:
- location: deploy_hooks/chef-solo.sh
ValidateService:
- location: deploy_hooks/verify_service.sh

25. Step 1: Package your application (with an AppSpec
file)
version: 0.0
os: linux
files:
- source: chef/
destination: /etc/chef/codedeploy
- source: target/hello.war
destination: /var/lib/tomcat6/webapps
hooks:
ApplicationStop:
- location: deploy_hooks/stop-tomcat.sh
BeforeInstall:
- location: deploy_hooks/install-chef.sh
AfterInstall:
- location: deploy_hooks/librarian-install.sh
ApplicationStart:
- location: deploy_hooks/chef-solo.sh
ValidateService:
- location: deploy_hooks/verify_service.sh

26. Step 1: Package your application (with an AppSpec
file)
version: 0.0
os: linux
files:
- source: chef/
destination: /etc/chef/codedeploy
- source: target/hello.war
destination: /var/lib/tomcat6/webapps
hooks:
ApplicationStop:
- location: deploy_hooks/stop-tomcat.sh
BeforeInstall:
- location: deploy_hooks/install-chef.sh
AfterInstall:
- location: deploy_hooks/librarian-install.sh
ApplicationStart:
- location: deploy_hooks/chef-solo.sh
ValidateService:
- location: deploy_hooks/verify_service.sh

27. Step 2: Set up your target environments
Agent Agent Agent
Staging
Agent Agent
Agent Agent
Agent
Agent
Production
Deployment groupDeployment group
Group instances by:
• Auto Scaling group
• Amazon EC2 tag
• On-premises tag

28. Step 3: Deploy!
aws deploy create-deployment
--application-name MyApp
--deployment-group-name TargetGroup
--s3-location bucket=MyBucket,key=MyApp.zip
AWS CLI & SDKs
AWS Console
CI / CD Partners
GitHub

29. Deployment config – Choose speed
v2 v1 v1 v1 v1 v1 v1 v1
v2 v2 v2 v2 v1 v1 v1 v1
v2 v2 v2 v2 v2 v2 v2 v2
One-at-a-time
Half-at-a-time
All-at-once

30. Rolling update – Deploy without downtime
v1v1 v1
Load Balancer

31. Rolling update – Deploy without downtime
v1v2 v1
Load Balancer

32. Rolling update – Deploy without downtime
v2v2 v1
Load Balancer

33. Rolling update – Deploy without downtime
v2v2v2
Load Balancer

34. Rolling update – Deploy without downtime
v2v2 v2
Load Balancer

35. Health tracking – Catch deployment problems
v2v2 v2
Load Balancer

36. Health tracking – Catch deployment problems
v3 v2 v2Stop
Load Balancer

37. Health tracking – Catch deployment problems
v2v2 v2
Load Balancer
Rollback

38. Product integrations

39. Demo: CodeDeploy & Atlassian Bamboo

40. Shipping artifacts to new environments
• What if we can quickly and easily build new environments
every time?
• CloudFormation
– Deploying AMIs
– Deploying containers
• CodeDeploy to manage discrete application versions

41. Shipping artifacts – Discrete environments
Tasks for AWS
AWS CloudFormation
Discrete stacks

42. Shipping artifacts – Immutability via containers
ECS cluster
Amazon EC2
Container Service
Immutable infrastructure with Docker and
Amazon EC2 Container Service (ECS)
Tasks for AWS

43. Shipping artifacts – Immutability via containers
Tasks for AWS
ECS now supports ELB, health checks,
scale-up and scale-down and update
management
ECS Task
Definition
ECS cluster with tasks

44. ECS Task Definitions
{
"containerDefinitions": [
{
"name": "wordpress",
"links": [
"mysql"
],
"image": "wordpress",
"essential": true,
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
],
"memory": 500,
"cpu": 10
},
{
"environment": [
{
"name": "MYSQL_ROOT_PASSWORD",
"value": "password"
}
],
"name": "mysql",
"image": "mysql",
"cpu": 10,
"memory": 500,
"essential": true
}
],
"family": "hello_world"
}

45. Build Engineering @ Atlassian
Providing CI / CD as a Service
Peter Leschev, Senior Team Lead
Build Engineering

46. SCALING THE BUILD GRID
INTRODUCTION
Introduction
AWS ADVANTAGES
FUTURE STATE

47. Build platform & services
used internally within
Atlassian to
build, test & deliver
software

48. Developers expect a
reliable infrastructure & fast
CI feedback

49. Build Engineering today @ Atlassian
• 10 Bamboo Servers
• maven.atlassian.com / 6 Nexus instances
• Monitoring - opsview / graphite / statsd
• 1000 build agents (own hardware + EC2 instances)
• Include SCM clients, JDKs, JVM build tools, databases,
headless browser testing, python builds, NodeJS, installers
& more
• Maintain 20 AMIs of various build configurations
• Nexus proxies

50. 3 years ago:
21kBuilds per month

51. Last month:
127kBuilds per month

52. Build Engineering today @ Atlassian

54. JIRA alone has
47kAutomated tests

55. SCALING THE BUILD GRID
INTRODUCTION
Scaling the Build Grid
AWS ADVANTAGES
FUTURE STATE

56. Build Agents

57. Own Hardware
using KVM guests
+
EC2 instances

59. Differences between KVM
guests & elastic build agents
+
the need for
Functional Parity

60. 2 Problems

61. Problem 1:
Network connectivity
differences

62. Solution:
AWS Direct Connect +
VPC enabled EC2
instances

63. Problem 2:
Configuration
differences between
KVM guests & elastic
build agents

64. Solution:
Removing tech debt using
https://www.packer.io/ https://www.atlassian.com/software/bamboohttps://puppetlabs.com/ https://cukes.info/

65. SCALING THE BUILD GRID
INTRODUCTION
AWS Advantages
AWS ADVANTAGES
FUTURE STATE

66. AWS Advantages
• Cost always decreasing, available instance types always expanding
• Able to handle the peaks / spikes in the build grid
• Difficult to predict future demand & provision hardware in time
• Usage characteristics change over time, existing hardware becomes sub-optimal
• Able to perform experiments / change instance types with ease
• Move faster as an organisation with an API & Credit Card
• Much faster than seeking approval for hardware, lead time to installation & being available
• We’ve gone to the extent of ‘Do Not Resuscitate’ our existing hardware
https://aws.amazon.com/blogs/aws/category/price-reduction/

67. Spot instances

68. Cheap!

69. Slower startup times

70. Not always available

71. Price Volatility

72. Expect Failure
Embrace It

73. Prebake at AMI burn
time rather than at
EC2 instance
startup time

74. Faster startup time

75. More reliable
instance startup
(+ retry on failure!)

76. Pre-caching of
git repositories,
docker images

77. RequestLimitExceeded.
The maximum request rate
permitted by the Amazon EC2
APIs has been exceeded for
your account.

78. Handle failure,
retry with backoff
& recover

79. We currently do not have
sufficient m3.large capacity
in the Availability Zone you
requested.

80. Multiple-AZ support
in Bamboo 5.8

81. “we are experiencing
increased error rates
and latencies for the
EC2 APIs”

82. SCALING THE BUILD GRID
INTRODUCTION
Future State
AWS ADVANTAGES
FUTURE STATE

83. Moving all Build
Eng Infra to AWS

84. EC2 to ECS for
build agents

85. Come help us!

86. Thank you!
PETER LESCHEV • TEAM LEAD • ATLASSIAN • @PETER LESCHEV

87. Futures: what are you really deploying?
• Application bundles?
• AMIs?
• Docker containers?
• Lambda functions?
• Microservices architectures…

88. • Service-Oriented
Architecture (SOA)
• Everything gets a
service interface
• Primitives
• “Microservices”

90. Thousands of teams +
Microservices architecture +
Multiple environments +
Continuous delivery

91. = 50 million deployments a year
Thousands of teams +
Microservices architecture +
Multiple environments +
Continuous delivery

92. Where to go next…
• AWS Training & Certification
– http://aws.amazon.com/training/
• Deployment and Management at AWS
– http://aws.amazon.com/application-management/
• Code Management and Deployment
– https://aws.amazon.com/blogs/aws/code-management-and-
deployment/
• Amazon EC2 Container Service
– http://aws.amazon.com/ecs/