DevOps, Continuous Integration & Deployment on AWS discusses practices for software development on AWS including DevOps, continuous integration, continuous delivery, and continuous deployment. It provides an overview of AWS services that can be used at different stages of the software development lifecycle such as CodeCommit for source control, CodePipeline for release automation, and CodeDeploy for deployment. National Novel Writing Month (NaNoWriMo) maintains its websites and services on AWS to support its annual writing challenge. It migrated to AWS to improve uptime and scalability. Its future goals include porting older sites to Rails, using Amazon SES for email, load balancing with ELB, implementing auto scaling, and using services like CodeDeploy, SNS
BOOK Call Girls in (Dwarka) CALL | 8377087607 Delhi Escorts Services
Â
DevOps, CI/CD on AWS: Automate Deployments & Scale for Peak Traffic
1. Š2015, Amazon Web Services, Inc. or its affiliates. All rights reserved
DevOps, Continuous Integration &
Deployment on AWS
Leo Zhadanovsky, Senior Solutions Architect, AWS
@leozh
Dave Beck, Technical Director, National Novel Writing Month
@DaveB_NaNoWriMo
3. What is DevOps?
⢠ DevOps is the practice of operations and
development engineers participating together in
the entire service lifecycle, from design through
the development process to production support
Âť
- theagileadmin.com
5. What is Continuous Integration?
⢠Changes to code automatically deployed to
mainline branch
â After passing unit and mock tests
⢠Makes changes to code, and deployments
iterative, not monolithic
⢠Bugs are detected quickly
⢠Helps automate deployments
⢠Allows rapid development and deployment
11. Source Control
Private Git repositories hosted on Amazon S3
⢠Full Git compatibility (use with existing tools)
⢠All the benefits of the cloud
(scalable, durable, reliable, low pay as you go
pricing)
⢠No size limits on repositories (store binary files)
⢠Online code tools with browse, edit, diff
10/13/14 11
13. Continuous Delivery
Customizable release automation, with integrated build and test
⢠Model and visualize custom release workflow
(source ď¨ build ď¨ beta ď¨ gamma ď¨ prod)
⢠Automate builds, tests, and deployments
⢠Enforce custom rules, approvals, and gates
⢠Integrate with third-party and custom tools
10/13/14 13
14.
15. Deployment
Coordinate software updates to fleets of EC2 instances
⢠Rolling updates for no downtime
⢠Deployment health checks and easy rollback
⢠Auto Scaling integration
⢠Works with any application
⢠Reuse existing setup tools
(Bash, Powershell, Chef, PuppetâŚ)
10/13/14 15
41. 11.6s
Mean time
between
deployments
(weekday)
1,079
Max number of
deployments in a
single hour
10,000
Mean number of
hosts
simultaneously
receiving a
deployment
30,000
Max number of
hosts
simultaneously
receiving a
deployment
DEPLOYMENTS AT
AMAZON.COM
(in 2011)
52. At a glance
What is NaNoWriMo?
⢠National
⢠Novel
⢠Writing
⢠Month
We are a community of novelists.
Every November, hundreds of
thousands of writers gather virtually
and in-person to write 50,000 words
each. (The Great Gatsby, by
comparison, is 47,094 words.)
Last year, 180,000 started new
novels, and 40,000 finished.
53. It all started in 1999
Our founder, Chris Baty,
challenged 20 over-caffeinated
friends to write 50,000 words each
in a single month.
Six of them finished.
55. NaNoWriMo now
NaNoWriMo Main: Ruby 1.9.3, Rails 3.2.2, MySQL 5.6
Camp NaNoWriMo: Ruby 1.9.3, Rails 3.2.2, MySQL 5.6
The Young Writers Program: Drupal 5, MySQL 5.6
NaNoWriMo Store: Drupal 5, MySQL 5.6
56. November in detail
⢠Nov 1: ~500k sessions as people begin
⢠All month: ~50k forum posts per day
⢠Nov 30: Last chance to post word counts
57. Annual Site Crash
⢠Every year from 2003 to 2011 the site would crash on
November 1.
⢠In 2012 we migrated the sites to Amazon Web Services.
⢠The result: No more outages!
⢠Naturally, this is how our participants reactedâŚ
59. Keeping NaNoWriMo Online
⢠Proxy servers (Varnish) on Debian 7.5 EC2
instances
⢠Ruby on Rails on EC2 (Nginx)
⢠Worker servers running Resque on EC2
⢠Support servers running Resque, Redis,
Sphinx
⢠MySQL on RDS instances
⢠Domains and subdomains on Route 53
⢠Image assets on S3 instances via EBS
⢠Deployment via Capistrano, Puppet
61. Next Steps: Goals
ďź To increase uptime for all services
ďź To make server management less time-consuming
ďź To reliably and quickly respond to user-driven events
ďź To reduce EC2 costs through autoscaling
ďź To create a faster and more dynamic email system
ďź To improve site search
62. Next Steps: Goodbye Drupal
What: Porting Drupal sites to Ruby 2.2, Rails 4.2
Why: The old Drupal-based sites are slow and use more
than their share of server resources.
63. Next Steps: Emailing via Amazon SES
What: Migrating outgoing emails from PHPlist/Sendmail to
SES
Why: Event-driven emails on the sites are backgrounded to
support servers, which has proved slow and unreliable.
Mass emails rely on an old and very sluggish application
which has limited layout tools and metrics.
64. Next Steps: Load Balancing
What: Converting proxy servers to Elastic Load Balancers
Why: We will benefit from better AWS dashboard tools,
metrics, and notifications. It is part of our autoscaling
strategy.
65. Next Steps: CodeDeploy
What: Using CodeDeploy for deployment process
Why: Currently deployment relies on many custom shell
scripts and tends to be unreliable. We intend to move to
stateless instance spin-ups as part of our autoscaling
strategy.
66. Next Steps: Autoscaling
What: Implement an Auto Scaling process
Why: In the past weâve had to predict our server needs
ahead of time. If we underestimated, the sites would
crash; if we overestimated, we werenât using our small
budget effectively. And the planning process consumed
too much staff time.
67. Next Steps: Amazon SNS Notifications
What: Increased use of Amazon SNS service
Why: Last year SNS event messaging helped us respond
to emerging crises very quickly. With our expanded AWS
presence, we will link event notifications to all of the
services we use.
68. Next Steps: On the Horizon
What: Amazon CloudSearch; AWS Lambda
Why: We are interested in easy-to-manage alternatives to
Sphinx â CloudSearch is one option. Lambda looks
intriguing for a variety of possible applications.
69. Potential future architecture
Internet
Load Balancer
ELB EC2 instances
Code servers
EC2 instances
Databases
RDS DB instances
Search
CloudSearch instance
Outgoing email
Amazon SES service