Amazon's journey to its current modern architecture and processes provides insights for all software development leaders. To get there, Amazon focused on decomposing for agility, making critical cultural and operational changes, and creating tools for software delivery. The result was enabling developers to rapidly release and iterate software while maintaining industry-leading standards on security, reliability, and performance. Whether you're developing for a small startup or a large corporation, learning the tools for CI/CD will make your good DevOps team great.

1. Building Like Amazon
Leo Zhadanovsky Julie Lawson
With: Moderated by:
TO USE YOUR COMPUTER'S AUDIO:
When the webinar begins, you will be connected to audio
using your computer's microphone and speakers (VoIP). A
headset is recommended.
Webinar will begin:
11:00 am, PDT
TO USE YOUR TELEPHONE:
If you prefer to use your phone, you must select "Use
Telephone" after joining the webinar and call in using the
numbers below.
United States: +1 (415) 655-0060
Access Code: 224-815-615
Audio PIN: Shown after joining the webinar
--OR--
The Modern Tech Stack
Webinar Series

2. AmazonWeb Services offers a broad set of global cloud-based products to equip technology leaders to
build better and more powerful solutions.
Partnering with CTO Universe,AWS is now offering an exclusive program to our members and audience.
The program includes up to $100,000 USD of value AWS credits, free consulting sessions with AWS
SolutionsArchitect to review your environment, strategies and optimize your cost, and other resources
to help you get started on migrating to AWS.
If you are interested in learning more, please reach out to aws-cto-program@amazon.com.

3. The Modern Tech Stack
Webinar Series
Click on the Questions panel to
interact with the presenters
https://www.ctouniverse.com/frs/14319860/make-your-good-devops-team-great-with-ci-cd

4. About Leo Zhadanovsky
Leo Zhadanovsky, a Principal Solutions Architect at Amazon Web Services, spends his days (and
occasional nights!) helping customers best leverage AWS services in order to help them build
highly-available, scalable and elastic architectures to fulfill their business needs. As a speaker, Leo
has delivered talks at conferences around the world, including Re:Invent, OmniTI Surge, and
PuppetConf. Previously the Director of Systems Engineering at the Democratic National
Committee, he’s also run the DNC's on-premise and cloud infrastructure, in use by the Obama
campaign, state and local Democratic parties. This infrastructure was used to support hundreds of
millions of dollars in online donations, and withstood significant amounts of election day traffic. In
his free time, he enjoys travelling the world, going for a bike ride, and trying out new restaurants in
whatever city he finds himself in.
About Julie Lawson
Julie Lawson majored in English at Boston University. She worked at the college radio station,
WTBU, where she developed a passion for producing shows with good music and good stories. She
started her career at a small publishing house in Los Angeles and went on to become Webinar
Coordinator at Aggregage, where she produces webinars and facilitates BTS webinar functions. In
her spare time, she enjoys going to the beach, camping and reading great books.
The Modern Tech Stack
Webinar Series

5. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Powerful ecosystem of tools
CI/CDTools Monitoring &Tracing
SDKs
IDE
IDE and DevOpsToolkits CLI and ScriptingTools
Infrastructure as Code Web Apps
MobileLanguages
AWS
CodeStar
AWS
CodePipeline
AWS
CodeDeploy
AWS
CodeCommit
AWS
CodeBuild
AWS
CloudFormation
AWS Cloud Dev.
Kit (CDK)
AWS X-Ray
Amazon
CloudWatch
AWS Cloud9 AWS Elastic
Beanstalk
Amazon
Corretto AWS Amplify

6. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Internal and external customers across industries

7. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Customers tell us they need to:
Build organizations, processes and systems that enable rapid
innovation and delivery of high performing digital solutions for
their customers.
Gain ‘how-to’ knowledge, grounded in real-world practice.

8. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Architecture, software delivery, and operations
By Amazon’s senior technical executives and engineers
Real-world practices with detailed explanations
Content available for free on the website
The Amazon Builders’ Library
How Amazon builds and operates software

9. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Architecture Software Delivery & Operations
Curated articles to help you
modernize and transform your
business in the cloud
Deep dive insights into practices
that have worked well at Amazon
and those that haven’t
A Point of Reference: ‘First hand’ Lessons &Techniques

10. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
The Amazon Builders’ Library

11. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Lessons learned
at Amazon for
developing and
delivering
software
Automate everything
Decompose for agility
(microservices, 2-pizza teams)
Standardized tools
Infrastructure as code
Belts and suspenders
(governance, templates)

12. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon’s Journey

13. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Our mission
Our task was to improve:
• Innovation
• Speed
• Agility
• Safety
What we did:
• Decomposed for agility
• Cultural and operational shift
• Created tools for software
delivery

14. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Just starting out
This is how many web architectures
started out, and it’s how Amazon
started too…
There any many bottlenecks, and
scaling of the web server was an
immediate factor
Table 3Table 1Table 1
User
Remote Facility
Business
Processing
Web server
Internet
Table 2 Table 3
Database

15. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
User
Remote Facility
Business
Processing
Web Server
Internet
Table 1 Table 2 Table 3
Database
Web DB
Table 1
Web DB
Table 3
Scaling v1
In 1998 the “Distributed Computing
Manifesto” came out and we began
breaking things down into separate
components…
This was a bit better,
still not very scalable

16. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Going further
Principles
• Make units a small as possible (Primitives)
• De-couple based on scaling factors,
not functions
• Each service operates independently
“Communication is terrible!” —Jeff Bezos
• APIs (contracts) between services

17. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Impact to our development

18. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Monolith development lifecycle

19. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Monolith development lifecycle

20. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Monolith development lifecycle
MonitorBuild Test Release
MonitorBuild Test Release
MonitorBuild Test Release
MonitorBuild Test Release
MonitorBuild Test Release
MonitorBuild Test Release
This led to changes in organization

21. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Impact to our organization

22. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
“Two-pizza” teams
• Own a service
• Minimizes social constraints
(Conway’s law)
• Autonomy to make decisions
Getting (re)organized

23. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Transformation timeline
Monolithic application +
teams
Microservices
+ 2-pizza teams
2001 2002

24. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Teams Own Everything
How can one team do all of this?
• Planning
• Security
• Performance
• Scalability
• Deployment
• Operation
• Bugs
• Documentation
• Testing…

25. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Now we have…

26. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Today we have Modern Applications
Modern Applications
• Use independently scalable
microservices (serverless,
containers...)
• Connect through APIs
• Deliver updates
continuously
• Adapt quickly to change
• Scale globally
• Are fault tolerant
• Carefully manage state
and persistence
• Have security built-in

27. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Measuring success

28. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
How we think about metrics
Business metrics Operational metrics Input goals Enablement
Growth Errors Features Principal reviews
Usage Throttling Use cases Security training
Feedback Failed deployments Performance Ops training
Performance

29. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Enabling our teams through
automation

30. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Software delivery at Amazon
Dev/Test Code review Pre-production
testing
Production

31. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Software delivery at Amazon
Dev/Test
Code review Pre-production
testing
Production

32. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Dev/Test
• Works on smallest
thing possible
• Only local code branches
• Develops and tests locally
• Explicitly defines
dependencies (including
those needed for testing)
• Publishes changes
for review
Developer
(or cloud)
workstation
AWS
CodeCommit
Developer
AWS account
Code review
This part should be fast
Gating Function

33. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Continuous Delivery at Amazon
Dev/Test Code review
Pre-production
testing
Production

34. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source

35. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build

36. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build

37. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha
Automated
tests
Pre-production

38. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha beta
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
browser
tests
Pre-production

39. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
Pre-production

40. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
Pre-production Production

41. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
Pre-production Production

42. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
Pre-production Production

43. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
Pre-production Production
Synthetic
Monitoring

44. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
Pre-production Production
Synthetic
Monitoring
Synthetic
Monitoring

45. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
Pre-production Production
Synthetic
Monitoring
Synthetic
Monitoring
Synthetic
Monitoring

46. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
…
Pre-production Production
Synthetic
Monitoring
Synthetic
Monitoring
Synthetic
Monitoring

47. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Amazon Continuous Delivery: Deep Dive
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
App code
Infrastructure
as code
Configuration
OS patching
…
Pre-production Production
Synthetic
Monitoring
Synthetic
Monitoring
Synthetic
Monitoring

48. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Pipeline Blockers: time windows, pipeline policies, code coverage, code
review, security scans, dependency updates, etc.
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
App code
Infrastructure
as code
Configuration
OS patching
…
Pre-production Production
Synthetic
Monitoring
Synthetic
Monitoring
Synthetic
Monitoring

49. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Fully automated: this part should be CAREFUL
Source Build alpha beta gamma
Automated
tests
Automated
integration
tests
Automated
load/perf
tests
Automated
integration
tests
Automated
synthetic
tests
API smoke
tests
Automated
browser
tests
App code
Infrastructure
as code
Configuration
OS patching
…
Pre-production Production
Synthetic
Monitoring
Synthetic
Monitoring
Synthetic
Monitoring

50. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
The Builders’ Library is a site composed of:
• Engineering practices
• Service Patterns
• Team norms and practices
• Helps to find Answers
• Architectural Pillars
• Design principles
• Asking Questions
Well-Architected is a tool composed of:
Well-Architected +The Builders Library = Innovation Flywheel

51. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Just the beginning
Along the way we have learned a lot about writing software
That’s performant, safe, and scalable
We have had to solve some really hard problems
At massive scale
We know our way is not the only way, and many of our solutions are not fancy
But we know they work
We are long obsessed with building things to help our customers
Now we want to share the benefit of what we learned along the way

52. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.
Architecture, software delivery, and operations
By Amazon’s senior technical executives and engineers
Real-world practices with detailed explanations
Content available for free on the website
The Amazon Builders’ Library
How Amazon builds and operates software
https://aws.amazon.com/builders-library

53. The Modern Tech Stack
Webinar Series
Q&A
Julie Lawson
With: Moderated by:
Principal Solutions Architect, AWS
Linkedin page: /in/leozhadanovsky/
Twitter ID: @leozh
Website: leozh.net
Leo Zhadanovsky
Webinar Coordinator, Aggregage
Linkedin page: /in/Julie-Lawson/
Email: julie@aggregage.com
https://www.ctouniverse.com/frs/14319860/make-your-good-devops-team-great-with-ci-cd

54. © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark.