1. S U M M I T
B AH RAI N

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Breaking up the Monolith
with Containers
Paul Maddox
Principal Architect
AWS

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
About me
Paul Maddox
Principal Architect, AWS Containers
• 17 years of dev, SRE, and systems architecture background
• 4.5 years at AWS
• Developer: Go, Rust, Java, JS
• Written some parts of AWS, some non-AWS
• I talk about containers a lot
Twitter: @paulmaddox@paulmaddox

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
When and why to use microservices
How to move to microservices
Overcoming common pitfalls
Wrap up, next steps, and resources

5. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon.com used to
be a monolith
• Large executable
• Long time to compile
• Long time to deploy
• Blurred ownership
https://www.flickr.com/photos/treehouse1977/3687758531

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
2001
Decentralization of technologies and teams transformation
at Amazon: 2001-2009
2009
Monolithic
application + teams
Microservices + two pizza teams

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon.com
Retail platform architecture
circa 2009
…10 years ago.

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Pros
• Teams can develop services
independently
• Team can deploy services independently
• Teams can scale services independently
• Teams can choose the best technologies
for their service
• Teams can onboard developers faster
(simple codebase)
Why microservices?
Cons
• Complexity is not removed, it is moved
from the codebase
• New techniques/tooling for observability
and communication required
• More language/technology agnostic
tooling required

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Key indicators for microservices
• You have more than one development team
• You suffer from too much change control process
• You want to scale your organization
Key indicators against microservices
• You are a small startup and getting an MVP shipped is your priority

11. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Move functionality to microservices in chunks
Prioritize services that change frequently

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Reducing change control overhead through APIs
If you are not making a breaking changing your API contract,
… and you're not changing your service availability contract,
… then every change becomes an implementation detail
internal to your team

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Choose your flavor of compute

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Benefits of containers for microservices
• Agility
• Tooling
• Community
• Containerization of existing applications
• Isolation
• Cost efficiency

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS container services landscape
Management
Deployment, scheduling,
scaling, and management of
containerized applications
Hosting
Where the containers run
Amazon Elastic
Container Service
(Amazon ECS)
Amazon Elastic
Kubernetes Service
(Amazon EKS)
Amazon EC2 AWS Fargate
Image Registry
Container image repository
Amazon Elastic
Container Registry
(Amazon ECR)

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Managed by AWS
No Amazon EC2 Instances to provision, scale or
manage
Elastic
Scale up and down seamlessly
Pay only for what you use
Integrated
with AWS: Amazon VPC networking, Elastic Load
Balancing, IAM permissions, CloudWatch, and more
AWS Fargate

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS Fargate: Security built in
Network Isolation
Use AWS Security Groups to secure and isolate your
container traffic
Kernel Level Isolation
AWS ensures that your containers are isolated from others
at the hypervisor level – they do not share an OS kernel
Access & Auditing
Use IAM to control access to your container resources, as
well as roles to assign permissions to containerized apps
AWS CloudTrail provides detailed auditing information
Compliance
Suitable for regulated workloads
Certified ISO, PCI, SOC 1, SOC 2, and SOC 3

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS Fargate customers
”We don't want to babysit
any clusters. That has
nothing to do with us.”
Shimon Tolts
CTO, DATREE
“We moved to Fargate because we
need the ability to scale quickly up
from baseline and get fine-grained
network control without having to
manage our own infrastructure.”
Product Hunt

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
migrated ~850 applications running in ~5,000 containers
to Fargate to reduce the undifferentiated heavy lifting
that came with managing Kubernetes

21. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Infrastructure Automation

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Consistency = trust = less manual process overhead
Trust is the key to reducing manual approvals and analysis paralysis.
Your organization needs to trust that you can catch bugs before
production and, in the case of a problem, recover quickly and safely.
Achieve this with:
• Consistency between test and production environments
• The correct amount of unit, integration, and end-to-end testing
• Safe zero-downtime deployment techniques (e.g. canary)

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Infrastructure automation with AWS CDK

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Test and release automation
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-cd-pipeline.html

28. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Common microservices questions
How do I control traffic routing?
• Service discovery
• Weighted routing
How can I make my services more reliable?
• Retry/back-off policies
• Connection pooling
• Circuit breakers
How do I observe my services?
• Rich protocol-aware metrics (e.g. P99 latency, HTTP response codes)
• Centralized logging
How do I secure my services?
• Only this service can talk to that service
• Encryption of traffic in transit
• Authentication of service-to-service communication

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Common microservices questions
How do I control traffic routing?
• Service discovery
• Weighted routing
How can I make my services more reliable?
• Retry/back-off policies
• Connection pooling
• Circuit breakers
How do I observe my services?
• Rich protocol-aware metrics (e.g. P99 latency, HTTP response codes)
• Centralized logging
How do I secure my services?
• Only this service can talk to that service
• Encryption of traffic in transit
• Authentication of service-to-service communication

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
History
3-tierarchitectures
NetflixHystrix
Nagios
Cacti
ServiceOrientedArchitectures(SOA)
NewRelic,AppDynamicsboth
foundedDatadogfounded
1995 2001 2002 2002 2008
AmazonCloudWatchlaunched
2009 2010 2011
Infrastructure
Monitoring
APM
Language Specific
Observability &
Failure Handling
JRugged
2013 2013
AirBNBSmartStack
LinkerD
2016
Istio
2016
AWSAppMesh
2019
Language and technology
agnostic service meshes for:
- Observability
- Routing
- Failure Handling
- Security

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Rust
Database
DB
Database
Rust
GoNode.is
Java
Node.is
Node.is
Infrastructure team
Service
teams

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Containers
Database
DB
Database
Containers
λContainers
VMs
AWS
Lambda
Managed
Service
Infrastructure team
Service
teams

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Option 1: Library/SDK
Java
Scala
Node.js
Python
C++
Django
.NET
GO
…
Languages
Application code changes
Consistency across services
SDK maintenance

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Option 2: Side-car proxy
Proxy runs
as a container
Task or Pod
External traffic
Application
Container
Proxy
Monitoring
Routing
Discovery
Deployment

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
OSS project
Wide community support, numerous integrations
Stable and production-proven
“Graduated Project” in Cloud Native Computing Foundation
Started at Lyft in 2016
App Mesh uses Envoy proxy

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
App Mesh control plane
Control
Plane
Proxy Configuration
Communication flows from proxy to control plane
Uses Envoy xDS protocol over GRPC
https://github.com/envoyproxy/data-plane-api/blob/master/xds_protocol.rst

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
App Mesh data plane
Control
Plane

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
App Mesh uses a managed control plane

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Advanced routing controls
• Weighted routing
• HTTP path based routing
• HTTP header based routing
• Automatic retry policies (per HTTP response code)
Examples
Route products to my products microservice
Send 99% of traffic to my old version, and 1% to my new version
If a 'beta-user' HTTP cookie is set, route to my beta version
If a downstream service has a 5XX error, transparently retry every 15s, a maximum of 3 times

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Envoy protocol level observability (layer 7)
StatsD
sends metrics to a StatsD
server
Logging
generates access logs
Integrations with
Datadog, Alcide, HashiCorp, Sysdig, Signalfx, Spotinst, Tetrate,
Neuvector, Weaveworks, Twistlock, Wavefront by VMware, Aqua
AWS X-Ray
forwards traces to AWS
Prometheus
Exporter
listens on tcp/9090

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Example App Mesh dashboard (mesh-wide)

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Example App Mesh dashboard (per-service)

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
A language and technology agnostic approach to microservices
AWS App Mesh is language agnostic
• No application modifications required
• App Mesh doesn't care what language your application is written in
AWS App Mesh is technology agnostic
• Create a single mesh across applications deployed to Amazon EC2, Amazon ECS, Amazon EKS, self-managed
Kubernetes, and in the future, AWS Lambda
There is no additional charge for using AWS App Mesh
• You pay only for the AWS resources (Amazon EC2 instances or requested Fargate CPU and memory) consumed
by the lightweight proxy that is deployed alongside your containers

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Summary
• Microservices is not the answer to all problems; it's a mechanism for scaling an organization
• Move to microservices by chipping away at the monolith piece by piece, prioritizing the areas
that change the most, and creating microservices for any new features/functionality
• Moving to microservices requires technology and language agnostic tooling
• AWS Container Services (Amazon ECS/Amazon EKS/Fargate) make running microservices
easier
• AWS App Mesh solves a lot of the common challenges for operating microservices at scale

46. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@paulmaddox

47. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.