Covers the evolution from monoliths to microservices, the properties of modern cloud applications and why we need service meshes. Takes a closer look at Envoy and how AWS AppMesh can provide a managed service mesh.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Developing ModernApplications in
theCloud
COBUS BERNARD
AWS SENIORTECHNICAL EVANGELIST
@cobusbernard

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Capabilitiesof a modern application
Secure Resilient Elastic
Modular Automated Interoperable

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Assess and
prioritize,
app by app
Pick path to
modernization
Lift & shift:
data center → EC2
Re-platform:
VMs → containers
Refactor:
monolith → microservices
Re-invent:
host fleets → serverless
Modern application development: ajourney withmany
paths

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tomaintaincompetitiveadvantage,
digitalbusinessesmustinnovateasrapidlyas possible
FeedbackIdeas
Experiment
Innovation
Flywheel

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Invention requires two things: the
ability to try a lot of experiments,
and not having to live with
the collateral damage of
failed experiments.“
Andy Jassy
CEO, Amazon Web Services

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Structureappsascollectionsofmicroservices
When theimpact ofchange issmall,
releasevelocitycan increase
Monolithic application
Does everything
Shared release pipeline
Rigid scaling
High impact of change
Hard to adopt new technologies
Microservices
Does one thing
Independent deployments
Independent scaling
Small impact of change
Choice of technology

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Structureappsascollectionsofmicroservices
Whentheimpact ofchange issmall,
releasevelocitycan increase
Microservices
Do one thing – Well

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Structureappsascollectionsofmicroservices
Properties of microservices
Microservices
• Independent
• Individually Deployed & Scaled
• Polyglot
• Modular - Easily Replaced
• Decentralized

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Structureappsascollectionsofmicroservices
DesignConcepts
• Use managed services
• Focus on writing your business logic
(not on maintaining infrastructure)
• Loosely coupled & event driven
• Simplify delivery and discovery
AWS Lambda
AWS Fargate

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Structureappsascollectionsofmicroservices
APIs and decoupled communications enable
automation and improves reliability
FUNCTION
API
API
MICROSERVICE
FUNCTIONEVENT
API
MICROSERVICE
EVENT
API
MICROSERVICE
APPLICATION

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Rigid Flexible
Abstractions
Easy Hard
1 System N Systems2 Systems

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
TechnicalDebtCost
Time
Absolute

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
TechnicalDebt
Time
Cost
Relative
Absolute

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Monolith
Does everything
Monoliths are OK

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Commondatacategoriesandusecases
Relational
Referential
integrity, ACID
transactions,
schema-
on-write
Lift and shift, ERP,
CRM, finance
Key-value
High
throughput, low-
latency reads
and writes,
endless scale
Real-time bidding,
shopping cart,
social, product
catalog, customer
preferences
Document
Store
documents and
quickly access
querying on any
attribute
Content
management,
personalization,
mobile
In-memory
Query by key
with
microsecond
latency
Leaderboards,
real-time analytics,
caching
Graph
Quickly and
easily create
and navigate
relationships
between
data
Fraud detection,
social networking,
recommendation
engine
Time-series
Collect, store,
and process
data sequenced
by time
IoT applications,
event tracking
Ledger
Complete,
immutable, and
verifiable history
of all changes to
application data
Systems
of record, supply
chain, health care,
registrations,
financial

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Buildwithserverlesstechnologiesasmuchaspossible
Automation and abstractionfrees you
No infrastructure to
provision or manage
Automatically scales
by unit of consumption
Pay for value
billing model
Highly available
and durable

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Usecodetomodelapplicationsandinfrastructure
Treatingeverythingas softwareincreases thespeed and
agilityof infrastructuredeployments
Design
Write application
code
Create infrastructure
templates
Create stacks Iterate

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Usecodetomodelapplicationsandinfrastructure
Infrastructure ascode
Declarative
I tell you
what I need
I tell you
what to do
Imperative

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Usecodetomodelapplicationsandinfrastructure
Infrastructure ascode goals
1. Make infrastructure changes repeatable and predictable
2. Release infrastructure changes using the same tools as code changes
3. Replicate production environment in a staging environment to enable
continuous testing

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Usecodetomodelapplicationsandinfrastructure
Model function environments withAWS
ServerlessApplication Model (SAM)
• Open source framework for building serverless
applications on AWS
• Shorthand syntax to express functions, APIs,
databases, and event source mappings
• Transforms and expands SAM syntax into AWS
CloudFormation syntax on deployment
• Supports all AWS CloudFormation resource types
https://aws.amazon.com/serverless/sam/

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Rapidlyreleasehigh-qualityfeatureswithCI/CD
TeamsthatpracticeCI/CD
shipmore code faster,and withmore confidence
Source: Puppet 2017 State of DevOps Report
5x
Lower change
failure rate
440x
Faster from
commit to deploy
46x
More frequent
deployments
44%
More time spent on
new features & code

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Rapidlyreleasehigh-qualityfeatureswithCI/CD
Monolith development lifecycle
monitorreleasetestbuild
developers
delivery pipelines
services

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
developers services
monitorreleasetestbuild
delivery pipelines
monitorreleasetestbuild
monitorreleasetestbuild
monitorreleasetestbuild
monitorreleasetestbuild
monitorreleasetestbuild
Rapidlyreleasehigh-qualityfeatureswithCI/CD
Monolithdevelopment lifecycle

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Containers have become the standard for how to ship and run your
application in the cloud

30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Containers and Docker
A container is a standard unit of software that packages up code and all its
dependencies so the application runs quickly and reliably from one
computing environment to another.1
1 https://www.docker.com/resources/what-container
Server
Operating System
Docker Engine
AppA
AppB
AppC
AppD

31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Docker Image
• Used to launch container
• Instructions documented in Dockerfile
• Merge layers into single image
• Read-only template
kernel
Base Image
Image layer
Image layer
Image layer
References
parent image
layer

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Containers and Microservices
• Do one thing, really well
• Any app, any language
• Isolated execution environment
• Test and deploy same artifact
• Faster startup
Container Container
Container Container

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Manually downloading and launching containers by hand is
inefficient and error prone

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Container orchestration

35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Buildwithserverlesstechnologiesasmuchaspossible
AWS container serviceslandscape
Management
Deployment, Scheduling,
Scaling & Management of
containerized applications
Hosting
Where the containers run
Amazon Elastic
Container Service
Amazon Elastic
Container Service
for Kubernetes
Amazon EC2AWS Fargate
Image Registry
Container Image Repository
Amazon Elastic
Container Registry

36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon ECS key components
Development cluster
Container instance Container instance
Container instance
Productioncluster
Container instance Container instance
Container instance
AmazonElastic Container Service
(AmazonECS)
Container
Container
Volume
Taskdefinition
AmazonElastic Container Registry

37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Kubectl
EKS Architecture

38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Microservicecommunication
• IP / Ports constantly changing
• TLS between services
• Metrics
• Monitoring
• Auth

39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
OSS Hystrix:
code changes required
Service Mesh:
decentral, language agnostic,
dumb endpoints
https://www.infoq.com/articles/microservices-post-kubernetes
Shift in Infrastructure Logic
ESB: clustered monolith

42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Open Source: Istio Service Mesh
Connect, secure, and observe services
• Shift in where functionality is located
• Control plane = Istio
• Data plane = set of all Envoy proxies
• Envoy proxy as sidecar in K8s pod
• Automatic or manual injection of proxy with EKS

43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data Plane (Proxy)
• Touches every packet / request
• Service discovery
• Health Checking
• Routing
• Load Balancing
• Authentication / Authorization
• Observability

44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Istio Service Mesh with Envoy Proxy

45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Envoy Proxy
• Level 7 proxy
• HTTP, HTTP/2, gRPC, AWS Dynamo DB, MongoDB
• C++11 code base , only 8 MB (statically linked)
• No language or framework dependencies
• Rquires no code changes
• Battle proved OSS, started at Lyft
• Works across compute options – also on EC2
• Envoy is not tightly coupled Istio

46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Service Mesh
But Docker / Kubernetes can do rolling updates!
Yes, but Istio sparates traffic flow
from replica deployment

47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A bath tub full of cold water ? K8s roling update
25%
1 pod at a time
… or just wetten your feet? Service Mesh
3%
Traffic routing
🛁 🛁 🛁
💦
🛀🏽🛁
🌊❄️🌊❄️🌊❄️
Fancy a Swim in the Arctic Sea ?
Blue / Green
100%
All services at once
Microservices Update Strategies

48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
User Based Routing Traffic Shifting

49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Control Plane (Istio)
• Routing information
• Policies & configuration

50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
App Mesh works across compute services
Amazon ECS
AWS Fargate
Amazon EKS
Amazon EC2
Kubernetes on EC2

51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Based on Envoy proxy
Start App Mesh from the AWS CLI, console or SDK
There is no additional charge for using AWS App Mesh
Supports any third-party tool that works with Envoy
App Mesh

52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
InstallAppMesh withHelmChart
# create ns and enable auto-injection
$ kubectl create ns appmesh-demo
$ kubectl label namespace appmesh-demo
appmesh.k8s.aws/sidecarInjectorWebhook=enabled
# deploy the demo
$ helm install -n aws-appmesh-demo
--namespace appmesh-demo
https://github.com/.../aws-appmesh-demo.tgz
https://github.com/PaulMaddox/aws-appmesh-helm

53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Logging
HTTP access logging
Amazon CloudWatch Logs
Available as container logs on Amazon ECS, Amazon EKS,
AWS Fargate
Metrics
CloudWatch metrics
StatsD (with tags)
Prometheus
Tracing
AWS X-Ray
Other Envoy tracing drivers
Observability

54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
VisualizeServiceCallGraph (Lambda)

55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Pinpoint errors (Lambda)

56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Open-SourceGrafana

57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Open-SourceGrafana

58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Traffic shaping
Load balancing
Weight targets
Service discovery (DNS + AWS Cloud Map)
Health checks
Retries*
Timeouts*
Circuit breakers*
*Coming soon
Traffic Management
Routing controls
Protocols support (HTTP, TCP, gRPC*)
Path-based
Header-based*
Cookie-based*
Host-based*

59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWSApp MeshRoadmap is Public
https://github.com/awslabs/aws-app-mesh-examples

60. Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
COBUS BERNARD
AWS SENIORTECHNICAL EVANGELIST
@cobusbernard