Running Containerized Workloads in Production Presentation from NY Event 4-10-19 and Boston Event 4-11-19

1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Containers
Journey to Cloud Native Modern Applications

2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Make AWS the
BEST PLACE
to run ANY containerized
applications
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Typical Use Cases
• Microservices: Java, Node.js, Go, Web Apps, etc.
• Continuous Integration and Continuous Deployment (CICD)
• Batch Processing and ETL jobs
• Common PaaS Stack for Application Deployment
• Legacy Application Migration to the Cloud
• Hybrid Workloads
• AI/ML
• Scale Testing
• Backend for IoT use cases

4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Container Services Landscape
MANAGEMENT
Deployment, Scheduling,
Scaling & Management of
containerized applications
HOSTING
Where the containers run
Amazon Elastic
Container Service
Amazon Elastic
Container Service
for Kubernetes
Amazon EC2 AWS Fargate
IMAGE REGISTRY
Container Image Repository
Amazon Elastic
Container Registry

5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
We Give You The Power To Choose:
ECS EKS
EC2 Fargate EC2 Fargate
1. Choose your
orchestration tool
2. Choose your
launch type

6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECS - Cluster
EC2 INSTANCES
LOAD
BALANCER
Internet
ECS
AGENT
TASK
Container
TASK
Container
ECS
AGENT
TASK
Container
TASK
Container
AGENT COMMUNICATION
SERVICE
Amazon ECS
API
CLUSTER MANAGEMENT
ENGINE
KEY/VALUE STORE
ECS
AGENT
TASK
Container
TASK
Container
LOAD
BALANCER

8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scheduling and Orchestration
Cluster Manager Placement Engine

9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS
Agent
Docker
Agent
OS
EC2 Instance
But you still end up managing more than just containers

10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Patching and Upgrading OS, agents, etc.
Scaling the instance fleet for optimal utilization

11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
An API for
containers

12. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Your
Containerized
Applications
MANAGED BY AWS
No EC2 Instances to provision, scale or manage
ELASTIC
Scale up & down seamlessly. Pay only for what you use
INTEGRATED
with the AWS ecosystem: VPC Networking,
Elastic Load Balancing, IAM Permissions, Cloudwatch and more.
AWS FARGATE

13. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ENABLE FOCUS ON APPLICATIONS

14. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RUNNING FARGATE CONTAINERS WITH ECS
Use ECS APIs to launch Fargate Containers
Easy migration – Run Fargate and EC2 launch
type tasks in the same cluster
Same Task Definition schema

15. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
TASK PROVISIONING MODEL
No instances
to manage
Container
native API
Resource
based pricing
Simple,
easy to use, powerful
consumption model
=

16. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS à Fargate

17. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
PRICING DIMENSIONS
{
"memory": “3GB”,
"cpu": “1 vCPU”,
"networkMode": ”AWSVPC",
"compatibilities": [”FARGATE", ”EC2"],
"placementConstraints": [],
"containerDefinitions": [
{
<snip>…....
Task level resources
• Configurable independently (within a range)
Dimensions: Task level CPU and memory
Per-second billing

18. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
TASK CPU & MEMORY CONFIGURATIONS
Flexible configuration options –
50 CPU/memory configurations
CPU Memory
256 (.25 vCPU) 512MB, 1GB, 2GB
512 (.5 vCPU) 1GB, 2GB, 3GB, 4GB
1024 (1 vCPU) 2GB, 3GB, 4GB, 5GB, 6GB, 7GB, 8GB
2048 (2 vCPU) Between 4GB and 16GB in 1GB increments
4096 (4 vCPU) Between 8GB and 30GB in 1GB increments

19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
COMPLIANCE
9001/27001/27017/27018

20. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
NETWORKING WITH FARGATE IN ECS
Internet
Gateway
172.31.0.0/16
Subnet 3
Fargate
Task
Public IP
54.191.135.69
172.31.3.0/24
ENI
Subnet 1
Fargate
Task
Public IP
54.191.135.66
172.31.1.0/24
ENI
Subnet 2
Fargate
Task
172.31.2.0/24
ENI
• AWS VPC Networking Mode – each task gets its own interface
• Full control of network access via Security Groups and Network ACLs
• Public IP support

21. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
LOAD BALANCING
APPLICATION LOAD BALANCER
NETWORK LOAD BALANCER

22. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
VISIBILITY AND MONITORING
Service-level metrics available
CloudWatch logs and
CloudWatch events supported

23. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Automatic Service Scaling
Publish metrics
Auto Scaling ECS service
Availability
Zone A
Availability
Zone B
TASK A
Add/Remove ECS
tasks
TASK C
TASK BScaling Policies
Amazon
CloudWatch
Amazon ECS
Application
Load Balancer

24. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example Microservice Architecture on ECS
Amazon
ECR
Amazon
RDS
Application Load
Balancer
ECS Cluster
ECS Cluster
IAM
Amazon API
Gateway*
Amazon
Route 53
Amazon CloudWatch

25. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Open source container
management platform
Helps you run
containers at scale
Gives you primitives
for building
modern applications
What is Kubernetes?

26. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
COMMUNITY, CONTRIBUTION, CHOICE

27. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
But where you run K8s matters
Q U A L I T Y O F T H E
C L O U D P L AT F O R M
Q U A L I T Y O F T H E
A P P L I C AT I O N S
Y O U R U S E R S

28. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
51%of Kubernetes workloads
run on AWS today
—CNCF survey

29. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
[mycluster].eks.amazonaws.com
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Kubectl
AWS Cloud
VPC

30. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EKS Overview
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
API Server API Server API Server
etcd etcd etcd
Worker nodes Worker nodes Worker nodes

31. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EKS is Kubernetes Certified
Kubernetes Conformance
• Guaranteed Portability and Interoperability
• Timely Updates
• Confirmability

32. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EKS Architecture
EKS VPCCustomer VPC
Worker Nodes
EKS-Owned
ENI
Exec, Logs,
Proxy
API Access via Private
end point
VPC for each customer cluster

33. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kubectl
3) Authorizes AWS Identity with RBAC
K8s API
1) Passes AWS Identity
2) Verifies AWS Identity
4) K8s action
allowed/denied
AWS Auth
IAM Authentication + Kubectl

34. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS VPC CNI plugin
ENI
Secondary IPs:
10.0.0.1
10.0.0.2
10.0.0.1
10.0.0.2
ENI
10.0.0.20
10.0.0.22
Secondary IPs:
10.0.0.20
10.0.0.22
ec2.associateaddress()
VPC Subnet –
10.0.0.0/24
Instance 1 Instance 2
VPC

35. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1.10….
Version
1.9
Version
1.12
Kubernetes Upgrades

36. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

37. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Pod Autoscaling Options
• HPA: de-facto method
• Cluster-proportional-autoscaler
• k8s-rabid-pod-autoscaler
• Kube-sqs-autoscaler

38. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EKS – Pod Scaling by HPA
• Horizontal pod scaling
• Automatically scales number of pods in replication
controller
• Implemented as a Kubernetes API resource and a
controller.
• The controller periodically adjusts # of replicas in a
replication controller or deployment to match the observed
average CPU utilization to the target specified by user.
• HPA can scale on custom and external metrics (any metric
collected by metrics server api) instead of simply CPU and
memory..

39. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster Autoscaling Options
• Cluster Auto-scaler (CA): de-facto method
• Kubernetes-ec2-autoscaler
• kube-aws-autoscaler
• Atlassian Escalator: batch or job optimized horizontal
autoscaler

40. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cluster auto-scaler
• Deployed As a daemon set
• Scale down cluster capacity if nodes are under-utilized
• Scale up in case of unschedulable pods
• Solely responsible for managing scaling!!
• Manages multiple ASG’s
• Only for EC2 Instance scaling

41. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Load Balancing
ALB Ingress Controller: Supported by AWS
Exposes ALB functionality to Kubernetes via Ingress
Resources
Layer 7 load balancing, supports content-based routing by
host or path

42. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ALB Ingress Controller
AWS Resources
Kubernetes Cluster
Node Node
Kubernetes
API Server ALB Ingress
Controller
Node
HTTP ListenerHTTPS Listener
Rule: /cheesesRule: /charcuterie
TargetGroup:
Green (IP Mode)
TargetGroup:
Blue (Instance
Mode)
NodePort NodePort

43. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THANK YOU
https://aws.amazon.com/containers