Weitere ähnliche Inhalte
Ähnlich wie Amazon ECS Deep Dive (20)
Mehr von Amazon Web Services (20)
Amazon ECS Deep Dive
- 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Paul Maddox - @paulmaddox
Developer Technologies, AWS
January 2018 (Nordic Dev Days)
Amazon ECS Deep Dive
From zero to production
- 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
About me
Paul Maddox
Specialist Solutions Architect
Amazon Web Services
• 16 years of dev, SRE, and systems architecture background
• 7 of 7 8 AWS certifications
• Developer: Go/Java/C/Node
Twitter: @paulmaddox
Email: pmaddox@amazon.com
@paulmaddox
Paul
- 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What to expect from this talk
• Build and deploy a containerized microservices application
• Twitter analyzer
• Go, RPC, Amazon Kinesis Firehose, AWS SSM Parameter Store
• Amazon ECS
• Deployment
• Availability
• Cost optimization
• Scaling
• Security
• Monitoring & logging
- 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key Components
Development cluster
Container instance Container instance
Container instance
Production cluster
Container instance Container instance
Container instance
Amazon Elastic Container Service
(Amazon ECS)
Container
Container
Volume
Task definition
Amazon Elastic Container Registry
(Amazon ECR)
- 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key Components
Development cluster
Container instance Container instance
Container instance
Production cluster
Container instance Container instance
Container instance
Amazon Elastic Container Service
(Amazon ECS)
Container
Container
Volume
Task definition
Amazon EC2 Container Registry
(Amazon ECR)
NEW
NEW
- 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Component: ECS
AWS is responsible for
operations of the cloud
You are responsible for operations in the cloud
using the building blocks provided.
Deployment
Security
Patching
Monitoring
Scaling
Availability
Cost Control
$ aws ecs create-cluster --cluster-name dev
AWS
Customer
- 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Component: ECR
Deployment
Security
Cost Control
AWS
Customer
Monitoring
Scaling
Availability
Patching
AWS is responsible for
operations of the cloud
You are responsible for operations in the cloud
using the building blocks provided.
- 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Component: Container Instances
Development cluster
Cluster instance Cluster instance
Cluster instance
Deployment Cost Control
Patching Monitoring
Scaling Availability
Security
AWS
Customer
AWS is responsible for
operations of the cloud
You are responsible for operations in the cloud
using the building blocks provided.
- 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Container Instances: Building Blocks Provided
Deployment
Security
Patching
Monitoring
Scaling
Availability
Cost Control
CloudFormation
Update your AMI, replace instances
CloudWatch
Auto Scaling group
Reserved Instances
CLI SDKs etc...
IAM Inspector VPC Flow Logs etc...
Spot Fleet
- 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Component: Tasks & Containers
Container
Container
Volume
Deployment
Security
Patching
Monitoring
Scaling
Availability
Logging
AWS
Customer
AWS is responsible for
operations of the cloud
You are responsible for operations in the cloud
using the building blocks provided.
- 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deployment
- 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How Should I Set This Up?
Use the AWS
Management
Console?
- 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How Should I Set This Up?
Flex your scripting skills?
What happens if
my script fails
halfway through?
How long
should I
pause?
How do I upgrade /
roll back?
- 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deployments should be:
- A self-contained, deployable unit
- Repeatable
- Auditable
- Self-documenting
- 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CloudFormation: Infrastructure-as-Code
- 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Time to deploy!
…or…
- 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Time to update…
…or…
- 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
When a new environment is required…
…or…
- 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CLI
ECR
CloudFormation (YAML)
Resources:
MyRepository:
Type: AWS::ECR::Repository
Properties:
Name: myapp
- 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Using ECR
Use AWS CLI to perform ‘docker login’
Tip: Use the Amazon ECR Credential Helper for automatic logins
https://github.com/awslabs/amazon-ecr-credential-helper
- 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS CLI
ECS Cluster
CloudFormation (YAML)
Resources:
ECSCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: preprod
- 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ECS Container Instances
• Highly available architecture,
distributed across multiple Availability
Zones
• VPC with public and private subnets
• Application Load Balancer with path
based routing for inbound traffic
• NAT gateways for outbound traffic
• Auto Scaling group of container
instances
• CloudWatch Logs for centralized
container logging
Private Subnet
Availability Zone Availability Zone
Internet
Gateway
Public Subnet Public Subnet
Private Subnet
Nat GatewayNat Gateway
AutoScaling GroupContainer InstanceContainer Instance Container InstanceContainer Instance
Application
Load Balancer
CloudWatch Logs
(container logs)
- 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Inbound Traffic
• Incoming HTTP/HTTPS traffic comes in
via the Application Load Balancer (ALB)
in public subnets
• The ALB uses path based routing to
route /products/* to the container
instances in private subnets running our
product’s service
• Supports dynamic host port mapping,
allowing multiple containers of the
same type on each host
Internet
Gateway
AutoScaling Group
Container Instance Container Instance
Application
Load Balancer
- 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Outbound Traffic
• Our container instances are in private
subnets, with no direct internet access
• At some point, they might need access
to external services
• NAT gateways provide a highly scalable
and available solution
Private Subnet
Internet
Gateway
Public Subnet Public Subnet
Private Subnet
Nat GatewayNat Gateway
Container Instance Container Instance
- 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Logging
Container Instance Container Instance
CloudWatch Logs
(container logs)
• ECS integrates directly with
CloudWatch Logs (as well as others)
• Centralized collection of container logs
• Centralized collection of instance logs
• Search, filter, and alert on log
conditions
• (more to come later…)
- 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
tl;dr - ECS Reference Architecture on GitHub
https://github.com/awslabs/ecs-refarch-cloudformation
- 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
hands-on-demo
(because slides are boring)
- 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Let's build an application
- 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Parameter Store
(for Twitter Credentials)
Overview
Tweet
Collector
Twitter API
Tweet
Archiver
Kinesis Firehose
Amazon S3 (archive)
AWS Lambda (realtime)
Elasticsearch (analyze)
- 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Microservices and RPC at Twitch
• Used for inter-service communication
• Structured RPCs are much easier to
design and maintain compared to
REST
• Focus on data models, not
transports/routing
• Works with protobufs or JSON
• HTTP/1 compatible (unlike gRPC)
• Simplicity
https://blog.twitch.tv/twirp-a-
sweet-new-rpc-framework-for-
go-5f2febbf35f
- 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
RPC with Twitch Twirp
• Write a spec describing your API
(using protobufs)
• Generate a client and server
from the specification
• Limited to Go today, but more
language support in progress.
• (hands-on demo in a few slides)
- 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Tweet Collector
• Written in Go
• Uses Twitter API to subscribe to
search terms
• Environment variables:
• SEARCH_TERMS
• ARCHIVE_ENDPOINT
• IAM role:
• AWS SSM Parameter Store
(for Twitter API credentials)
• Sends tweets to archiving service via
client SDK generated by Twitch
Twirp.
AWS Parameter Store
(for Twitter Credentials)
Tweet
Collector
- 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Tweet Archiver
• Written in Go
• Hosts RPC server that receives tweets
• Sends tweets to Amazon Kinesis via aws-sdk-go
• Environment variables
• KINESIS_STREAM_NAME
• IAM role
• Write access to Kinesis stream
• Responds with Kinesis sequence number or error
Tweet
Archiver
Kinesis Firehose
- 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Local
• Run locally with docker-
compose
• Logs to stdout/stderr
• Local AWS credentials
• Build/push containers
Development Workflow
AWS
• Deploy to ECS with
CloudFormation
• Logs in AWS CloudTrail Logs
• IAM Task Role
• Metrics in CloudWatch
- 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
https://github.com/paulmaddox/rpc-demo
- 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Taking it further
• Sentiment analysis with
Amazon Comprehend
• Dashboards with Amazon
Quicksight
https://aws.amazon.com/blo
gs/machine-learning/build-a-
social-media-dashboard-
using-machine-learning-and-
bi-services
- 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What else do we need for
production?
- 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cost Optimization
- 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Reserved Instances
Up to 75%
Savings*
• Use Auto Scaling groups
• Reserve ECS container
instances when you have
known baseline capacity
requirements.
• Use On-Demand pricing for
capacity peaks.
* Dependent on specific AWS service, size/type, and region
- 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Spot Instances
Up to 90%
Savings*
• Use Spot Fleet to maintain
instance availability and
define cluster based on
required CPU/memory.
* Compared to On-Demand price based on specific EC2 instance type, region, and Availability Zone
- 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Multiple ECS Clusters
Creating multiple ECS clusters is easy, and often more cost efficient.
Consider availability and compute requirements.
Example: Development Cluster
Spot Fleet
Example: Production Cluster
Auto Scaling group with Reserved Instances for baseline and
On-Demand for capacity peaks
Example: Batch Processing Cluster
Spot Fleet of GPU Instances
- 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scaling
- 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scaling ECS Container Instances Automatically
Min
Desired
Scale out as needed
Max
• Use Auto Scaling groups
• Set Auto Scaling group
min, max, desired
• Scale in and out based
on CloudWatch alarms
- 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Scaling ECS Container Instances Automatically
Tip
Use the ECS cluster
MemoryReservation
CloudWatch metric
Tutorial: Scaling Container Instances with CloudWatch Alarms
- 45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Application Auto Scaling for ECS Services
- 46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Application Auto Scaling for ECS Services
- 47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Security
- 48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Patching ECS Container Instances
ECSLaunchConfiguration:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
ImageId: ami-1924770e
ECSAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
MinSize: 2
MaxSize: 8
DesiredCapacity: 2
AutoScalingRollingUpdate:
MinInstancesInService: 2
MaxBatchSize: 2
PauseTime: PT15M
WaitOnResourceSignals: true
1. Ensure you have an
AutoScalingRollingUpdate
policy on your Auto Scaling group
2. Update the AMI in your
CloudFormation template
3. aws cloudformation update-stack
4. Let CloudFormation perform a rolling
update to your ECS container
instances
- 49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Patching Containers
- 50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Minimal Containers
• Use the smallest FROM base
container to minimize
surface attack
• FROM scratch is ideal for
Go and other languages
that compile a (near) static
binary
- 51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IAM Roles
IAM roles for container instances:
• Bound to the ECS container instance
• Applies to all containers running on the host
• Pulling images from ECR
• CloudWatch Logs
IAM roles for tasks:
• Bound to specific ECS tasks
• Task-specific access to AWS services
Tip Use principle of least privilege – prefer IAM roles for tasks where applicable
- 52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Monitoring & Logging
- 53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Monitoring with CloudWatch
- 54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Monitoring with CloudWatch
- 55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Prometheus
https://github.com/slok/ecs-exporter
- 56. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Centralized Logging with CloudWatch Logs
• Defined within the task definition
• Available log drivers
• awslogs
• fluentd
• gelf
• journald
• json-file
• splunk
• Syslog
- 57. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Centralized Logging with CloudWatch Logs
- 58. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Tip: Use Metric Filters with CloudWatch Logs
5
- 59. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Everything about everything ECS.
https://github.com/nathanpeck/awesome-ecs
- 60. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you