New AWS Services

New AWS Services
AWS PHOENIX MEETUP
Josh Padnick
Gruntwork
Monday, August 29, 2016

Today’s talk is about
three recent updates
in AWS.

Today’s talk is about
three recent updates
in AWS.
‣ Application Load Balancer (ALB)
‣ EC2 Container Service (ECS)
‣ Kinesis Analytics

For each service,
we’ll discuss…
‣ The big idea
‣ What’s new
‣ Examples

Intended Audience
Executives Developers

We’ll start simple.
But we’ll get progressively
more technical.
At a certain point, we’ll
dive deep into the
technical nuances of the
topic.
In such cases, look for the
Nerd Alert ribbon.
Nerd Alert

Hi, I’m Josh Padnick.
‣ Published A Comprehensive Guide  
to Building a Scalable Web App on AWS.
Received 500+ up votes on Hacker News.
‣ Consulted on DevOps & AWS with ~25
companies worldwide including Intel and
Infusionsoft.
‣ Full-stack engineer for 10+ years
‣ Co-founder at Gruntwork.

‣ We setup software teams on AWS with DevOps
best practices and world-class infrastructure.
‣ But we do it in about 2 weeks!
‣ The secret sauce is we offer battle-tested, pre-
written “Infrastructure Packages” for common
AWS needs.
‣ Plus consulting & support as needed.
http://gruntwork.io
I work at Gruntwork.

Application Load Balancer
(ALB)

Let’s start by talking about the
generic concept of a Load
Balancer.

The Big Idea
Hi, I’m an EC2 Instance!
M
y
A
pp

The Big Idea
With a single VM,
users can connect
directly to the VM.

The Big Idea
But if that VM fails,
our entire service
goes down.
X
X

The Big Idea
With multiple VMs,
we gain High
Availability (HA)!

The Big Idea
If one VM goes
down, we can just
serve trafﬁc from
the other.
X
X
✅

The Big Idea
But how do we route
requests to more
than one VM?
?

The Big Idea
We use a Load
Balancer. This is
sometimes called a
Reverse Proxy.
Load
Balancer

The Big Idea
There are a few properties we want out of this load
balancer:

The Big Idea
There are a few properties we want out of this
generic load balancer:
‣ It should itself be HA!
‣ It should elastically scale as we get more trafﬁc.
‣ It should do a Health Check on each VM.

The Big Idea
Keep going…
‣ It should support the latest protocols  
(TCP, UDP, HTTP(S) 1.1, HTTP/2, WebSockets).
‣ It should log all requests.
‣ It should emit helpful metrics.

The Big Idea
Keep going…
‣ It should allow routing a single user to the
same VM, but spread different users across
different VMs (sticky sessions).
‣ It should route a request for /apples to one
set of VMs and /oranges to another (path-
based routing).

The Big Idea
Keep going…
‣ It should have ﬁrst-class support for routing to
Docker containers in EC2 Container Service (ECS)
‣ Route to an app running in a container, not just
to a VM.
‣ Route to multiple different containers on the
same VM.
‣ Know about new containers when I launch
them (service discovery).
Nerd Alert

In 2012, Amazon released the
Elastic Load Balancer.
Elastic Load Balancer (ELB)

Nerd Alert
Old ELB was a Layer 4 Load Balancer
Open Systems Interconnection (OSI) Network Model
Physical / Data Link1 / 2
Network (IP, ICMP)3
Transport (TCP, UDP)4
Session5
Presentation (TLS)6
Application (HTTP, FTP, DNS, SSH)7

But there’s a problem…
‣ Helpful metrics like “Sum HTTP 5XX errors” only
apply to HTTP traffic.
‣ Path-based routing requires inspecting the HTTP
traffic.
Some of our feature asks are HTTP-specific.

But there’s a problem…
‣ Route to more than one port on the same VM
Some of our feature asks are DOCKER-speciﬁc.

So AWS has released the new
Application Load Balancer (ALB).

So AWS has released the new
Application Load Balancer (ALB).
An updated load balancer opinionated to:  
- modern apps built with HTTP 
- Docker

Updated Terminology
Elastic Load Balancing
Application Load Balancer (ALB) Classic Load Balancer 
(Sometimes called “ELB”)

Nerd Alert
ALB is a Layer 7 Load Balancer
Open Systems Interconnection (OSI) Network Model
Physical / Data Link1 / 2
Network (IP, ICMP)3
Transport (TCP, UDP)4
Session5
Presentation (TLS)6

Nerd Alert
ALB is a Layer 7 Load Balancer
Translation
‣ The ALB inspects HTTP trafﬁc and makes
routing decisions based on this.
‣ But the ALB doesn’t deal with “OSI Layer 3”
forwarding, so no TCP or UDP forwarding.

Support for HTTP/2
‣ Did you know HTTP 1.1 came out in 1999
when this was what the Web looked like?

Support for HTTP/2
‣ The web of 2016 is different than the web of
1999:

HTTP/2 Benefits
‣ Sends headers/cookies just once instead of on
every request.
‣ Encodes all data in binary versus a textual
format.
‣ Transmits all data over a single, multiplexed
TCP connection versus multiple blocking
connections in HTTP/1.1.
Nerd Alert

Your Backend App Can Still  
Speak HTTP/1.1
Nerd Alert
HTTP/2 HTTP/1.x
Note that HTTP/2 requires that 
you use HTTPS on the ALB.

All modern browsers
support HTTP/2
Nerd Alert
SOURCE: http://caniuse.com/#search=http2

Support for WebSockets
‣ A long-time ask for ELBs has been WebSocket
support. ALBs now support this!
Nerd Alert
ws://…
ws://…

Content-Based Routing
‣ Route /blue to one service.
‣ Route /green to another service.
‣ Previously, this required two load balancers.
Now, it requires just one!

‣ LIMITATION
‣ We don’t get path rewriting.
‣ So you can’t send /blue to /hello/blue
unless your backend app handles that.
Nerd Alert

New Concepts in Elastic Load Balancing
‣ Target Groups
The Classic Load Balancer includes as part of its
conﬁguration which EC2 Instances it will route to.
ELB

‣ Target Groups
With ALBs, the concept of Load Balancer is separated from
the concept of Target EC2 Instances.
ALB
Target  
Group
Target  
Group

‣ Target Groups
Our ALB needs a list of “targets” where it can send trafﬁc.
We’ll group all such targets into a Target Group.
Empty Target Group

‣ Target Groups
Let’s add one Target:
i-123
Port 8000
Notice we have both an
instance id and port number.

‣ Target Groups
Let’s add a second Target:
i-123
Port 8000
i-123
Port 8001
This target has the
same instance id
but a different
port number.

‣ Target Groups
Let’s add a third Target:
i-123
Port 8000
i-123
Port 8001
i-789
Port 3034

‣ Target Groups
Our ALB will send trafﬁc to any 
Healthy Target in the Target Group.
i-123
Port 8000
i-123
Port 8001
i-789
Port 3034

‣ Target Groups
Note that the Classic ELB does not use a
Target Group and can only send to the
same port on different EC2 Instances.
i-123
Port 8000
i-789
Port 8000

‣ Target Groups
The big takeaway is you can group your (micro)services into
Target Groups, even if multiple target groups include the
same EC2 Instance!
i-123 i-456 i-789
Service B
Service A
Nerd Alert

‣ Route /blue to one service Target Group.
‣ Route /green to another service Target Group.
‣ Previously, this required two load balancers.
Now, it requires just one!

Support for Container-Based Apps
‣ We often want to run the same Docker image
on the same EC2 Instance on different ports.
‣ Target Groups means the ALB can route to
either to two different ports on the same
server!
‣ This also means we can dynamically select our
container ports in an EC2 Container Service
Cluster!
Nerd Alert

Nerd Alert
Support for Container-Based Apps
Port 
8523
Port 
8000
Port 
4738
Port 
8713

Target Group metrics.
‣ We get CloudWatch Metrics on Target Groups.
‣ This is a nice way to get metrics speciﬁc to a
service.
Nerd Alert

Better metrics.
‣ Many new metrics on the ALB!
Nerd Alert
‣ ClientTLSNegotiationErrorCount
‣ TargetTLSNegotiationErrorCount
‣ TargetConnectionErrorCount
‣ TargetResponseTime
‣ NewConnectionCount
‣ ActiveConnectionCount
‣ RejectedConnectionCount
‣ ProcessedBytes

Other Cool Features
‣ Load-balancer generated sticky-session
cookies (client must support cookies).
‣ Slightly less expensive.
‣ Faster performance in general.
Nerd Alert

When to Use the ALB
‣ When running any HTTP-based service.
‣ When using WebSockets with a load balancer.
‣ When using Docker, especially with EC2
Container Service.

When to Use the Classic ELB
‣ You need OSI Layer 4 Routing (i.e. TCP / UDP)
‣ Your app listens on a protocol other than HTTP.

Alternatives to the ALB/ELB
‣ Set up your own load balancer using Nginx or
HAProxy.
‣ But this means you need to build auto-
scaling, auto-failover, automated DNS
updates, conﬁgure metrics, conﬁgure
logging, manage upgrades, and a few more
items.
‣ Conclusion: don’t do this unless you have to.

Recent Updates to
EC2 Container Service (ECS)

The Big Idea
Hi, I’m an EC2 Instance!

The Big Idea
I can offer you resource isolation.
And I can be launched in just minutes!

Limitations of a VM
But minutes could be an eternity.
If deploying multiple times a day, 
we’re just waiting for VMs to launch.
Building an Amazon Machine Image 
also takes on the order of minutes.

Limitations of a VM
And I can’t run that AMI locally.
If I want to run the same “Golden
Image” locally, I’m out of luck.
X

Sometimes a single app uses a  
tiny portion of available resources.
Mem Usage: 12%
CPU Usage: 7%

So it’d be nice if we could pack
multiple apps in a single EC2 Instance.
Mem Usage: 85%
CPU Usage: 90%
App 1 App 2
App 3

Why developers love containers.
‣ A container is just an isolated OS process, so it runs
directly on your EC2 Instance.
‣ It’s similar to a “lightweight VM” and can start in
milliseconds.
‣ You can run multiple containers on a single EC2
Instance.
‣ You can run the same docker image on any platform.
‣ You can download pre-built docker images for almost
all custom software.

So we want to run our apps as containers.
‣ But we don’t want to run containers on just a
single EC2 Instance.
If I go down, I’m taking  
all apps with me!

We want to run multiple containers across  
multiple EC2 Instances.

But running a “docker cluster” is hard.
‣ Way to bootstrap the cluster
‣ Container scheduler
‣ Service Discovery solution
‣ Load balance to containers
‣ Auto-restart failed containers
‣ Cluster-wide metrics
We need…

There are multiple options to solve this
problem today.

But my favorite solution is  
Amazon EC2 Container Service (ECS)
Amazon EC2 Container Service

Benefits of ECS
‣ Built-in cluster bootstrapping
‣ Built-in scheduler  
(with ability to use a custom scheduler)
‣ Built-in service discovery
‣ Built-in load balancer (ALB)
‣ Built-in auto-restart on failed containers
‣ NEW! Auto-scale your service
‣ NEW! Fine-grained AWS permissions on your service

What’s Missing from ECS
‣ Service-to-service authentication
‣ Run background jobs within the cluster (you
can still do this with Lamba’s run on cron
schedules, though)
‣ DNS namespacing
‣ Built-in persistent volumes
‣ Built-in support for log aggregation (on
services other than CloudWatch Logs)

Then why is it my favorite?
‣ Because most teams don’t need those features.
‣ If you’re ok with the limitations, ECS is easier to
setup than anything else.
‣ The new ALB plus the new features we’ll talk
about make this even more compelling.

ECS Task Definition
Declares what kind of ECS Task should be run. 
(e.g. docker run properties)

EC2 Instances get permissions to
AWS Resources via IAM Roles.
EC2 Instance
Authenticates to AWS
via IAM Role

EC2 Instances get permissions to
AWS Resources via IAM Roles.
EC2 Instance
S3 Bucket

Feature #1:
IAM Role for ECS Tasks

IAM Roles for EC2 Instances
Previously, ECS Tasks could only get permission
to other AWS resources (e.g. a ﬁle in S3) by using
the IAM Role of the ECS Instance.
ECS Instance
IAM Role

IAM Roles for EC2 Instances
This meant that the BLUE and YELLOW app both
got the same AWS permissions.
ECS Instance IAM Role
ECS Instance IAM Role

IAM Roles for ECS Tasks
With IAM Roles for ECS Tasks, now each ECS Task
can get its own IAM Role!
ECS Task IAM Role
ECS Task IAM Role

IAM Roles for ECS Tasks
This means that each ECS Task can have its own
set of permissions to other AWS resources.
ECS Task IAM Role
ECS Task IAM Role
Bucket A
Bucket B

How It Works
‣ When we create an ECS Task Deﬁnition, we can
now specify a Task Role.

Feature #2:
ECS Service Auto-Scaling

Previously, we could auto-scale the ECS Instances
but not the ECS Tasks.
This meant that we could not auto-scale an ECS
Service without lots of hackery.

Now we can!

We deﬁne Scaling Policies just like with classic
Auto Scaling.

We Scale on Just Two Metrics
‣ CPUUtilization
‣ MemoryUtilization
Total CPU/Memory In Use
Total CPU/Memory Reserved
=

Feature #3:
EC2 Container Registry (ECR)

When you work with Docker, you need
a place to store your Docker images.
‣ Classic Docker build pipeline example:
Git Commit
to Master Branch
Build Docker
Image
Push to Docker
Registry

There are a few options for the
Docker Registry
‣ Docker Hub
‣ Quay.io by CoreOS
‣ Artifactory by jfrog

But there are some challenges.
‣ Docker Hub can sometimes be slow or
unreliable.
‣ Authenticating to any solution means you have
to store the credentials somewhere.
‣ Download speeds and proximity to the service
make a difference.

So Amazon has released  
EC2 Container Registry (ECR)
Amazon EC2 Container Registry

ECR Features
‣ Fully managed by Amazon
‣ Relatively fast
‣ Accessible by a typical docker client
‣ Integrated with IAM Policies and IAM Users

ECR Limitations
‣ You can only store up to 1,000 images per
docker repo.
‣ Pricing model requires you cull your unused
docker images from the ECR repo.
‣ No hosting of public docker images.
‣ Docker repo names can be awkwardly long.

But I still prefer ECR.
‣ One less vendor to deal with.
‣ One integrated security model.
‣ Repo limits are probably appropriate.
‣ Not hosting public repos gives clear separation
of public and private repos.

Big Idea
‣ As companies grow, they eventually evolve out
of the monolithic app and into a microservices
architecture.
Microservice A Microservice B

‣ Usually, companies will start with two microservices.
‣ Then they’ll keep factoring out monolithic code into
more and more microservices.

‣ Eventually, teams will want an individual
microservice to publish an event stream.

‣ This way Microservice B can do something
when Microservice A publishes a certain event.

‣ But if we have n services, and each service reads
the event stream of the other n - 1 services, now
we have a combinatorial explosion:

‣ What if instead all services published their event streams to a
central service.
‣ And all services read event streams from that same central service.

‣ Now we have n connections, which is
manageable!

‣ These are the insights that LinkedIn had around 2011 when it wrote
Apache Kafka.
‣ The central “event publishing service” would need to be:
‣ scalable
‣ resilient
‣ temporarily persist data to support consumers that go down
‣ not lose any data, even as data volume surges

‣ The details are published in an epic blog post
by LinkedIn engineer and Kakfa author Jay
Kreps:

‣ It turns out the concept of a scalable,
performant, resilient centralized event stream
can apply to lots of domains!
‣ IoT events
‣ Logging events
‣ Social media clickstreams
‣ Basically, any real-time data source

‣ But running a Kafka cluster is highly non-trivial.
‣ So AWS introduced its own version of Kafka
and offered it as a managed service.
Amazon Kinesis Streams

‣ At ReInvent 2014, Amazon shared a wicked
cool example of how Major League Baseball
was tracking data from the ﬁeld and using it to
generate stats, visualizations, and more:

‣ Here’s an excerpt from their architecture.

‣ But what happens after the data gets into
Kinesis?
Amazon Kinesis
?

‣ The answer is that we can have Kinesis
Consumers that periodically read the data.
Amazon Kinesis
Me Want Moar Data!

‣ The consumer can then do anything with it
‣ Store it in S3 for later retrieval.
‣ Store it in RedShift for later querying.
‣ Store it in a relational database.
‣ Or any other custom operation.

‣ Previously, we had to write our own custom
worker to do any processing.

‣ But what if we just want to query windows of
incoming data and write it to a database? Isn’t
that pretty common?

‣ But now we don’t have to!
‣ That’s why Amazon has introduced:
Amazon Kinesis Analytics

Input - Query - Output
‣ Inputs
‣ Streaming Data Sources: Kinesis Streams, Kinesis Firehose
‣ Reference Data Source: Data in S3
‣ Query
‣ Write ANSI SQL against the data stream
‣ Outputs
‣ S3
‣ Redshift
‣ Kinesis Firehose ( —> Amazon Elasticsearch)
‣ Kinesis Streams

Core Features
‣ Use Standard SQL to query data streams.
‣ Kinesis will inspect your data stream and automatically
create a baseline schema against which you can write
your queries.
‣ Built-in live SQL editor to test queries against live data.
‣ Pre-written queries for common use cases.
‣ Query continuously, by Tumbling Windows, or Sliding
Windows.

Let’s combine it all in a
sample architecture!

Business Problem
‣ Ice Cream shop
‣ IoT Enabled
‣ We track weight of each tub of ice cream
continuously as a way to know in real-time how
much ice cream we need to order.
‣ Our customer wants a slick real-time
dashboard of everything.

Architecture
IoT Weight Monitors
Kinesis Streams
Kinesis Analytics
S3 Bucket
ECS Cluster
App to Query S3 Data  
and return Dashboard data
App that serves static assets 
for a Single-Page App
ALB
Users get dashboard
updates with WebSockets
RDS PostgreSQL

Caveats
‣ If you had a low enough volume of data, you could just have
your sensors write directly to RDS Postgres and reduce lots of
cost and complexity.
‣ But if you have enough data volume that you need the power of
Kinesis, then this architecture makes sense.
‣ Querying S3 for real-time data is probably a bad idea, so it may
make more sense to write a worker to read from S3 and write
data to RDS Postgres or to use Redshift.
‣ Serving a static web app from an ECS app isn’t bad, but using S3
(+ CloudFront) is more efﬁcient (but also more complex to
setup).

Thank you!
Want to keep up with the latest news on
DevOps, AWS, software infrastructure,
and Gruntwork?
http://www.gruntwork.io/newsletter/

New AWS Services

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie New AWS Services

Ähnlich wie New AWS Services (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

New AWS Services