AWS 101 Event December 2013

AWS 101
Alistair McLaurin
AWS Solution Architecture

Agenda

10:00 - 10:45 Presentation – AWS 101
Introducing the concepts behind AWS, such as utility computing and elasticity.

10:45 - 11:00 Coffee Break
11:00 - 11:45 Presentation and Demonstration
Live demonstration and interactive walkthrough

What we are going to cover

Keypairs

Amazon Machine Images

Security groups

S3

EC2 instances

CloudFront

Metadata service

Elastic Load balancer

Autoscaling

RDS

Feel free to follow along on your laptops

Consumer
Business
Tens of millions of
active customer
accounts

8 countries:
US, UK, Germany,
Japan, France,
Canada, China, Italy

Seller
Business

IT Infrastructure
Business

Sell on Amazon
websites

Cloud computing
infrastructure for
hosting web-scale
solutions

Use Amazon
technology for your
own retail website
Leverage Amazon’s
massive fulfillment
center network

Hundreds of
thousands of
registered
customers in over
190 countries

About Amazon
Web Services

How did Amazon…

?

Deep experience in
building and
operating global web
scale systems

…get into cloud computing?

AWS Mission
Enable businesses and
developers to use web
services* to build scalable,
sophisticated applications.
*What people now call “the cloud”

Each day AWS adds the equivalent server
capacity to power Amazon when it was a
global, $7B enterprise

Objects in S3
2 Trillion
1.7 Trillion

Over 1.1 Million
requests per second

762 Billion

262 Billion
2.9 Billion

14 Billion

40 Billion

Q4 2006

Q4 2007

Q4 2008

102 Billion
Q4 2009

Q4 2010

Q4 2011

Q4 2012

Total Number of Objects Stored in Amazon S3

Q2 2013

Utility computing

On demand

Uniform

Pay as you go

Available

Utility computing

On demand

Compute
Security
DNS

Scaling
CDN Backup

Database

Storage

Load Balancing

Workflow

Uniform

Pay as you go

Monitoring

Networking
Messaging

Available

On a global footprint
Region
US-WEST (N. California)

EU-WEST (Ireland)
ASIA PAC
(Tokyo)

GOV CLOUD

US-EAST (Virginia)

US-WEST (Oregon)

ASIA PAC
(Singapore)

SOUTH AMERICA (Sao
Paulo)
ASIA PAC
(Sydney)

At the end of a web service
ec2-run-instances ami-b232d0db
--instance-count 3
--availability-zone eu-west-1a
--instance-type m1.small

ec2-run-instances ami-b232d0db
--instance-count 5
--availability-zone eu-west-1c
--instance-type m1.medium

Elastic capacity

Traditional IT
capacity

Capacity

Time

Your IT needs

Elastic capacity

On and Off

Fast Growth

Variable peaks

Predictable peaks

Elastic capacity
WASTE

On and Off

Fast Growth

Variable peaks

Predictable peaks

CUSTOMER DISSATISFACTION

Typical weekly traffic to
Amazon.com

Sunday

Monday

Tuesday

Wednesday

Thursday

Friday

Saturday

November traffic to Amazon.com

November

Provisioned capacity

November

76%
Provisioned capacity

November

24%

November 10th 2010
Turned off last physical web server of
Amazon.com

November 10th 2010
Turned off last physical web server of
Amazon.com

October 31st 2011
Turned off last web servers supporting
European business

40 servers to 5000 in 3 days

Number of EC2 Instances

EC2 scaled to peak
of 5000 instances

“Techcrunched”
Launch of Facebook
modification
Steady state of ~40
instances

4/12/2008

4/13/2008

4/14/2008

4/15/2008

4/16/2008

4/17/2008

4/18/2008

4/19/2008

4/20/2008

Reference Model
security

Deployment & Administration
App Services

Compute

Storage

Database

Networking
AWS Global Infrastructure

Global infrastructure

App Services
Compute

Storage

Database

Regions
An independent collection of AWS resources in a

Networking

defined geography
A solid foundation for meeting location-dependent


privacy and compliance requirements


App Services
Compute

Storage

Database

Availability Zones
Designed as independent failure zones

Networking

Physically separated within a typical metropolitan
region



App Services
Compute

Storage

Database

Edge Locations
To deliver content to end users with lower latency

Networking

A global network of edge locations
Supports global DNS infrastructure (Route53) and


Cloud Front CDN

Networking
Direct Connect
Dedicated connection to AWS

VPN Connection
Secure internet connection to AWS
App Services
Compute

Storage

Virtual Private Cloud
Private, isolated section of the AWS Cloud

Database

Networking

Route 53
Highly available and scalable Domain Name

Service

Compute
Vertical
Scaling
From $0.02/hr

Elastic Compute Cloud (EC2)
Basic unit of compute capacity
Range of CPU, memory & local disk options
13 Instance types available, from micro to cluster

compute

Feature

Details

App Services

Run windows or linux distributions

Scalable


Flexible

Wide range of instance types from micro to cluster
compute

Machine Images

Full control

Compute

Storage

Database

Secure

Configurations can be saved as machine images
(AMIs) from which new instances can be created

Full root or administrator rights
Full firewall control via Security Groups


Monitoring

Publishes metrics to Cloud Watch

Inexpensive

Networking

On-demand, Reserved and Spot instance types

VM Import/Export

Import and export VM images to transfer
configurations in and out of EC2

Compute
Trigger autoscaling policy

as-create-auto-scaling-group MyGroup
--launch-configuration MyConfig
--availability-zones eu-west-1a
--min-size 4
--max-size 200


Auto-scaling
Automatic provisioning of compute resources based

App Services

upon demand, configuration or schedule
Compute

Storage

Database

Feature
Control

Networking

Details
Define minimum and maximum instance pool sizes
and when scaling and cool down occurs
Use metrics gathered by CloudWatch to drive scaling

Instance types


Integrated to
CloudWatch

Run auto scaling for on-demand instances and spot.
Compatible with VPC

Compute
Elastic Load Balancing
Create highly scalable applications
Distribute load across EC2 instances in multiple
availability zones


Feature
Auto-scaling

App Services

Available

Compute

Storage

Database

Health checks
Session stickiness

Networking

Secure sockets layer
Monitoring

Details
Automatically scales to handle request volume
Load balance across instances in multiple availability
zones
Automatically checks health of instances and takes
them in or out of service
Route requests to the same instance
Supports SSL offload from web and application
servers with flexible cipher support
Publishes metrics to Cloud Watch

Storage
S3 - Durable storage, any
object
99.999999999% durability of objects

Unlimited storage of objects of any type
Feature
Details
Up to 5TB size per object
Flexible object store

Access control


Server-side encryption
Multi-part uploads

Buckets act like drives, folder structures within

Granular control over object permissions
256bit AES encryption of objects
Improved throughput & control

App Services
Object versioning

Compute

Storage

Database

Object expiry
Access logging

Networking

Web content hosting
Notifications


Import/Export

Archive old objects and version new ones
Automatically remove old objects
Full audit log of bucket/object actions
Serve content as web site with built in page handling
Receive notifications on key events
Physical device import/export service

Storage
Elastic Block Store
High performance block storage device
1GB to 1TB in size
Mount as drives to instances

Feature
High performance file
system


Flexible size
Secure

App Services

Available

Compute

Storage

Database

Backups
Monitoring

Networking

Details
Mount EBS as drives and format as required

Volumes from 1GB to 1TB in size
Private to your instances
Replicated within an Availability Zone
Volumes can be snapshotted for point in time restore
Detailed metrics captured via Cloud Watch

Database
Relational Database Service
Database-as-a-Service
No need to install or manage database instances
Scalable and fault tolerant configurations


Feature
Platform support

App Services

Preconfigured

Details
Create MySQL, SQL Server and Oracle RDBMS
Get started instantly with sensible default settings

Automated patching

Storage

Database

Networking

Backups

Automatic backups and point in time recovery and full
DB backups

Backups

Volumes can be snapshotted for point in time restore

Failover

Compute

Keep your database platform up to date automatically

Automated failover to slave hosts in event of a failure

Replication

Easily create read-replicas of your data and
seamlessly replicate data across availability zones

Database
Amazon Relational Database Service
(Amazon RDS) databases stores forum
threads, site content, and project
configuration data.
High availability Multi-AZ database
deployment to handle live game metadata
and user-generated content.
App Services
Compute

Storage

Database

Networking

Enterprise-grade fault tolerance for
protecting customer data.
By managing time-consuming database
administration tasks, Amazon RDS allows
SEGA to focus on business critical
applications.

Database
DynamoDB
Provisioned throughput NoSQL database
Fast, predictable performance
Fully distributed, fault tolerant architecture

Feature


Provisioned throughput

Details
Dial up or down provisioned read/write capacity

Predictable
performance
Strong consistency

App Services
Compute

Storage

Database

Average single digit millisecond latencies from SSD
backed infrastructure
Be sure you are reading the most up to date values

Fault tolerant
Monitoring

Networking

Secure
Elastic MapReduce


Data replicated across availability zones
Integrated to Cloud Watch
Integrates with AWS Identity and Access
Management (IAM)
Integrates with Elastic MapReduce for complex
analytics on large datasets

Database

RDS

Dynamo
DB

Redshift

App Services
Compute

Storage

Database

Redshift
Managed Massively Parallel Petabyte Scale Data

Networking

Warehouse
Streaming Backup/Restore to S3
Extensive Security
2 TB -> 1.6 PB

Application Services

CloudFront

3

Served from S3
/images/*

World-wide content distribution
network
Easily distribute content to end users
with low latency, high data transfer
speeds, and no commitments.

2

London

Served from EC2
*.php

Paris

1

Single CNAME

NY

www.mysite.com

App Services
Feature
Compute

Storage

Database

Networking

Fast

Details
Multiple world-wide edge locations to serve content
as close to your users as possible

Integrated with other
services

Works seamlessly with S3 and EC2 origin servers

Dynamic content

Supports static and dynamic content from origin
servers

Streaming

Supports rtmp from S3 and includes support for live
streaming from Adobe FMS and Microsoft Media
Server

Amazon SQS

Processing results

Reliable, highly scalable, queue
service for storing messages as they

Amazon SQS

travel between instances


Processing

task/processing trig

App Services
Feature
Compute

Storage

Database

Reliable
Simple

Networking

Scalable

Secure

Details
Messages stored redundantly across multiple
availability zones
Simple APIs to send and receive messages
Unlimited number of messages

Authentication of queues to ensure controlled access


Simple Workflow

1

Reliably coordinate processing steps
across applications

2

Task A

Integrate AWS and non-AWS resources

Manage distributed state in complex
systems

3

Task B
(Auto-scaling)

Task C

App Services

Feature

Details

Process state
Tracking

Compute

Storage

Maintain application state across complex workflows
in a reliable and available manner
Tracks executions and log process for audit purposes

Database

Networking

Consistency
Simple


Ensures processing tasks are executed and duplicity of
events does not occur
Simple Decider and Task programming model for
rapid integration


Cloud Search

Document
Server

Elastic search engine based upon
Amazon A9 search engine
Fully managed service with

sophisticated feature set

Search
Server

Scales automatically

App Services

Results
Feature
Auto-scaling

Compute

Storage

Database
High performance

Networking

Sophisticated features
Low cost


Details
Automatically scales based upon request volumes and
data volumes
In memory operation means consistently low latency
for search results
Support for faceting, stemming, synonyms, stop
words and custom rank expressions
Elastic service, pay for what you use

Deployment & Admin
Elastic Beanstalk
One-click deployment from Eclipse, Visual Studio and
Git
Rapid deployment of applications
All AWS resources automatically created


Feature
Platform support

App Services

Details
Containers for Java, .net and PHP

Resource creation

Compute

Storage

Database

Creates load balancer, instances, autoscaling and
monitoring automatically

Monitoring & Logs

Integrated with Cloud Watch and consolidates server
logs

Versioning

Networking
Notifications


Full resource access

Manage versions of applications and easily rollback
deployments
Receive alerts on key events
Access all underlying AWS resources as necessary

Deployment & Admin
OpsWorks
DevOps focused managed application stacks
Underlying Chef recipes allow for complete
customisation

Feature
App Services
Compute

Storage

Platform support

Resource creation

Database

Networking

Layered

Details
Chef recipes allows for community expansion for
platform components such as Solr, NgniX etc
Customizable deployments, rollback, partial
deployments, patch management, automatic
instance scaling, and auto healing

Manage logical application layers and combine
into stacks.

Deployment & Admin

Cloud Formation
Automate creation of ‘stacks’ in a repeatable way
Scripting framework for AWS resource creation

Feature
Platform support

App Services

Details
Support for AWS resources from EC2 to IAM

Resource creation

Compute

Storage

Database

Creates AWS resources behind the scenes and reports
on progress

Declarative

Specify stacks in JSON format and source control your
environments

Customizable

Networking

Drive stack creation with paramaters

Deployment & Admin
Identity & Access Management
Granular control of user rights with AWS
Automated granting of EC2 service rights

Software Developer Kits
Comprehensive support of programming models for
App Services
Compute

Storage

Database

Networking

using AWS services

+ others
Simple Email Service
Simple Notification Service
ElastiCache (Memcache & Redis)
Elastic MapReduce
CloudWatch

…and more to come!

Shared responsibility

Amazon

Foundation Services
Compute

Storage

Database

Networking

Availability Zones

AWS Global
Infrastructure

Edge Locations
Regions

Shared responsibility

You

Customer Data
Platform, Applications, Identity & Access Management
Operating System, Network & Firewall Configuration
Client-side Data Encryption & Data
Integrity Authentication

Server-side Encryption
(File System and/or Data)

Network Traffic Protection
(Encryption/Integrity/Identity)

Amazon

Foundation Services
Compute

Storage

Database

Networking

Availability Zones

AWS Global
Infrastructure

Edge Locations
Regions

Security standards
Certifications

Physical Security

HW, SW, Network

SOC 1 Type 2 (formerly
SAS-70)

Datacenters in
nondescript facilities

Systematic change
management

ISO 27001

Physical access strictly
controlled

Phased updates
deployment

Must pass two-factor
authentication at least
twice for floor access

Safe storage
decommission

PCI DSS for EC2, S3,
EBS, VPC, RDS, ELB,
IAM
FISMA Moderate
Compliant Controls

HIPAA & ITAR
Compliant Architecture

Physical access logged
and audited

Automated monitoring
and self-audit
Advanced network
protection

http://aws.amazon.com/security

So what are
we going to
build today?

Instance

Availability Zone

Region

Instance

Availability Zone

S3

Region

Cloud
Front

Instance

Availability Zone

S3

Region

Cloud
Front

S3

Instance

RDS
Availability Zone

Region

Cloud
Front

Elastic Load
Balancer

Instance

Instance

S3

RDS
Availability Zone

Region

Cloud
Front

Elastic Load
Balancer

Instance

Instance

S3

Auto
scaling
Group

RDS
Availability Zone

Region

Enjoy some coffee /
tea,
come back at 11:00,
and we’ll build it

Ask questions
(it will fill time when we wait for
things to launch)
There will be a recording of this
demo on YouTube, so don’t
worry if you miss anything

bootstrapping
passing data to an instance

Instance
request

User
data

Meta-data
service

Instance
request

User
data

Meta-data
service

Instance

Script executed on launch:
<script>
ipconfig /all > c:ipconfig.txt
netstat > c:netstat.txt
</script>

Script executed on launch:
#!/bin/sh
yum -y install httpd php php-mysql
chkconfig httpd on
/etc/init.d/httpd start

security groups
instance firewalling

Port 22
(SSH)
Port 80
(HTTP)

Security Group

instance

Name
Description
Protocol
Port range
IP Address, range,
or another security
group

Public Key
Inserted by Amazon into
each EC2 instance that
you launch

EC2
Instance
Comms secured
with private key

Private Key

Downloaded and stored
by you

index.php
Reads instance meta-data

Some php code that gets the data
<?php
// get the instance id
$url = "http://169.254.169.254/latest/meta-data/instance-id";
$instance_id = file_get_contents($url);

// get the AZ where the instance is running
$url = "http://169.254.169.254/latest/meta-data/placement/availability-zone";
$zone = file_get_contents($url);
// get the security group it is in
$url = "http://169.254.169.254/latest/meta-data/security-groups";
$group = file_get_contents($url);
// get the public DNS name
$url = "http://169.254.169.254/latest/meta-data/public-hostname";
$hostname = file_get_contents($url);
?>

And diplays it

Instance ID: <?php echo $instance_id; ?>
Availability Zone: <?php echo $zone; ?>
Security Group: <?php echo $group; ?>

Amazon RDS
Managed Relational DB

Cloud
Front

Elastic Load
Balancer

S3

Instance

RDS
Availability Zone

Region

Elastic Load Balancer

Instance

Instance

Availability Zone

Instance

Instance

Availability Zone

Region

Instance

Instance

Availability Zone

auto-scaling
elastic server pool

Launch Configuration

Auto-Scaling Group

Auto-Scaling Policy

Describes what Auto
Scaling will create when
adding
Instances

Auto Scaling managed
grouping of EC2
instances

Parameters for
performing an Auto
Scaling action

AMI
Instance Type
Security Group
Instance Key Pair

Automatic health check to
maintain pool size

Scale Up/Down and by
how much

Automatically scale the
number of instances by
policy – Min, Max, Desired

ChangeInCapacity (+/- #)
ExactCapacity (#)
ChangeInPercent (+/- %)

Automatic Integration with
ELB

Cool Down (seconds)

Only one active launch
configuration at a time
Auto Scaling will terminate
instances with old launch
configuration first
rolling update

Automatic distribution &
balancing across AZs

Policy can be triggered by
CloudWatch events

Create a launch configuration:

aws autoscaling create-launchconfiguration
--launch-configuration-name aws-101 -image-id ami-d79b78a0
--security-groups ssh-only
--instance-type m1.small
--region eu-west-1

Create an auto-scaling group:

aws autoscaling create-auto-scaling-group -auto-scaling-group-name aws-101-sg
--region eu-west-1
--launch-configuration-name aws-101
--min-size 0 --max-size 3
--desired-capacity 0
--availability-zones eu-west-1a eu-west-1b
eu-west-1c
--load-balancer-names aws-101

Create an auto-scaling policy (scale up):

aws autoscaling put-scaling-policy
--region eu-west-1
--auto-scaling-group-name aws-101-sg
--policy-name cpu-up
--scaling-adjustment 1
--adjustment-type ChangeInCapacity
--cooldown 0
{
"PolicyARN": "arn:aws:autoscaling:eu-west1:887210671223:scalingPolicy:47a05c37-dc0a-4366-89916272cc1816fd:autoScalingGroupName/aws-101sg:policyName/cpu-up"
}

Create a scaling Trigger from CloudWatch
aws cloudwatch put-metric-alarm
--alarm-name aws-101-scale-up
--metric-name CPUUtilization
--namespace "AWS/EC2"
--statistic Average
--threshold 80
--period 60
--comparison-operator
GreaterThanOrEqualToThreshold
--evaluation-periods 2
--alarm-actions arn:aws:autoscaling:eu-west1:887210671223:scalingPolicy:47a05c37-dc0a4366-89916272cc1816fd:autoScalingGroupName/aws-101sg:policyName/cpu-up
--region eu-west-1

To do this you will need:
An AWS Account
Auto-scaling tools installed

Find out more:

aws.amazon.com

AWS 101 Event December 2013

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (19)

Ähnlich wie AWS 101 Event December 2013

Ähnlich wie AWS 101 Event December 2013 (20)

Mehr von Amazon Web Services

Mehr von Amazon Web Services (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

AWS 101 Event December 2013