Amazon Elastic Block Store for Application Storage

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Amazon Elastic Block Store for Application Storage
David Stein
Business Development Manager

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Cloud Data Migration
Direct ConnectSnow* data
transport
family
3rd Party
Connectors
Transfer
Acceleration
Storage
Gateway
Kinesis Firehose
AWS Storage Platform and Solutions
Object
Amazon GlacierAmazon S3
Block
Amazon EBS
(persistent)
Amazon EC2
Instance Store
(ephemeral)
File
Amazon EFS

AWS
block storage
offerings
EC2 instance
store
I2/I3 D2

What is Amazon EC2 instance store?
EC2 instances • Local to instance
• Non-persistent data store
• Data not replicated (by default)
• No snapshot support
• SSD or HDD
Physical Host
Instance Store
or

AWS
block storage
offerings
EC2 instance
store
sc1st1
io1gp2
EBS
SSD-backed
volumes
EBS
HDD-backed
volumes
I2/I3 D2

What is EBS?
EBS
volume
EC2
instance
• Block storage as a service
• Create, attach volumes through an API
• Service accessed over the network

What is EBS?
EBS
volume
EC2
instance
!=

What is EBS?
EBS
volume
EC2
instance

What is EBS?
EBS
volume
Availability Zone
AWS Region
EC2
instance

What is EBS?
EBS
volume
EC2
instance
EC2
instance
• Volumes persist independent
of EC2
• Detach and attach between
instances
• Volume and instance must be
in the same AZ
Availability Zone
AWS Region

What is EBS?
EBS
boot
volume
Availability Zone
AWS Region
EC2
instance
EBS
data
volume
EBS
data
volume
• Volumes attach to one
instance at a time
• Many volumes can attach to
an instance
• Separate boot volume from
data volumes

EBS is designed for:
What is EBS?
99.999% service availability
0.1% to 0.2% annual failure rate (AFR)

What is an EBS snapshot?
EBS
volume
Availability Zone
AWS Region
Amazon
S3
EBS snapshot
Availability Zone
Replica

How does an EBS snapshot work?
EBS
volume
• Point-in-time backup of modified volume blocks
• Stored in S3, accessed via EBS APIs
• Subsequent snapshots are incremental
• Deleting snapshot will only remove data exclusive to
that snapshot
EBS
snapshot

What can you do with a snapshot?
EBS
volume
Availability Zone
AWS Region
EC2 instance
EBS snapshot
AMI

EBS
volume
Availability Zone
AWS Region
Amazon
S3
EBS snapshot
Availability Zone
EBS
volume
Replica Replica

EBS
volume
Availability Zone
AWS Region
Amazon
S3
EBS snapshot
EBS
volume
Availability Zone
AWS Region
EBS snapshot
Replica Replica

AWS Region
Public datasets on
AWS available as EBS
snapshots:
Availability Zone
EBS
volume
https://aws.amazon.com/public-data-sets/
• Genomic
• Census
• Global weather
• Transportation
Replica

What is an EBS-optimized instance?
EBS
volume
Availability Zone
AWS Region
EBS-optimized
EC2 instance

EBS
EC2
instances
Internet
Databases
~ 125 MB/s
S3
Shared
c3.2xlarge

EBS
EC2
instances InternetDatabases
c4.2xlarge
S3
~ 125 MB/s
Shared

More details:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSOptimized.html
• Dedicated network bandwidth for EBS I/O
• Enabled by default on c4, d2, m4, p2, and x1 instances
• Can be enabled at instance launch or on a running instance
• Not an option on some 10 Gbps instance types (c3.8xlarge,
r3.8xlarge, i2.8xlarge)

What is EBS encryption?
Encryption
• Attach both encrypted and unencrypted
• No volume performance impact
• Any current generation instance
• Supported by all EBS volume types
• Snapshots also encrypted
• No extra cost
• Boot and data volumes can be encrypted

What is the Elastic Volumes feature?
• Simple; Flexible; Non-disruptive; Automated
• Modify the configuration of live volumes attached to instances
• Dynamically increase size, tune performance, and change the type of existing and new
current generation volumes
• No downtime, no performance impact.
• You can automate changes using CloudWatch with Lambda or CloudFormation
• No need to plan ahead, provision what you need today and change the configuration as
business needs change.

Elastic Volume – Limitations
• Limit – You can modify a volume once per 6 hours
• Limit – Supported only for current generation volumes (gp2/io1/st1/sc1). Not supported
for Magnetic/Standard volumes
• Limit - Live changes supported for volumes attached to current generation instances

How does it work?
• Three steps
– Issue the modification command
– Monitor the progress of the modification
– If size change extend the volume's file system

Recap: Benefits of EBS
• Persistent – delete instances without losing your data, detach and reattach volumes
without having to replicate data between nodes
• Elastic –dynamically increase capacity, change volume types, and tune performance
• Available – designed for 99.999% availability
• Durable – designed for an annual failure rate (AFR) of between 0.1% - 0.2%. EBS
Snapshots for backups.
• Secure – seamless support for data-at-rest and data-in-transit encryption between
EC2 instances and EBS volumes

EBS volume types

EBS volume types
Hard disk driveSolid state drive

EBS volume types
General Purpose
SSD
gp2
Provisioned IOPS
SSD
io1
Throughput Optimized
HDD
st1
Cold
HDD
sc1
SSD HDD

Relational Databases
Relational – MySQL,
PostgreSQL, SQL,
Oracle, SAP
Media
Transcoding,
Encoding,
Rendering, File
Big Data
Hadoop/EMR, Logs,
Stream Processing,
Data Warehousing
NoSQL Databases
Cassandra, MongoDB,
CouchDB, Aerospike
Solid state drive (SSD) Hard disk drive (HDD)
EBS Use Cases

Throughput?
or
IOPS
Choosing an EBS volume type
What is more important to your workload:

Don’t know
Choosing an EBS volume
your workload
yet?

EBS volume types: I/O Provisioned
General Purpose SSD
gp2
Throughput: 160 MB/s
Latency: Single-digit ms
Capacity: 1 GB to 16 TB
Baseline: 3 IOPS per GB up to 10,000
Burst: 3,000 IOPS (for volumes up to 1 TB)
Great for boot volumes, low-latency applications, and bursty databases

Burst & baseline: General Purpose SSD (gp2)IOPS
0 1 16
1,000
2,000
3,000
8,000
10,000
BASELINE IOPS
(Baseline of 3 IOPS/GB)
Burstable to
3,000 IOPS
3 90.5
Volume size (TB)
~ 3334 GB
100 IOPS
Minimum
300 GB volume

i2
gp2 io1
Latency ?
< 1 ms Single-digit ms
Which is more important ?
Cost Performance
IOPS
≤ 80,000> 80,000
is more important

EBS volume types: I/O Provisioned
Provisioned IOPS SSD
io1
Baseline: 100 to 20,000 IOPS
Throughput: 320 MB/s
Latency: Single-digit ms
Ideal for critical applications and databases with sustained IOPS

Scaling Provisioned IOPS SSD (io1)IOPS
0 2 16
1,000
5,000
10,000
15,000
20,000
6 90.4
MAX PROVISIONED IOPS
(Maximum IOPS:GB ratio of 50:1)
Available Provisioned IOPS
Volume Size (TB)
~ 400 GB

i2
gp2 io1
Latency ?
< 1 ms Single-digit ms
Cost Performance
IOPS
≤ 80,000> 80,000
is more important
Throughput?

Throughput
is more important
Small, random I/O Large, sequential I/O
i2
gp2 io1 st1
d2
Latency ?
< 1 ms Single-digit ms ≤ 1,750 MB/s
Aggregate throughput?
> 1,750 MB/s
Cost Performance
IOPS
≤ 80,000> 80,000
is more important
Cost Performance

EBS volume types: Throughput Provisioned
Throughput
Optimized HDD
st1
Baseline: 40 MB/s per TB up to 500 MB/s
Burst: 250 MB/s per TB up to 500 MB/s
Ideal for large-block, high-throughput sequential workloads

Throughput Optimized HDD – burst and base
0
100
200
300
400
500
600
0.5 1 2 4 6 8 10 12 14 16
ThroughputinMB/s
Volume Size in TB
Burst Base
320
ST1

Throughput
is more important
Small, random I//O Large, sequential I/O
Which is more important?
Latency?
i2
gp2 io1 sc1 st1
d2
IOPS
≤ 80,000> 80,000
> 1,750 MB/s
is more important
Cost Performance
Cost Performance

Cold HDD
sc1
EBS volume types: Throughput Provisioned
Baseline: 12 MB/s per TB up to 192 MB/s
Burst: 80 MB/s per TB up to 250 MB/s
Ideal for sequential throughput workloads, such as logging and backup

Cold HDD – burst and base
0
50
100
150
200
250
300
0.5 1 2 4 6 8 10 12 14 16
ThroughputinMB/s
Volume size in TB
Burst Base
192
SC1

Throughput
is more important
Small, random I/O Large, sequential I/O
Latency?
i2
gp2 io1 sc1 st1
d2
IOPS
≤ 80,000> 80,000
> 1,750 MB/s
is more important
Cost Performance
Cost Performance

I/O Provisioned Volumes Throughput Provisioned Volumes
sc1st1io1gp2
$0.10 per GB $0.125 per GB
$0.065 per PIOPS
* All prices are per month, and from the us-west-2 Region as of April 2016
$0.045 per GB $0.025 per GB
Snapshot storage for all volume types is $0.05 per GB per month

EBS Use Case: MySQL
• Use Case
• India’s leading taxi aggregator mobile application
• Amazon EBS and Elastic Volumes for MySQL database
• Value Proposition
• Flexibility: Scale volume capacity as needed, scale performance, automated through Lambda
• Cost: 20-30%, $50k/month savings
• Data availability: EBS is automatically replicated for availability, no replicating data between
nodes
• Data Persistence: Detach and reattach volumes to resize your cluster
• Ola Example
• Ola chose EBS GP2 and IO1 volumes for it’s MySQL databases
• AWS is helping the innovative startup realize its ambitious mission: “We want to build mobility for a
billion Indians,” says Jain. “We want to make transportation hassle free for everyone.”
“By using Amazon EBS, we can scale volume capacity as needed up to 16TB and dynamically increase or
decrease provisioned IOPS.”

EBS Use Case: Cassandra (NoSQL)
• Use Case
• EBS GP2 (SSD), ST1 and SC1 Hard Disk Drive (HDD) volumes
• Faster performance, agility, and scalability for their real-time cloud-monitoring solution
• Flexibility: Right size instance - CPU and Memory, not storage. With EBS, choose any instance
• Cost: Reduced cost 35%
• Backup: Schedule EBS Snapshots for consistent backups
• Data Persistence: Replacing failed nodes now < 15 minutes. Full ringscale up now <1 hour.
• Monitoring: Get better insights into disk usage with EBS volumes
• Librato Example
• Chose V4.4xlarge + GP2, ST1 and SC1 vs. I2
““We have reduced write latencies across our Cassandra rings by up to 500 milliseconds simply by moving from
I2 to C4 instances with Amazon EBS volumes.That has also contributed to reduced latency for customers when
they interact with our API, giving them faster time to data analysis.”

EBS Use Case: Cassandra (NoSQL)
• Use Case
• Elasticity on demand at a cost-effective endpoint to analyze PB of network traffic
• Process, store, and analyze billions of network packets (metadata) and PBs of raw packet data
• Cost: Less expensive than on-prem, saved 30% costs by leveraging multi-tiered storage system
with EBS
• Performance: Search petabytes of stored data with 1-3 second response time
• Data Persistence: Detach and reattach volumes to resize your cluster, minutes to move data that
used to take days or weeks
• Protectwise Example
• Chose I2 EC2 (hot data) + GP2, ST1 EBS (warm data), and S3 (cold data)
““Using AWS, we have reduced our storage costs by 95%, meaning we are spending $1 for every $20 we would have spent
on a traditional system.”

EBS Use Case: Big Data
• Use Case
• Big data ELK and Splunk to process application logs for their cloud based support platform.
• Cost: Reduced cost 60%
• Retention: Increased data retention 3x
• Zendesk Example:
““We have a more flexible way to manage our logging solution using multiple types of Amazon EBS volumes.
That helped us save more than 60% of the overall costs of running the solution.”
gp2 st1
Hot data
0–7 Days
Warm data
8–30 days
sc1
Cold data
31–60 days

EBS Use Case: Hadoop (Big Data)
• Use Case
• EBS GP2 (SSD), ST1 and SC1 Hard Disk Drive (HDD) volumes
• Cloudera Hadoop to analyze 15TB of data / day so their customers can run advertising campaigns
• Cost: More capacity and less processing at a lower price per request
• Elastic: increase capacity, tune performance, and change volume types live (no downtime)
• Operational Efficiency: Easy to expand, no more babysitting cluster
• Videology Example
• Chose M4.10xlarge + ST1 HDD Throughput Optimized EBS
““We were able to save $15,000 per month, increase our available storage by 5%, and turn off
eight server nodes by moving to Amazon EBS ST1 volumes to support our Hadoop cluster”

Hybrid volume use cases
gp2
st1
Amazon EMR Apache Hadoop Example
Frameworks on YARN
HDFS
sc1
EMR cluster
instance
• Random, small I/O
• Shuffle, spill, and temp operations
• Large, sequential I/O
• Multiple volumes for more parallelism
or

EBS Use Case: Splunk (Big Data)
• Use Case
• Born in the cloud cyber defense solution
• Process, store, and analyze PB of data
• Cost: Less expensive than on-prem
• Shell Example
• Search Nodes: 4 x search nodes have 500GB GP2 for dispatch handling and splunk handling
• Index nodes: 40 x EC2 nodes with 10GB GP2 volumes (warm) + 10TB SC1 (cold), 800TB total

EBS Use Case: Kafka (Big Data)
• Use Case
• Kafka for internal streaming data framework to ingest 17TB of data per day
• Push data to Cloudera Impala Hadoop cluster and time series database for analysis
• Cost: Less expensive than on-prem, GP2 volumes, and D2 instances
• Performance: more consistent IO performance, high throughput
• Data Persistence: Detach and reattach volumes to resize your cluster.
• ViaSat Example
• Chose 30 x M4.2slarge + 6TB EBS ST1 vs D2 instance store or GP2 volumes

EBS Use Case: Media & Entertainment
• Use Case
• Ability to handle large files, high IO with high availability in a bursty environment
• Transcode media to multiple formats, add watermarking and perform additional workflows
• Cost: Less expensive than on-prem,
• Performance: Have seen above 2.5 Gbps in and out of filer infrastructure
• Throughput: ST1 ideal for sequential workflow. Write once, read many.
• Data Persistence: Detach and reattach volumes to resize your cluster, minutes to move data that
used to take days or weeks
• Pix Systems Example
• Chose C4.2xlarge with 8TB SC1 volumes for high throughput

Hybrid volume use cases
st1
Case Study:
Info: https://aws.amazon.com/solutions/case-studies/infor-
ebs/
Transaction logs
“We’ve seen much stronger performance for our database backup
workloads with the Amazon EBS ST1 volumes, and we’re also saving
75 percent on our monthly backup costs.”
Randy Young, Director of Cloud Operations, Infor
i2
st1
Full backups
st1
Partial backups
SQL Server
Database
EBS
snapshots

Summary
Use encryption if you
need it
Take snapshots,
tag snapshots
Select the right
instance for your
workload
Select the right
volume for your
workload

Amazon Elastic Block Store for Application Storage

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Amazon Elastic Block Store for Application Storage

Similar to Amazon Elastic Block Store for Application Storage (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Amazon Elastic Block Store for Application Storage