Learn about Amazon Elastic Block Store (EBS) and how to get consistent High IO performance using Provisioned IOPS volumes. Also hear from CopperEgg about how they use EBS to get the consistent high IO for their service

1. Consistent High I/O Performance
with Amazon Elastic Block Store
Jafar Shameem
Business Development Manager, Amazon Web Services
Eric Anderson
CTO and Co-Founder, CopperEgg

2. Agenda
• AWS Storage Options and EBS
• Introducing Provisioned IOPS
• Maximize your application’s IO through
Provisioned IOPS
• How CopperEgg benefits from Provisioned IOPS

3. Storage Options on AWS
Block Storage Object Storage
(Elastic Block Store) (S3, Glacier)
Use for: Use for:
• Access to raw • Pictures, videos, hi
unformatted block ghly durable media
level storage storage
• Persistent Storage • Cold storage for
long-term archive

4. Amazon Elastic Block Store (EBS)
Elastic Block Storage: Persistent Storage for EC2
Feature Details
High Mount EBS as drives and
performance format as required
file system
Flexible size Volumes from 1GB to 1TB in
size
Secure Private to your instances
Available Replicated within an
High performance block storage Availability Zone
device Backups Volumes can be snapshotted
Mount as drives to instances for point in time restore
Persistent and independent of
Monitoring Detailed metrics captured via
instance lifecycle Cloud Watch

7. $ sudo mdadm --create /dev/md0 --level=0 --chunk=64 --raid-devices=3 /dev/sdf /dev/sdg
/dev/sdh

9. Introducing
Provisioned
IOPS Volumes

10. Introducing Provisioned IOPS Volumes
❶ Select a new type of Provisioned IOPS volumes
❷ Specify the volume capacity
❸ Specify the number of IOs per second your
application needs, up to 2000 PIOPS per
volume. The volume will deliver the specified
IO per second.
$ ec2-create-volume --size 500 --availability-zone us-east-1b --type io1 –iops 2000

11. Consistent IO performance
Provisioned IOPS volumes deliver within 10% of the IOPS
performance 99.9% of the time over a given year.

12. When should I use Provisioned IOPS?
• Standard volumes are designed for boot volumes, running
applications with moderate and bursty IO, sequential IO access
• Provisioned IOPS volumes are
designed for running transactional
applications that require high and
consistent IO:
o Relational Databases
o NoSQL Databases, e.g. MongoDB
o Productivity applications, e.g.
Microsoft Exchange
o Enterprise Applications

13. Simple pricing

14. Getting more out of
Provisioned
IOPS
Volumes

15. Tip # 1: Use EBS-Optimized instances

16. Tip #2: Understanding Provisioned IOPS
Volume Throughput

17. Tip #3: Stripe multiple volumes for
more IOPS
sudo mdadm --verbose --create /dev/md0 --level=10 --chunk=256 --raid-
devices=4 /dev/sdh1 /dev/sdh2 /dev/sdh3 /dev/sdh4

18. Tip #4: Achieving consistent performance
• There is a 5 to 50 percent reduction in IOPS when you first access
the data on a volume.
• $ dd if=/dev/md0 of=/dev/null
• To minimize the impact of snapshots on performance of a master
node, create snapshots from a read replica of your data.
Ideally, create these snapshots during off-peak usage.

19. Tip #5: Understanding Queue Depth
Maintain a number of pending I/O requests to get the most out of your Provisioned
IOPS volume. Queue depth of 5 per 1000 IOPS.

20. How CopperEgg
uses Provisioned
IOPS

21. CopperEgg’s IO Use Case
• Large metric capture
volumes
• Real-time dashboards
• Historical data analysis
• Durability/Availability
• Real-time alerting

22. How CopperEgg uses Provisioned IOPS
• Used in our main metric data store
• Stores Egg-zillions of metrics
• Rock-solid predictable IO that’s affordable
• Snapshots for backups
• XFS file system on top

23. CopperEgg’s IO Demands
• Store many terabytes of data
• Persist the data over long periods of time
• Backups (use snapshots)
• High IO: 10-15k+ ops/s per node minimum
• Consistent IO behavior (non-spikey)

24. CopperEgg IO Sweet Spot
Data
Persistence
Sweet spot
Cost Stability
IO
Performance
Predictability

25. Provisioned IOPS hits the spot
• High Performance: get what you ask for
o More than 20,000 IOPS on a single node
• Extremely predictable IO characteristics
• Cost is reasonable (and predictable!)
• Same persistence as standard EBS
• Data replicated in the same AZ just like Standard
EBS

26. Cost Effective?
• For steady IO
use patterns, it
is!
• For bursty
IO, do the math

27. CopperEgg’s Provisioned IOPS rule of
thumb
• Need predictable performance? Provisioned IOPS
• Less than 100 ops/s on average? Standard EBS
• Need more than 100 ops/s per volume?
Provisioned IOPS

28. Get started on Provisioned IOPS
today!
aws.amazon.com/ebs
Questions: e-mail: shameemj@amazon.com
• Amazon Provisioned IOPS
o http://copperegg.com/amazon-provisioned-iops-ebs/
• Benchmarking EBS performance:
o http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSPerfo
rmance.html
• Stratalux: Putting Amazon’s Provisioned IOPS to the test
o http://www.stratalux.com/2012/08/09/putting-amazon’s-
provisioned-iops-to-the-test/