With AWS, you can choose the right storage service for the right use case. Given the myriad of choices, from object storage to block storage, this session will profile details and examples of some of the choices available to you, with details on real world deployments from customers who are using Amazon Simple Storage Service (Amazon S3), Amazon Elastic Block Store (Amazon EBS), Amazon Glacier, and AWS Storage Gateway.

1. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Storage & Archiving: Choosing the
Right Data Storage Solution
Timothy K. Harder, Global Business Development
harder@amazon.com March 26, 2014

2. Agenda
• Common Data Storage Terms
• Introduction to AWS Storage Platforms
• Use Cases Drive Platform Selection
• In their own words : Mapbox.com
• TCO
• What to do next

3. Goals for this session
• 101 Level Introduction to AWS Data Storage
– Survey of AWS Data Storage Platforms
– Learn how AWS Cloud Storage compares to classic models
– Cloud Storage Use Cases
– How to get started

4. AWS Storage Options
• Simple Storage
• General Purpose Storage
• Archive Storage
• Storage Gateway

5. We are constantly producing more data

6. From all types of industries

7. Classic and Contemporary
Applications
File Systems
Database(s)
User Generated Content
Machine Generated Content
Messaging, Collaboration
Backup, Archive, DR

8. First, let’s visit some storage basics

9. Storage Basics - Block vs. File vs. Object
• Block Storage : SAN : EBS
– Data organized as an array of unrelated blocks
– Host File System places data on disk: Microsoft NTFS or Unix ZFS
– Structured data is predicted to grow at 34% CAGR until 2018
• File Storage : NAS :EBS
– Unrelated data blocks managed by a file (serving) system
– Native file system places data on disk: EXT or Filers
– Unstructured data is predicted to grow at 45.6% CAGR until 2018
• Object Storage :S3:RRS:Glacier
– A new data access, data storage, and data management model
• API access to data vs. traditional block or file system access
• Metadata Driven, Policy-based, Self Managing storage
• No host overhead for storage functions
– A system that stores virtual containers that encapsulate the data, data
attributes, metadata, and Object IDs

10. Storage Basics – SAN, NAS , Object
SERVERS
Object
NAS
SAN
Datacenter Proximity
Ethernet / Optical
Datacenter Proximity
Ethernet
Cloud Proximity
API / HTTP(S)
FILESYSTEMS
APPLICATIONShttps://
nfs / cifs
FC / iSCSI / FCoE
• SAN and NAS are integrated with clients via operating system
• Cloud Storage is an application level integration

11. No Up-Front
Capital Expense
Pay Only for
What You Use
Self-Service
Infrastructure
Easily Scale
Up and Down
Improve Agility &
Time-to-Market
Low Cost
Amazon Storage Service Benefits

12. AWS Global Infrastructure
10 Regions
25 Availability Zones
51 Edge Locations

13. AWS Regions and Availability Zones
You decide where your data resides

14. Storage Services
Amazon EBS
EBS
Block storage for use
with Amazon EC2
Amazon S3
Images
Videos
Files
Binaries
Snapshots
Internet scale
storage via API
AWS Storage Gateway
S3,
Glacier
Integrates on-premises
IT and AWS storage
Amazon Glacier
Images
Videos
Files
Binaries
Snapshots
Storage for archiving
and backup

15. #1
Object Storage
●○○○○

16. AMAZON S3
SIMPLE STORAGE SERVICE

17. 99.999999999%
Durability

18. Trillions
Of Unique Customer Objects

19. 1.5 Million+
Peak Transactions Per Second

20. Storage Tiers: Buckets + Unlimited Objects

21. Reduced Redundancy Option
99.99% saves 20-30%

22. Amazon S3 Website: Content

23. Use Amazon S3 When You Need
• Application level connectivity
• Unlimited storage capacity
• Extremely High durability
• Storage for user or machine generated content
• Content Depots and Data Lakes
• Gold copy, Backup and Data Life Cycle

24. #2
Block Storage for
General Computing
●●○○○

25. AMAZON EBS
ELASTIC BLOCK STORAGE

26. 10GB
1TB

27. IOPS
Provisioned
4000
IOPS

29. Amazon EBS
Snapshots

31. Use Amazon EBS When You Need
• Persistent storage
• Data changes frequently
• Block storage for your databases – Provisioned IOPS
volumes
• Filesystem enablment NTFS, ExtFS, RAID, LVM…
• Access to raw, unformatted block-level storage

32. AMAZON GLACIER
LOW-COST ARCHIVING SERVICE

33. $0.01
Per GB / Month

34. $120
Per TB / Year

35. 99.999999999%
Durability

36. 3-5 Hours
Data Retrieval

37. Use Amazon Glacier When You Need
• Inexpensive, Long-term archiving. Cold Use.
• Unlimited storage capacity
• Eliminated Tape Museums
• Eliminate Tech Refresh
• High durability

38. Amazon S3 / Amazon Glacier Integration
POLICY-BASED ARCHIVING SERVICE

39. Lifecycle Rule

40. Archive Recovery Process with Tape
+ Days or Weeks

41. Archive Recovery Process with AWS &
Amazon Glacier
$$
Hours
Glacier S3 EC2
/HPC
CloudFront Generating
Business
Value

42. Use Amazon S3 and Amazon Glacier When You Need
• Backup and Archive in the cloud
• Archive data from Amazon S3/RRS to Amazon Glacier
by policy
• Delete data from Amazon Glacier by policy
• Tiering Strategy to support Information Lifecycle
Management

43. AWS STORAGE GATEWAY

44. The Good News is that Cloud isn’t an ‘All or Nothing’
Corporate
Data
Centers
On-Premises
Resources
Cloud
Resources
Integration

45. Active Directory
Network Configuration
Encryption
Backup Appliances
Your On-Premises
Apps
Corporate
Data Centers
Users & Access Rules (IAM)
Your Private Network (VPC)
Encryption (S3, RDS, HSM)
Backups (Storage Gateway)
Your Cloud Apps
AWS Direct Connect
Integrating AWS with your existing On-Premises
Infrastructure

46. AWS Storage Gateway

47. What is AWS Storage Gateway?
• Integrates on-prem IT environments with Cloud
storage for departmental and remote office backup
and DR
• Utilizes a virtual appliance that sits in your
datacenter
• Exposes storage and backup interface on front end
• Stores primary data on-AWS in Amazon S3 or on-
premise with data backed-up to Amazon S3 as
Amazon EBS snapshots or Glacier

53. Scale: for hundreds of millions of users
Global: anywhere in the world
Speed: as fast as Google
Mobile: on any device

56. The future of mapping is open.
OpenStreetMap provides
coverage with street level detail

57. OpenStreetMap is
constantly improving.
Total users
1 million
as of January 2013

58. Mapbox Satellite

59. We have 4.2 billion satellite
images (map tiles) on S3

64. We need: - Always hot (fast) - Fast
anywhere in the world- Redundant -
Geo-redundant - Integrated into our
processing pipeline- Highly scalable

65. Integrated processing pipeline

77. hmmm, this is
slow going upgrade
EC2 type
w00t!
killing it
spiked regional
spot pricing :p
increases $ for
spot pricing

78. Enterprise Scale

80. Launch
Imagery refresh
Wired

81. 160,000
(views per hour)

82. aka 240,000,000
tiles/ hour

83. +600tiles/ second

84. ENTERPRISE

85. IT’S ALL ABOUT
CHOICEPERFORMANCE-ORIENTED
COST-ORIENTED

86. Trusted by Enterprises Around the World

87. Used by Government Agencies & Educational Institutions
Worldwide

88. 2014 Premier Tier Partners

89. TCO
• AWS Storage Platforms Provide Lowest TCO by
13-44% as compared to classic deployments.
We take cost out for customers in every
engagement.
• When is your next capacity upgrade or refresh?

90. Goals for this session
• 101 Level Introduction to AWS Data Storage
– Survey of AWS Data Storage Platforms
– Learn how AWS Cloud Storage compares to classic models
– Cloud Storage Use Cases
• How to get started
– AWS.AMAZON.COM/FREE
– AWS.AMAZON.COM/BACKUP-STORAGE

91. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Storage & Archiving: Choosing the
Right Data Storage Solution
Timothy K. Harder, Global Business Development
harder@amazon.com March 26, 2014
Thank you!