This document provides an overview and summary of AWS storage services that can be used for migrating data to AWS. It discusses AWS Snowball and Snowmobile appliances that can physically move large amounts of data to AWS storage services like S3. It also describes the AWS Storage Gateway, which allows on-premises applications to access AWS storage using standard storage protocols. Additional services covered include Amazon Kinesis Firehose for loading streaming data, AWS Direct Connect for private connectivity, and AWS Migration Hub and Application Discovery Service for discovery and tracking of servers and databases during migration.

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ric Harvey
Technical Evangelist, Amazon Web Services
@ric__harvey
AWS Storage and Edge Processing

2. 160 ZB by 2025
- IDC
847 ZB/YR by 2021
- Cisco
1 Zettabyte = 1021 = 1 000 000 000 000 000 000 000 bytes

3. Data
Data in the cloud has gravity
Deliver new
insights
(data lakes, analytics)
Accelerate
innovation
(active archive, IoT,
Artificial Intelligence)
Realize benefits
(cost, management, scale)
Build or migrate
an application

4. AWS storage customers

5. Compliance
Industry
certifications
Lockable with audit
trails
Secure
Enterprise
Applications
Easier lift-and-shift
migrations
Integrated with
major vendors
Fully managed
infrastructure
Active
Archive
Media workflows
Tape replacement
Public Sector,
FinServ,
Healthcare/Life
Sciences
Databases &
Analytics
Tailored database
or Hadoop
workloads
Bespoke database
lift-and-shift
projects
Backup and
Restore
Non-disruptive
Easy place to start
Integrated with all
major vendors
Data Lakes
& IoT
400% faster
queries
Built for
streaming data
Optional data
visualization
Common storage workloads on AWS

6. Why AWS Storage?
The best reliability and largest scale The most complete portfolio
The most data movement choices The most comprehensive
support and consulting
More than twice the partners
The most secure,
compliant, and auditable

7. Complete set of building blocks
Data movement Data security
and management
File
Storage
Block
Storage
Object
Storage
Archival
Storage
Encryption
Access Controls
Monitoring and Metrics
Audit Trails
Automation
Serverless Computing
Data Discovery and
Protection
Data Visualization
Physical Appliances
Hybrid Storage
Private Networks
File Data
WAN Acceleration
Third-party
Applications
Streaming Data

8. AWS storage services
Data movement
OnlineOffline
Data security
and management
Amazon
EFS
Amazon
EBS
Amazon
S3
Amazon
Glacier
AWS KMS
AWS IAM
Amazon CloudWatch
AWS CloudTrail
AWS CloudFormation
AWS Lambda
Amazon Macie
Amazon QuickSight
AWS Snow Family
AWS Storage Gateways
AWS Direct Connect
Amazon EFS File Sync
Amazon S3 Transfer
Acceleration
Third-party
Applications
Amazon Kinesis Firehose

9. Enterprise
Applications
Disaster Recovery Analytics
Primary Storage Backup and Restore Archive
Many partnerships
Complete partner list at https://aws.amazon.com/backup-recovery/partner-solutions/

10. Experienced training, support, and consulting
Helping organizations
adopt AWS through:
- Digital courses
- Classroom training
- Certification exams
Operation:
AWS Training
Support packages for
environments that are:
- Experimental
- Production
- Business-critical
Escalation:
AWS Support Plans
Supplemental, specialized
experience and skills:
- APN Consulting Partners
- AWS ProServe
- AWS Managed Services
Customization:
AWS Professional Services

11. Amazon S3

12. 2017 Gartner Magic Quadrant
- Gartner Magic Quadrant for Public Cloud Storage Services, Worldwide
Raj Bala, Arun Chandrasekaran, John McArthur, July 24, 2017
“AWS sets the boundaries in the market
for public cloud storage services
by which all other vendors operate.”

13. Reliability and Scale
“…the scale at which AWS operates its public
cloud storage services dwarfs the other vendors
in this Magic Quadrant.”
- Gartner Magic Quadrant for Public Cloud Storage Services, Worldwide
Raj Bala, Arun Chandrasekaran, John McArthur, July 24, 2017
For example: Amazon S3 holds trillions of objects and
regularly peaks at millions of requests per second
TIME
OBJECTS

14. Amazon S3
Analyze
Store
Collect
Built for:
backup and restore, data lakes, analytics, cloud-native applications
• More than a decade of experience and continuous innovation
• Only AWS has the infrastructure to place storage near workloads
• Only AWS gives storage admins granular object controls
• Only AWS moves data in so many varied ways
• Only AWS storage can help address CISO concerns
• Only AWS can analyze and recommend cost savings
• Only AWS can accelerate application performance up to 400%
• Only AWS offers inventory and visualization across entire datasets
• Only AWS supports queries across structured and unstructured data

15. Object storage classes
S3 Standard GlacierS3 Standard -
Infrequent Access
Active data
Milliseconds
$0.023/GB/mo
Archive data
Minutes to Hours
$0.004/GB/mo
Infrequently accessed data
Milliseconds
$0.0125/GB/mo
Automated Lifecycle Policies

16. Amazon Glacier
Cost-
effective
Secure
Durable
Built for:
Active archive, tape replacement, regulatory compliance
• Certifications supporting nearly any regulatory compliance program
• Locking, encryption, audit and alerting tools to prevent tampering
• Built on the most reliable global infrastructure
• Withstands multiple facility failures
• Replication options across global regions
• Designed for archives and backup
• Expedited retrievals in minutes, bulk retrievals in hours
• Opens archives to analytics applications

17. Amazon S3 and Glacier Durability
Designed for
99.999999999% durability
GlacierS3 Standard S3 - IA
OR:
99.999% durability
99.99% durability
Traditional model with two
copies on one site
Traditional model with copies on
two sites
S3 – IA OZ

18. Amazon Object Storage Availability and Durability
“Zones”
Or worse, this:
AWS Region
This:
Availability Zone
Availability Zone
Availability Zone
Not this:
“Region”

19. S3 Use Cases

20. Security-as-a-Service for 4000 customers
using 25 PB and growing 110% per year
Colocate simply not agile enough or cost effective
• Built an Amazon S3 data lake and avoided
$1.6M CAPEX - in the first year alone
• Stress-tested 100x larger load with zero CAPEX
• 4x better “I/O per $” ratio
• Gained new insights into their customers
through Amazon S3 data management
capabilities
• No 40-Gbps network infrastructure worries
“AWS storage is
Fully redundant, multi-region,
Mmore secure, and faster
at less than half the cost.”
- Paul Fisher
Technical fellow
S 3 D A T A L A K E
Data Lakes

21. Threat analysis company ingesting
and analyzing 50 TB daily
Right-sizing clusters cost weeks and lost data
• Saved 95% through re-architecting to a “hot”
index on Amazon EBS with an analytics data
lake on Amazon S3
• Amazon EBS shortened indexing times from
weeks to hours while cutting OPEX
• Now getting consistent 1–3 sec. search
response times across 5 PB of growing data in
Amazon S3
• Managing 1 billion Amazon S3 objects and
2,500 instances with just six engineers
“AWS storage completely changed our
business operations, time to market and
manpower. EBS volumes cut our cluster
indexing times from weeks to hours. Moving
data into Amazon S3 saved us 95% and our
data lake now outperforms our clusters—the
harder we push it the faster it gets for
extremely large datasets. We simply could not
do this anywhere else.”
- Gene Stevens
CTO and cofounder
Data Lakes
A m a z o n S 3 D A T A L A K E

22. Amazon Elastic Block Store (EBS)

23. Amazon EBS
Performant
Persistent
Reliable
Built for:
Hadoop/Amazon Amazon EMR, relational and NoSQL databases, log processing,
and data warehousing
• Dedicated, detachable volumes for EC2 instances
• Helps customers manage compute and storage separately
• Highly secure Multi-AZ design
• Built-in backup options
• Performance options to fit most workloads
• Optimized for latency, throughput, or cost
• Elastic volumes expand capacity on the fly

24. Amazon EBS Use Case

25. Databases and analytics
Global broadband service operator processing
17 TB of daily device data streams at 200 MB/s
Modifying Kafka clusters required an
8 hours resync every time
• Moved from instance stores to EBS volumes
• Cut storage costs by 25%
• Cut production cluster node count by 33%
• Dropped resync times to 20 minutes
“Our AWS service use is about making the
necessary easy. Storage should be as boring
as possible—it should just work. Amazon
EBS makes it trivial to do things that were
impractical before, driving experimentation,
creativity, and faster delivery.”
- Daniel Woodlins
Software engineer

26. Amazon Elastic File Store (EFS)

27. Amazon EFS
Scalable
Simple
Elastic
Built for:
Web serving, content management, media and entertainment workflows, home
directories, container storage, big data, and analytics
• Share files between EC2 instances in minutes
• True file system interface with file system semantics
• Fully managed – no capacity planning surprises
• Pay-as-you-go consumption and pricing
• Automatically grows and shrinks
• Much lower TCO than DIY or third-party workarounds
• Consistent performance even as data grows

28. Amazon EFS Use Cases

29. Newly acquired streaming media product
depended on a local file server
Had to launch at global scale in 90 days – with
minimal changes
• DIY was too complex and took too long
• Lift-and-shift to Amazon EFS took 2 hours
• EFS with EC2 autoscaling met global scale
agility needs
• Seamless integration between partner
application and existing AWS systems
• Post-mortem TCO analysis showed that EFS
was still the best choice
Enterprise applications
“Good, fast, and cheap. We picked two and
got all three with Amazon EFS. It gave us
the agility to deliver a new product on
schedule, eliminated scale and
performance concerns, and operates below
our
OPEX expectations.”
- Chris DeAcosta
Sr. director software engineering

30. Enterprise applications
Builds 3d digital maps relying on 28 TB of
waypoints generated daily
Unreliable on-premises repository and
high maintenance DIY cloud version
• Amazon EFS dropped infrastructure provisioning
time from 90 days to 7
• Now handling 800,000 daily file transfers up to
38% faster with zero failures
• Seamless JFrog workflow integration
• Gained high availability at no extra cost
• Also tiering JFrog backups into Amazon S3 and
Amazon Glacier
Prior to Amazon EFS, we experienced
timeouts for up to 10% of uploads over
100 MB. Now, all of the JFrog build
artifacts (from infrastructure-as-code
components to Docker images) are in one
place, and we’ve increased large file
transfer speeds by 38%.”
- Suresh Prem, Murty Chitti,
and Rajesh Sivaraman
System engineers

31. Migrating Data

32. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Moving to the Cloud
Common drivers of moving data to the cloud:
Pain points of data migration:
While every cloud adoption journey is unique, all require a data migration step, which
can be hindered by blockers such as:
Cost
effectiveness
Workforce
productivity
Business
agility
Operational
resilience
Access to
innovative
services
Disruption Speed Costs Channels Cloud
compatibility

33. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Amazon
S3
EC2
Amazon
EFS
Amazon
EBS
Aurora Amazon
EMR
Amazon Glacier
Amazon
RDS
IoT
Amazon
Redshift
DATA
Migrating Data: Five Key Questions
1) What kind of data is it, and where is it going?
4) How much data & time do you have?
2) One-time or continuous movement?
3) One-way or bidirectional access?
5) How might your WAN be a factor?
Files Block Volumes Databases IoT Devices

34. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
AWS Migration Capabilities
AWS
Direct
Connect
Establishes
private
connectivity
between AWS
and your data
center, office, or
colocation
environment.
AWS Snow
Family
(Snowball,
Snowball Edge,
Snowmobile)
Move terabytes
to petabytes of
data to AWS
using
appliances
designed for
secure, physical
transport.
AWS Storage
Gateway
(File, Volume, &
Tape Gateways)
Connects
on-premises
applications to
AWS. Uses
common storage
interfaces to
migrate data.
AWS
Partner
Network
Migration &
Storage
Competency
Partners provide
solutions and
expertise to help
businesses move
to AWS.
Amazon
Kinesis
Firehose
Capture,
process, & load
streaming data
into AWS to be
used with
business
intelligence and
analytics tools.
Amazon
S3 Transfer
Acceleration
Makes Internet
transfers to S3
faster. Ideal for
recurring jobs
such as
uploads,
backups, &
local processing
tasks
Discovery & Migration
Tracking
AWS Migration
Hub
AWS Application
Discovery Service
Server & Database
Migration
AWS Server
Migration Service
AWS Database
Migration Service
VMware on AWS
Data Migration Services Streaming
Data
Network OptimizationsPartners
Amazon
EFS
File Sync
Sync files from a
source file
system into a
destination
Amazon EFS file
system. The
source file
system can be
on-premises or
in the cloud
File

35. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Deeper Look Into
AWS Data Migration Services

36. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
1. AWS Snow family (1/2)
AWS Snowball AWS Snowball Edge AWS Snowmobile
• Petabyte-scale data migration
• Rugged 8.5 G impact case
• Rain and dust-resistant
• Data encryption end-to-end
• 80 TB capacity/10 Gb network
• Move 250 TB in 1 Week!
• Compute and storage for
hybrid/edge workloads
• 100-TB local storage
• Local compute equivalent to an
Amazon EC2 m4.4xlarge instance
• 10GBase-T, 10/25 Gb SFP28, and
40 Gb QSFP+ copper, and optical
networking
• Ruggedized and transportable
• Exabyte-scale data migration
• Up to 100-PB capacity
• Data encryption end-to-end
• Dedicated security personnel
• GPS tracking, alarm monitoring,
24/7 surveillance, and optional
additional security
(cont.)

37. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
How Snowball moves data into and out of AWS
Create
a job
Connect the
Snowball
Copy data to
the Snowball
Your data move
to Amazon S3
In transit to you Delivered to you Delivered to AWS At AWS
Job created Job completed
1. AWS Snow family (2/2)

38. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
2. AWS Storage Gateway
AWS Storage
Gateway
Amazon
EBS
snapshots
Amazon S3
Amazon
Glacier
AWS CloudTrailAmazon CloudWatch
Files
Volumes
Tapes
On-premises
AWSCloud
Use standard storage protocols locally to use AWS storage services and migrate data
IAM (IAM) AWS KMS
Gateways for different migrations
Volume
Tape
File
• Shift files and file workloads to cloud
• Files stored as objects in S3 buckets
• One-to-one mapping from files-to-objects, with
metadata
• Accessible as files locally, or as objects via
Amazon S3 API
• Amazon S3 lifecycle policies, versioning, & CRR
• Move block data to EBS/EC2, and access locally
• iSCSI volumes or snapshots tiered to Amazon S3
• Migrate or backup on-premises volumes to
Amazon EBS via Amazon EBS Snapshots
• Create on-premises volumes from Amazon EBS
Snapshots
• Compression of data in transit and at rest
• Migrate tape backups to AWS
• Durable & secure storage on Amazon Glacier &
Amazon S3
• Supports leading backup applications
• Data compressed, in transit and at rest
• Unlimited number of virtual tapes

39. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
3. Amazon Kinesis Firehose
• Load streaming data into data stores and analytics tools
• Enable near real-time analytics with existing BI tools and dashboards
• Automatically scales to match throughput of data
• Batch, compress, and encrypt data before sending it to the cloud
How it works
Capture & submit
streaming data to Firehose
Firehose transforms & loads streaming data into S3,
Redshift, & Amazon Elasticsearch domains
Analyze streaming data
using your favorite BI tools
Amazon
Kinesis
Firehose
Am
azo
n
S3
Amazon
Redshift
ES

40. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
4. Amazon EFS File Sync
Sync data from existing file systems into Amazon EFS file systems
Fast
Up to 5x faster than standard
Linux copy tools
Secure
Encrypted parallel data
transfer to AWS
Simple
Set up and manage easily from
the AWS Console
File systems from on-premises to Amazon EFS
DIY in-cloud file systems to Amazon EFS
Amazon EFS file systems between AWS Regions
Use Amazon EFS File Sync to copy…

41. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
The AWS Partner Network (APN) supports the largest ecosystem of technology and consulting partners so that you can
leverage existing resources and seamlessly integrate AWS services into your IT environment. APN also recognizes select
partners who have demonstrated solution deployment in the following areas: data migration, backup & recovery, archive,
primary storage, business continuance / disaster recovery (BC/DR). Our network also includes consulting partners who can
design and implement the migration and storage solutions that best support your organization.
5. AWS Partner Network – migration & storage
Backup & Restore Archive Primary Storage BC/DRData Migration

42. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
6. AWS Direct Connect
Reduce
Bandwidth
Costs
Consistent
Network
Performance
Compatible
with all AWS
Services
Private
Connectivity to
VPC
Elastic Simple
• Establish private connectivity between AWS and your data center
• Dedicated connection can be partitioned into multiple virtual interfaces
• Maintain network separation between public and private environments
BENEFITS

43. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
7. Amazon S3 Transfer Acceleration (1/2)
S3 Bucket AWS Edge
Location
Uploader
Optimized
Throughput!
Change your endpoint, not your code
Leverages 59 global edge locations
Optimized protocols
No firewall exceptions
No client software required
(cont.)

44. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
How fast is Amazon S3 Transfer Acceleration?
Rio De
Janeiro
Warsaw New York Atlanta Madrid Virginia Melbourne Paris Los Angeles Seattle Tokyo Singapore
Time[hrs]
500 GB upload from these edge locations to a bucket in Singapore
Public InternetAmazon S3 Transfer
Acceleration
On average, we have seen 171% improvement over
regular Amazon S3 when uploading over long distances
7. Amazon S3 Transfer Acceleration (2/2)

45. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.