Migrating Your Data to the Cloud: Exploring AWS Transfer Options

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migrating Your Data to the Cloud:
Exploring Your Options from AWS

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Overview of AWS Transfer Services
Data Migration
AWS Direct Connect
AWS DataSync
AWS Snowball Family
AWS Transfer for SFTP
AWS Storage Gateway Family
S3 Transfer Acceleration
AWS Partners

Portfolio breadth and depth
Amazon Elastic
Block Store
(Amazon EBS)
Amazon Simple
Storage Service
(Amazon S3)
Amazon Glacier
Data movement
OnlineOffline
AWS Snow family
AWS Storage Gateway family
AWS Direct Connect
AWS DataSync
Amazon S3
Transfer Acceleration
Storage partners
Amazon Elastic File
System (Amazon EFS)
New! Devices
New! Storage classes
New! Management features
New!
Transfer
services

AWS data transfer & hybrid storage
Online
data transfer
Hybrid
storage
Offline
data transfer
Private
network
connections
to AWS
Ship static
data into and
out of
Amazon S3
Access AWS
storage from
on-premises
Edge locations
for Amazon
S3 enabled
applications
Online
transfer of
active data
AWS
DataSync
AWS
Transfer
for SFTP
SFTP transfers
into Amazon S3
NEWNEW
Storage and
compute in
disconnected
environments

Why are we here?
We’ll be talking about moving your data
• Data tied to applications you’re
migrating
• Data in bulk storage
(e.g., NAS filers)
• Data connected to
on-premises applications
• Data as part of continuous edge
& hybrid workflows
You have IT resources and want to move them to the cloud

Why move to the cloud?
Common drivers of moving data to the cloud:
Pain points of data migration:
While every cloud adoption journey is unique, all require a data migration step, which
can be hindered by blockers such as:
Cost
effectiveness
Workforce
productivity
Business
agility
Operational
resilience
Access to
innovative
services
Disruption Speed Costs Channels Cloud
compatibility

Migrating data: Five key questions
1) What kind of data is it, and where is it going?
4) How much data and time do you have?
2) One time or continuous movement?
3) One way or bi-directional?
5) How about the WAN?
Amazon S3
Amazon EC2
Amazon
EFS
Amazon
EBS
Amazon
Aurora
Amazon
EMR
Amazon
Glacier
Amazon
RDS
AWS
IoT
Amazon
Redshift
Files Block
volumes
Databases IoT Streams

AWS Direct Connect
Reduce
bandwidth Costs
Consistent
network
performance
Compatible with
all AWS services
Private
connectivity
to VPC
Elastic Simple
• Establish private connectivity between AWS and your data center
• Dedicated connection can be partitioned into multiple virtual interfaces
• Maintain network separation between public and private environments
BENEFITS

What use cases need to transfer active data?
Migration of active
application data
Transferring data
for time sensitive
in-cloud analysis
Replication of data
for business
continuity
Online data transfer

What is AWS DataSync?
Online transfer service that simplifies, automates, and accelerates
moving data between on-premises storage and AWS
Fast data
transfer
Cost-
effective
Enterprise
ready
Combines the speed and reliability of network acceleration
software with the cost-effectiveness of open source tools
Easy to use Secure and
reliable

AWS DataSync
• Up to 10Gbps with incremental transfers, inline compression, sparse file detection,
and optimized read/write for Amazon S3 and Amazon Elastic File System (Amazon EFS)Fast data
transfer
Easy to use
Secure and
reliable
Enterprise
ready
Cost-
effective
• Managed through AWS Management Console, AWS Command Line Interface (AWS CLI), or SDK,
monitoring with Amazon CloudWatch, AWS infrastructure auto-deployed, and on-premises agents
automatically updated
• Data validated and encrypted in-transit and at-rest with automatic recovery from I/O or
transmission errors
• Full bandwidth management, PCI-DSS compliance, HIPPAA eligible, IAM and AWS CloudTrail
integrated
• $0.04 per-GB copied

“At Celgene, our research teams are focused intently on the
discovery and development of treatments for cancer and other
severe conditions. Our labs generate petabytes of data. We use AWS
DataSync to get the data into Amazon S3 and Amazon EFS easily,
quickly and cost-effectively. Without the data in AWS, there’s no
way we could innovate as fast.”
Sample use case – Celgene

As simple as one, two, three ...
On-premises NFS file
systems, Amazon S3 buckets, or
Amazon EFS file systems
Create a task by configuring
storage locations and options
Monitor in the
console or through
Amazon CloudWatch
Start the task and
monitor progress
VMware ESXi
8 vCPU
32 GB memory
Deploy the agent and
associate to your AWS account
1 2 3

Shared
file system
NFS TLS
How AWS DataSync works
On-Premises
Amazon S3
bucket
AWS Storage resources
AWS
DataSync
Agent deployed
on-premises for
fast access to
local storage
Region
Amazon EFS
file system
AWS DataSync
agent
Data transfer
over the WAN via
efficient purpose-
built protocol
Managed from the
console or AWS
Command Line
Interface (AWS CLI)
Service in AWS
writes or reads
data from AWS
storage services

Reduce infrastructure with AWS DataSync
and Storage Gateway
AWS DataSync transfers to Amazon S3 as part of migration incremental sync keeps Amazon S3 up to date1
2 Latency-sensitive applications can access data through file gateway, can decommission NAS
NAS
Amazon S3
bucket
On-Premises Region
Applications 2
AWS
DataSync
AWS DataSync
agent
1

What is Snowball and why
did we build it?
Moving large volumes of data over the internet
can take years – we ship secure physical devices to
you to transfer your data at the source before
shipping it back for bulk import to the cloud.
The cloud is not always accessible from remote
locations where connectivity is sparse or
intermittent – deploy ruggedized devices at the
edge with local storage and compute capacity to
process data without network dependencies.

Need for edge analytics in remote and harsh environments
Various industries ranging from research to transportation necessitate edge
computing capabilities to help cope with environments where data generation is
decentralized, data volumes are significant, and network connectivity is either
inaccessible or intermittent.
Research Industrial Healthcare Transportation

AWS Snowball
AWS Snowball Edge
Compute or Storage Optimized AWS Snowmobile
• 80-TB storage capacity
• 10GE networking
• Data encryption end-to-end
• Rugged 8.5-G impact case
• Rain and dust-resistant
• 42 or 100-TB storage capacity
• Rugged 8.5-G impact case
• Rain and dust resistant
• AWS Greengrass support for local
compute, messaging, and caching
• Amazon EC2 support for edge
compute
• Optional GPU
• Exabyte-scale storage in a 45-ft
container
• Dedicated security personnel
• GPS tracking, alarm monitoring,
24/7 surveillance, and optional
additional security
AWS Snow Family

AWS Snow Family - Who's Using It?
ISV
PartnersCustomers

AWS Snowball Edge
• Eight NFC tags
embedded in the
case. Opening the
device renders the
tags unreadable.
• With AWS
Snowball Edge
Verification mobile
app, customers
can check the tags
on a device as part
of their security
best practices.

Snowball Edge options summary
Amazon S3 Compatible Storage 42TB 100TB
Compute Comparable to m5d.2xlarge Comparable to m4.4xlarge
AWS Services available AWS Lambda, File Gateway, Amazon EC2 AWS Lambda, File Gateway, Amazon EC2
Memory 208GB 32GB
vCPUs 52 24
Disk Space for Instances 7.68TB 1TB
Clustering Available Available
Typical job lifetime Can be long lasting About 1 month
Networking Max 100Gb Max 40Gb
GPU Optional Nvidia Tesla V100 No
Storage
Optimized
Compute
Optimized

Amazon Simple Storage
Service (Amazon S3)
Compatible Endpoint
NFS File
Interface
Storage and
compute cluster
AWS Greengrass and
AWS Lambda support
Line rate
data transfer
Hardware enabled
data encryption
Snowball Edge key features
Amazon Elastic Compute
Cloud (Amazon EC2)
Instance support

SFTP: It’s here; it’s everywhere
Protocol is deeply embedded in workflows across a variety of industries
Financial services
$
Retail
Healthcare … and more

Migrate SFTP workflows to AWS
To benefit from
• Durable storage in Amazon S3
• Processing, archival and retention of data
• Integration with AWS services
SFTP SFTP server and data
Will your hundreds of partners
change their workflows to
integrate with the cloud??

Fully managed service enabling transfer
of data over SFTP while stored in Amazon S3
Seamless migration
of existing workflows
Native integration
with AWS services
Simple
to use
Cost
effective
Fully managed
in AWS
Enterprise
ready

FINRA regulates a critical part of the US securities industry. It
takes in up to 135 billion market events per day. Managing SFTP
infrastructure for over 1,000 external users doing 6 million file
transfers per month was ‘a bit of a strain’. They built their new
FileX platform on AWS, using AWS Transfer for SFTP service to
reduce operational burden.
Sample use case – FINRA

As simple as one, two, three …
Your users can now use your AWS SFTP server endpoint to transfer data
Map your hostname Select your Amazon S3
bucket(s)
Set up your users
1 2 3

Your SFTP architecture now
The same hundreds of users
now serviced by AWS Transfer,
without requiring changes
+ Enterprise ready
+ No end user disruption
+ Fully managed servers
+ Simple to use
+ Pay as you use
+ Native cloud integrations
Amazon S3AWS SFTP
AWS Transfer
for SFTP

AWS Storage Gateway
Existing Data
Archives
New Data
Corporate
Data Centers or
Offices
AWS Storage Gateway
HTTPS
NFS
SMB
iSCSI
VTL
Hybrid storage service enabling applications to seamlessly use AWS storage
On-Premises Gateway Service
Amazon S3
Amazon EBS
snapshots
Amazon Glacier

The AWS Storage Gateway family
File gateway
Store and access objects in Amazon S3 from file-based
applications with local caching
Volume gateway
Block storage on-premises backed by cloud storage with local
caching, Amazon EBS snapshots, and clones
Tape gateway
Drop-in replacement for physical tape infrastructure backed by
cloud storage with local caching
Three gateway types provide file, block, and tape storage interfaces

“Storage Gateway is at the core of our Disaster Recovery (DR) and
Business Continuity Management (BCM) processes, handling
our Co-Lo'd and off-site data backups, as well as our in-office BCM.
It works transparently, in a lights out way, archiving off to a
separate AWS account with a simple grandfather-father-son
snapshot plan in place.”
Sample use case - JustGiving

AWS Storage Gateway key features
Standard storage
protocols
Seamlessly connect on-premises
applications to AWS storage without
modifying applications
Fully managed cache
Frequently used data cached locally
For low-latency access
Optimized data transfer
Secures upload of changed data and
downloads requested data
AWS integrated
Manage on-premises gateways in AWS Management
Console and use your data across the entire portfolio
of AWS applications and services

File Gateway Objects in your
Amazon S3 bucket
NFS v3/v4
SMB v2/v3
Key
Capabilities
• Accessed via NFS or SMB
• Local cache for low-latency access to data in cloud
• Ideal for hybrid workflows such as data generation pipelines
HTTPS
Access Amazon S3 from file-based applications with local caching
File gateway overview

Cloud-backed block storage presented on-premises
Volume gateway overview
Application
server
Amazon EBS
snapshots
ServiceVolume
Gateway
Volume stored in
Amazon S3
HTTPSiSCSI
Customer Premises
Key
Capabilities
• Supports iSCSI block protocol
• Available in two modes: cached and stored
• Compresses and tiers partial or whole volumes to the cloud to reduce
on-premises SAN/NAS footprint
• Take EBS snapshots of on-premises volumes and store them in cloud

Backup
server
Tape library Tape shelf
On-Premises
iSCSI HTTPS
Media
changer
Tape Gateway
Tape
drives
Key
Capabilities
• Emulates physical tape device with a media changer and tape drives
• Virtual tapes written to Amazon S3
• Ejected virtual tapes are marked “read only” and moved to “tape shelf” in
Amazon Glacier
• Recovery: Retrieve tapes to library (3–5 hours) and read data from a gateway
Tape gateway overview
Virtual tapes presented to on-premises backup applications

Storage Gateway is available on a variety of form factors
Virtual machines Amazon EC2
Hardware
appliance
Corporate data centers In AWS
Remote locations,
branch offices
WhereWhen
Leverage existing VM
infrastructure
In-cloud data
retrieval/access/DR
Out of the box
simplified experience

AWS Storage Gateway - Who's Using It?

Amazon S3 Transfer Acceleration
AWS Edge
Location
Optimized
Throughput!
Change your endpoint, not your code
Leverages Amazon CloudFront
global edge locations
Optimized protocols
No firewall exceptions
No client software required

Getting Started
Enable Amazon S3 Transfer Acceleration for their bucket
DNS resolution
<bucket>.s3-accelerate.amazonaws.com resolves to the
nearest POP location for the client
uses the client IP and CloudFront latency based routing
Update application / destination URI to:
<bucket-name>.s3-accelerate.amazonaws.com
File is uploaded to:
<bucket-name>

Rio De Janeiro Warsaw New York Atlanta Madrid Virginia Melbourne Paris Los Angeles Seattle Tokyo Singapore
Time[hrs]
500 GB upload from these edge locations to a bucket in Singapore
Public Internet
How Fast is Amazon S3 Transfer Acceleration?
S3 Transfer Acceleration
On average, we have seen 171%
improvement over regular Amazon S3
when uploading over long distances

Backup & restore Archive Primary storage BC/DRData migration
AWS Partner Network (APN): Migration & storage

Migrating Your Data to the Cloud: Exploring AWS Transfer Options

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Migrating Your Data to the Cloud: Exploring AWS Transfer Options

Ähnlich wie Migrating Your Data to the Cloud: Exploring AWS Transfer Options (20)

Mehr von Amazon Web Services

Mehr von Amazon Web Services (20)

Migrating Your Data to the Cloud: Exploring AWS Transfer Options