Learning Objectives: - Recognize why and when to use Amazon EFS and the ecomonic benefits versus other solutions - Understand key elements to optimize the movement of your data to Amazon EFS - See Amazon EFS in action with live demo

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Prashanth Bungale – Sr. Product Manager
Darryl Osborne – Solutions Architect
December 14, 2017
Moving your File Data to
Amazon Elastic File System
Amazon (EFS)

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agenda
Features Ingest Wrapping up

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Features

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Before Amazon EFS… DIY file storageAZ-a
Clients
Storage
volumes
AZ-b
Storage
volumes
File server
File server

5. Amazon EFS
A fully managed file service

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key features of Amazon EFS
ScalableSimple Elastic
Highly available and durable

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key features of Amazon EFS
ScalableSimple Elastic
Highly available and durable

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key features of Amazon EFS
ScalableSimple Elastic
Highly available and durable

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key features of Amazon EFS
ScalableSimple Elastic
Highly available and durable

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key features of Amazon EFS
ScalableSimple Elastic
Highly available and durable

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Designed for a wide spectrum of needs
Scale-out jobs Metadata-intensive jobs
Analytics
Media workflows
Web serving
Content management
Database backups
Enterprise apps
and messaging
Dev tooling
Home directories
Container storage
Low latency and serial I/OHigh throughput and parallel I/O

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon EFS customers
Web serving + content management
Media and entertainment workflowsDatabase backups
Big data and analytics
Enterprise applications
Developer tools Home directories Container storage

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Security model and security features
Control
network traffic
using Amazon VPC security
groups and network ACLs
Control file and
directory access
using POSIX
permissions
Control administrative
access (API access)
using AWS IAM
(action-level and
resource-level
permissions)
Encrypt
data at rest
using keys
managed in
AWS KMS

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Access EFS from various environments
Amazon
EFS
Amazon
EC2
Corporate data
center
VMware Cloud
on AWS

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon EFS economics
$0.30/GB-Month (US Regions)
$0.33/GB-Month (EU Ireland)
$0.36/GB-Month (EU Frankfurt)
$0.36/GB-Month (AP Sydney)
No minimum commitments
or up-front fees
No need to provision
storage in advance
No other fees, charges,
or billing dimensions

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Before EFS…costs of DIY file storageAZ-a
Clients
Storage
volumes
AZ-b
Storage
volumes
File server
File server
Amazon EC2
instance costs
Inter-AZ data
transfer costs
Amazon EBS
volume costs

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
TCO example: For storing 500 GB,
Amazon EFS is 70% less than DIY
Amazon EFS cost: (500GB * $0.30/GB-month) = $150 per month
For DIY, you might provision 600 GB of Amazon EBS (i.e., ~85% utilization):
Storage (2x 600 GB EBS gp2 volumes): $120 per month
Compute (2x m4.xlarge instances): $290 per month
Inter-AZ data transfer costs (est.): $130 per month
Total $540 per month

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Where is Amazon EFS available today?
• US West (Oregon)
• US East (N. Virginia)
• US East (Ohio)
• EU (Ireland)
• EU (Frankfurt)
• Asia Pacific (Sydney)
More coming soon!

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ingest

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data movement
OnlineOffline
Data security
and management
Complete set of data building blocks
Amazon
EFS
Amazon
EBS
AWS Snow Family
AWS Storage Gateway
Family
AWS Direct Connect
Amazon EFS File Sync
Amazon S3
Transfer Acceleration
Storage Partners
Amazon Kinesis
Data Streams
Amazon Kinesis
Video Streams
Amazon
S3
Amazon
Glacier
AWS KMS
AWS IAM
AWS CloudWatch
AWS CloudTrail
AWS CloudFormation
AWS Lambda
Amazon Macie
AWS QuickSight

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What do you think about?
Where it’s coming from How to move it
as fast and easily
as possible

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Where is it coming from?
Corporate data
center
Amazon
S3
Amazon EBS Other on-cloud
file systems

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Direct Connect
AWS
Direct Connect
EFS in your Amazon VPCOn-premises servers

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Third-party VPN
Amazon
EFS
Mount targetVPN
instanceNFS client VPN

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
By the way, Amazon EFS over AWS DX or
third-party VPN isn’t just for migration
Bursting
Migration
• Move entire data set permanently to EFS
• Access the data from applications running on EC2 instances
• Move data set temporarily to EFS
• Access the data from applications running on EC2 instances
• Move data back on premises when processing finishes
Backup and
disaster recovery
• Maintain copy of entire data set on EFS
• Restore the data to on-premises storage or (for DR) access
the data from failed-over applications running on EC2
instances

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Introducing EFS File Sync
Fast
Up to 5x faster than
standard Linux copy tools
Sync data from existing file systems into Amazon EFS file systems
Secure
Encrypted parallel data
transfer to AWS
Simple
Set up and manage easily
from the AWS Console
file systems from on-premises to EFS
DIY in-cloud file systems to EFS
EFS file systems between AWS
Regions
Use EFS File Sync to copy…

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Hands-on EFS File Sync demo

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Between Amazon S3 & Amazon EFS
Mount
target
NFS clientS3 bucket
Amazon
EFS

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Between Amazon S3 & Amazon EFS
S3 bucket
Amazon
EFS
Mount
target
NFS
clients

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Hands-on Amazon S3 to Amazon EFS
demo

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How to copy data faster from Amazon S3
S3 cp
S3 cp + GNU
parallel
multi-threaded multi-threaded
Good Best

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Large File Results
EC2 instance: c4.2xlarge
Transfer test:
22 ~1 GB files from Amazon S3 to Amazon EFS
110.55
172.0
210
135
0
50
100
150
200
250
0
20
40
60
80
100
120
140
160
180
200
s3 cp + GNU Parallel
Duration(sec.)
MB/s
s3 cp
Throughput (MB/s) Duration (seconds)

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Small File Results
EC2 instance: c4.2xlarge
Transfer test:
5200 ~1MB files from Amazon S3 to Amazon EFS
41.15
100.42
122
50
0
20
40
60
80
100
120
140
0.00
20.00
40.00
60.00
80.00
100.00
120.00
s3 cp + GNU Parallel
Duration(sec.)
MB/s
s3 cp
Throughput (MB/s) Duration (seconds)

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Wrapping up

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Summary
EFS File Sync when coming from:
• On-premises
• Amazon EBS (multi-VPC/multi-account)
S3 cp + GNU Parallel when coming from:
• Amazon S3

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Resources
Amazon EFS webpage https://aws.amazon.com/efs
What is a Cloud File
System?
https://aws.amazon.com/what-is-cloud-file-storage
Amazon EFS TCO https://aws.amazon.com/efs/tco
YouTube AWS channel https://www.youtube.com/user/amazonwebservices
Reference architecture https://aws.amazon.com/architecture
Amazon EFS tutorials https://github.com/aws-samples/amazon-efs-tutorial
GNU Parallel https://www.gnu.org/software/parallel

37. Thank you!
Next: Q&A