Weitere ähnliche Inhalte Ähnlich wie Deep Dive on S3 Glacier Deep Archive (20) Mehr von Amazon Web Services (20) Deep Dive on S3 Glacier Deep Archive1. P U B L I C S E C T O R
S U M M I T
WASHINGTON, DC
2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Storing data long term with Amazon Simple
Storage Service Glacier (Amazon S3 Glacier)
Deep Archive
Rob Czarnecki
Principal Product Manager, Amazon S3 Glacier
3 2 0 9 9 6
3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Agenda
• Amazon Simple Storage Service (Amazon S3) storage classes
• Using Amazon S3 Glacier Deep Archive
• AWS Partner Network integrations
• Economics
• Questions
4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Services I’ll Mention Here
Amazon Simple Storage Service (Amazon S3)
Amazon Simple Storage Service Glacier (Amazon S3 Glacier)
AWS Command Line Interface (AWS CLI)
Amazon Simple Queue Service (Amazon SQS)
Amazon Simple Notification Service (Amazon SNS)
AWS Lambda (Lambda)
5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Amazon S3 storage classes
Access frequencyFrequent Infrequent
• Active data
• Millisecond access
• > 3 AZ
• $0.023/GB-mo.
• Data with changing
access pattern
• Millisecond access
• > 3 AZ
• $0.023 to $0.0125/GB-
mo. + object
monitoring
• 30-day min. duration
• Inactive data
• Millisecond access
• > 3 AZ
• $0.0125/GB
• $0.01 per GB
retrieved
• 30-day min. duration
• 128KB min. object
S3 Standard S3 Standard-IA S3 One Zone-IA S3 Glacier
• Inactive data
• Millisecond access
• 1 AZ
• $0.01/GB
• $0.01 per GB retrieved
• 30-day min. duration
• 128KB min. object
• Archive data
• Minutes to hours
• > 3 AZ
• $0.004/GB-mo.
• $0.0025–$0.03/GB
retrieved
• 90-day min.
duration
S3 Intelligent-
Tiering
S3 Glacier
Deep Archive
• Archive data
• Hours access
• > 3 AZ
• $0.00099/GB-mo.
• $0.0025–$0.02/GB
retrieved
• 180-day min.
duration
6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Amazon S3 Glacier Deep Archive
No tape to
manage
Designed for
99.999999999%
durability
Recover data in
hours vs.
days/weeks
$0.00099 per GB-
month
Less than 1/4 the
cost of Amazon
S3 Glacier
7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Common use cases for Amazon S3
Backup & recovery
Easy place to start
Integrated with all
major vendors
Pay only for what you use
Photo & video
Media archives and
workflows
Tape replacement
Cost optimization with
Amazon S3 storage classes
Data lakes & analytics
Broad analytics and
visualization integrations
Streaming or batch data
Scalable storage and
parallel processing
8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
What is Amazon S3 Glacier Deep Archive?
Designed to reduce costs on data you access once per year or less
API matches Amazon S3 Glacier
• Asynchronous
• --storage-class DEEP_ARCHIVE
Three differences
• Standard retrievals in 12 hours
• Bulk retrievals in 48 hours
• No expedited retrieval, no S3 Glacier Select
• Minimum object duration is 180 days
9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Storing data in S3 Glacier Deep Archive
Copy another S3 objectDirect PUT to
Glacier Deep Archive
Store data in any Amazon S3
storage class
AWS Management Console
AWS CLI
AWS SDK
Put object
storage_class: DEEP_ARCHIVE
Copy object
storage_class: DEEP_ARCHIVE
11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Storing data in Amazon S3 Glacier Deep Archive
Cross-region replication to
Amazon S3 Glacier Deep
Archive
Lifecycle S3 objects
Store data in any storage class
Define lifecycle policies—
tags, object age
S3 continuously
evaluates policy, shifts data
Define CRR policies, with
remote storage class
Data replicated as
it’s uploaded
12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Cross-region replication
Automatically replicate data to any other AWS Region
Configure by bucket, prefix, or tag values
Replicate data to Glacier Deep Archive
Change the object owner in the destination region
13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Lifecycle management
Create rules to automatically transition or expire your storage
Configure by bucket, prefix, or tag values
Lifecycle rules take action based on object age
Example policy:
Move all objects older than 30 days to Amazon S3 Standard-Infrequent Access
Move all objects older than 90 days to Amazon S3 Glacier Deep Archive
14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Lifecycle management
Create rules to automatically transition or expire your storage
Configure by bucket, prefix, or tag values
Lifecycle rules take action based on object age
Example policy:
Move all objects older than 180 days to Amazon S3 Glacier
Move all objects older than 730 days to Amazon S3 Glacier Deep Archive
15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
AWS tape gateway
Benefits
On premise
iSCSI VTL
Tape gateway
HTTPS
Application
AWS Storage Gateway service
(S3 Glacier Deep Archive)
• Drop-in replacement for physical tape infrastructure
• Allows you to backup & archive data to AWS without changing your backup applications or workflows
• Reduces your TCO compared to using physical tapes and offsite warehousing
• Extensive compliance and security features: HIPAA, PCI, SOC (1, 2, 3), ISO (9001, 27001, 27017, 27018)
NEW
OR (S3 Glacier)
Tape library
(Amazon S3)
Tape shelf
16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Multiple data transfer services
AWS SnowballAWS Snowmobile AWS Direct Connect
AWS Transfer for SFTP AWS DataSync
17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Accessing data in Amazon S3 Glacier Deep Archive
Restore, followed by a get
Issue a restore request for the object (select speed, duration of restore)
Amazon S3 restores object into Amazon S3 Reduced Redundancy Storage; notification issued
Issue a Get request for the object (millisecond access)
Use restore notification to drive workflows through Amazon SNS,
Amazon SQS, or Lambda functions
$ aws s3api restore-object --bucket mybucket --key dir1/example.obj --
restore-request
'{"Days":25,"GlacierJobParameters":{"Tier":"Standard"}}'
18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Amazon S3 Glacier restore – Architecture pattern
Lambda function
Instance
Amazon S3
Initiate restore
Container
Lambda function
Object
restored
Amazon SNS
Amazon SQS
19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Multiple restore speeds
Retrieval tier S3 Glacier S3 Glacier Deep Archive
Expedited 1–5 mins* not available
Standard 3–5 hours within 12 hours
Bulk 5–12 hours within 48 hours
20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Archive partner integrations
21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Typical workflow
250TB stored 50,000 objects
(avg. size 25GB)
0.5% accessed
monthly
23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Long-term retention with standard retrieval costs
Amazon S3 Glacier Deep Archive
PUT requests
$2.50 (50,000 @ $0.05/1k)
Storage
$3,041.28 (250TB @ $0.00099/GB-mo.)
Standard retrieval requests
$0.30 (250 @ $0.025/1k per mo.)
Standard retrieval bytes
$307.20 (1.25TB @ $0.0025/GB per mo.)
Total cost (12 months)
$3,351.28
Amazon S3 Glacier
PUT requests
$2.50 (50,000 @ $0.05/1k)
Storage
$12,288.00 (250TB @ $0.004/GB-mo.)
Standard retrieval requests
$0.15 (250 @ $0.025/1k per mo.)
Standard retrieval bytes
$153.60 (1.25TB @ $0.0025/GB per mo.)
Total cost (12 months)
$12,444.71
24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Long-term retention with bulk retrieval costs
S3 Glacier Deep Archive
PUT requests
$2.50 (50,000 @ $0.05/1k)
Storage
$3,041.28 (250TB @ $0.00099/GB-mo.)
Bulk retrieval requests
$0.075 (250 @ $0.025/1k per mo.)
Bulk retrieval bytes
$38.40 (1.25TB @ $0.0025/GB per mo.)
Total cost (12 months)
$3,082.255
S3 Glacier
PUT requests
$2.50 (50,000 @ $0.05/1k)
Storage
$12,288.00 (250TB @ $0.004/GB-mo.)
Bulk retrieval requests
$0.075 (250 @ $0.025/1k per mo.)
Bulk retrieval bytes
$38.40 (1.25TB @ $0.0025/GB per mo.)
Total cost (12 months)
$12,328.975
25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Compare to on-premises tape libraries
Opportunity cost
Tape generation
migrations
Administration
& operations
Offsite storage &
transportation
Capital: servers,
libraries, tapes
Hosting (space, power,
cooling, network)
26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
How do the storage classes compare?
Access times
27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Storage cost to retrieval cost
S3
S3-IA
S3 Glacier
S3 Glacier
Deep Archive
-$0.0050
$0.0000
$0.0050
$0.0100
$0.0150
$0.0200
$0.0250
$0.0300
-$0.0100 $0.0000 $0.0100 $0.0200 $0.0300 $0.0400 $0.0500 $0.0600 $0.0700
StorageCosts(perGB-mo.)
Standard Retrieval Costs
28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Economics: Other factors to consider
Frequency of access Object size
29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
30. Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Rob Czarnecki
czarnr@amazon.com
31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T