SlideShare ist ein Scribd-Unternehmen logo
1 von 31
P U B L I C S E C T O R
S U M M I T
WASHINGTON, DC
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Storing data long term with Amazon Simple
Storage Service Glacier (Amazon S3 Glacier)
Deep Archive
Rob Czarnecki
Principal Product Manager, Amazon S3 Glacier
3 2 0 9 9 6
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Agenda
• Amazon Simple Storage Service (Amazon S3) storage classes
• Using Amazon S3 Glacier Deep Archive
• AWS Partner Network integrations
• Economics
• Questions
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Services I’ll Mention Here
Amazon Simple Storage Service (Amazon S3)
Amazon Simple Storage Service Glacier (Amazon S3 Glacier)
AWS Command Line Interface (AWS CLI)
Amazon Simple Queue Service (Amazon SQS)
Amazon Simple Notification Service (Amazon SNS)
AWS Lambda (Lambda)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Amazon S3 storage classes
Access frequencyFrequent Infrequent
• Active data
• Millisecond access
• > 3 AZ
• $0.023/GB-mo.
• Data with changing
access pattern
• Millisecond access
• > 3 AZ
• $0.023 to $0.0125/GB-
mo. + object
monitoring
• 30-day min. duration
• Inactive data
• Millisecond access
• > 3 AZ
• $0.0125/GB
• $0.01 per GB
retrieved
• 30-day min. duration
• 128KB min. object
S3 Standard S3 Standard-IA S3 One Zone-IA S3 Glacier
• Inactive data
• Millisecond access
• 1 AZ
• $0.01/GB
• $0.01 per GB retrieved
• 30-day min. duration
• 128KB min. object
• Archive data
• Minutes to hours
• > 3 AZ
• $0.004/GB-mo.
• $0.0025–$0.03/GB
retrieved
• 90-day min.
duration
S3 Intelligent-
Tiering
S3 Glacier
Deep Archive
• Archive data
• Hours access
• > 3 AZ
• $0.00099/GB-mo.
• $0.0025–$0.02/GB
retrieved
• 180-day min.
duration
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Amazon S3 Glacier Deep Archive
No tape to
manage
Designed for
99.999999999%
durability
Recover data in
hours vs.
days/weeks
$0.00099 per GB-
month
Less than 1/4 the
cost of Amazon
S3 Glacier
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Common use cases for Amazon S3
Backup & recovery
Easy place to start
Integrated with all
major vendors
Pay only for what you use
Photo & video
Media archives and
workflows
Tape replacement
Cost optimization with
Amazon S3 storage classes
Data lakes & analytics
Broad analytics and
visualization integrations
Streaming or batch data
Scalable storage and
parallel processing
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
What is Amazon S3 Glacier Deep Archive?
Designed to reduce costs on data you access once per year or less
API matches Amazon S3 Glacier
• Asynchronous
• --storage-class DEEP_ARCHIVE
Three differences
• Standard retrievals in 12 hours
• Bulk retrievals in 48 hours
• No expedited retrieval, no S3 Glacier Select
• Minimum object duration is 180 days
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Storing data in S3 Glacier Deep Archive
Copy another S3 objectDirect PUT to
Glacier Deep Archive
Store data in any Amazon S3
storage class
AWS Management Console
AWS CLI
AWS SDK
Put object
storage_class: DEEP_ARCHIVE
Copy object
storage_class: DEEP_ARCHIVE
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Storing data in Amazon S3 Glacier Deep Archive
Cross-region replication to
Amazon S3 Glacier Deep
Archive
Lifecycle S3 objects
Store data in any storage class
Define lifecycle policies—
tags, object age
S3 continuously
evaluates policy, shifts data
Define CRR policies, with
remote storage class
Data replicated as
it’s uploaded
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Cross-region replication
Automatically replicate data to any other AWS Region
Configure by bucket, prefix, or tag values
Replicate data to Glacier Deep Archive
Change the object owner in the destination region
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Lifecycle management
Create rules to automatically transition or expire your storage
Configure by bucket, prefix, or tag values
Lifecycle rules take action based on object age
Example policy:
Move all objects older than 30 days to Amazon S3 Standard-Infrequent Access
Move all objects older than 90 days to Amazon S3 Glacier Deep Archive
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Lifecycle management
Create rules to automatically transition or expire your storage
Configure by bucket, prefix, or tag values
Lifecycle rules take action based on object age
Example policy:
Move all objects older than 180 days to Amazon S3 Glacier
Move all objects older than 730 days to Amazon S3 Glacier Deep Archive
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
AWS tape gateway
Benefits
On premise
iSCSI VTL
Tape gateway
HTTPS
Application
AWS Storage Gateway service
(S3 Glacier Deep Archive)
• Drop-in replacement for physical tape infrastructure
• Allows you to backup & archive data to AWS without changing your backup applications or workflows
• Reduces your TCO compared to using physical tapes and offsite warehousing
• Extensive compliance and security features: HIPAA, PCI, SOC (1, 2, 3), ISO (9001, 27001, 27017, 27018)
NEW
OR (S3 Glacier)
Tape library
(Amazon S3)
Tape shelf
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Multiple data transfer services
AWS SnowballAWS Snowmobile AWS Direct Connect
AWS Transfer for SFTP AWS DataSync
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Accessing data in Amazon S3 Glacier Deep Archive
Restore, followed by a get
Issue a restore request for the object (select speed, duration of restore)
Amazon S3 restores object into Amazon S3 Reduced Redundancy Storage; notification issued
Issue a Get request for the object (millisecond access)
Use restore notification to drive workflows through Amazon SNS,
Amazon SQS, or Lambda functions
$ aws s3api restore-object --bucket mybucket --key dir1/example.obj --
restore-request
'{"Days":25,"GlacierJobParameters":{"Tier":"Standard"}}'
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Amazon S3 Glacier restore – Architecture pattern
Lambda function
Instance
Amazon S3
Initiate restore
Container
Lambda function
Object
restored
Amazon SNS
Amazon SQS
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Multiple restore speeds
Retrieval tier S3 Glacier S3 Glacier Deep Archive
Expedited 1–5 mins* not available
Standard 3–5 hours within 12 hours
Bulk 5–12 hours within 48 hours
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Archive partner integrations
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Typical workflow
250TB stored 50,000 objects
(avg. size 25GB)
0.5% accessed
monthly
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Long-term retention with standard retrieval costs
Amazon S3 Glacier Deep Archive
PUT requests
$2.50 (50,000 @ $0.05/1k)
Storage
$3,041.28 (250TB @ $0.00099/GB-mo.)
Standard retrieval requests
$0.30 (250 @ $0.025/1k per mo.)
Standard retrieval bytes
$307.20 (1.25TB @ $0.0025/GB per mo.)
Total cost (12 months)
$3,351.28
Amazon S3 Glacier
PUT requests
$2.50 (50,000 @ $0.05/1k)
Storage
$12,288.00 (250TB @ $0.004/GB-mo.)
Standard retrieval requests
$0.15 (250 @ $0.025/1k per mo.)
Standard retrieval bytes
$153.60 (1.25TB @ $0.0025/GB per mo.)
Total cost (12 months)
$12,444.71
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Long-term retention with bulk retrieval costs
S3 Glacier Deep Archive
PUT requests
$2.50 (50,000 @ $0.05/1k)
Storage
$3,041.28 (250TB @ $0.00099/GB-mo.)
Bulk retrieval requests
$0.075 (250 @ $0.025/1k per mo.)
Bulk retrieval bytes
$38.40 (1.25TB @ $0.0025/GB per mo.)
Total cost (12 months)
$3,082.255
S3 Glacier
PUT requests
$2.50 (50,000 @ $0.05/1k)
Storage
$12,288.00 (250TB @ $0.004/GB-mo.)
Bulk retrieval requests
$0.075 (250 @ $0.025/1k per mo.)
Bulk retrieval bytes
$38.40 (1.25TB @ $0.0025/GB per mo.)
Total cost (12 months)
$12,328.975
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Compare to on-premises tape libraries
Opportunity cost
Tape generation
migrations
Administration
& operations
Offsite storage &
transportation
Capital: servers,
libraries, tapes
Hosting (space, power,
cooling, network)
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
How do the storage classes compare?
Access times
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Storage cost to retrieval cost
S3
S3-IA
S3 Glacier
S3 Glacier
Deep Archive
-$0.0050
$0.0000
$0.0050
$0.0100
$0.0150
$0.0200
$0.0250
$0.0300
-$0.0100 $0.0000 $0.0100 $0.0200 $0.0300 $0.0400 $0.0500 $0.0600 $0.0700
StorageCosts(perGB-mo.)
Standard Retrieval Costs
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Economics: Other factors to consider
Frequency of access Object size
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
Rob Czarnecki
czarnr@amazon.com
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R
S U M M I T

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

AWS Security Hub
AWS Security HubAWS Security Hub
AWS Security Hub
 
(SEC318) AWS CloudTrail Deep Dive
(SEC318) AWS CloudTrail Deep Dive(SEC318) AWS CloudTrail Deep Dive
(SEC318) AWS CloudTrail Deep Dive
 
민첩하고 비용효율적인 Data Lake 구축 - 문종민 솔루션즈 아키텍트, AWS
민첩하고 비용효율적인 Data Lake 구축 - 문종민 솔루션즈 아키텍트, AWS민첩하고 비용효율적인 Data Lake 구축 - 문종민 솔루션즈 아키텍트, AWS
민첩하고 비용효율적인 Data Lake 구축 - 문종민 솔루션즈 아키텍트, AWS
 
From Batch to Streaming - How Amazon Flex Uses Real-time Analytics
From Batch to Streaming - How Amazon Flex Uses Real-time AnalyticsFrom Batch to Streaming - How Amazon Flex Uses Real-time Analytics
From Batch to Streaming - How Amazon Flex Uses Real-time Analytics
 
Amazon S3 & Amazon Glacier - Object Storage Overview
Amazon S3 & Amazon Glacier - Object Storage OverviewAmazon S3 & Amazon Glacier - Object Storage Overview
Amazon S3 & Amazon Glacier - Object Storage Overview
 
Getting Started with Amazon WorkSpaces
 Getting Started with Amazon WorkSpaces Getting Started with Amazon WorkSpaces
Getting Started with Amazon WorkSpaces
 
ABCs of AWS: S3
ABCs of AWS: S3ABCs of AWS: S3
ABCs of AWS: S3
 
AWS CloudWatch
AWS CloudWatchAWS CloudWatch
AWS CloudWatch
 
Storage with Amazon S3 and Amazon Glacier
Storage with Amazon S3 and Amazon GlacierStorage with Amazon S3 and Amazon Glacier
Storage with Amazon S3 and Amazon Glacier
 
AWS Big Data Platform
AWS Big Data PlatformAWS Big Data Platform
AWS Big Data Platform
 
AWS 스토리지 서비스 소개 및 실습 - 김용기, AWS 솔루션즈 아키텍트
AWS 스토리지 서비스 소개 및 실습 - 김용기, AWS 솔루션즈 아키텍트AWS 스토리지 서비스 소개 및 실습 - 김용기, AWS 솔루션즈 아키텍트
AWS 스토리지 서비스 소개 및 실습 - 김용기, AWS 솔루션즈 아키텍트
 
Amazon S3 Masterclass
Amazon S3 MasterclassAmazon S3 Masterclass
Amazon S3 Masterclass
 
Introduction to Amazon Athena
Introduction to Amazon AthenaIntroduction to Amazon Athena
Introduction to Amazon Athena
 
Monitor All Your Things: Amazon CloudWatch in Action with BBC (DEV302) - AWS ...
Monitor All Your Things: Amazon CloudWatch in Action with BBC (DEV302) - AWS ...Monitor All Your Things: Amazon CloudWatch in Action with BBC (DEV302) - AWS ...
Monitor All Your Things: Amazon CloudWatch in Action with BBC (DEV302) - AWS ...
 
AWS S3 Tutorial For Beginners | Edureka
AWS S3 Tutorial For Beginners | EdurekaAWS S3 Tutorial For Beginners | Edureka
AWS S3 Tutorial For Beginners | Edureka
 
Amazon simple storage service (amazon s3)
Amazon simple storage service (amazon s3)Amazon simple storage service (amazon s3)
Amazon simple storage service (amazon s3)
 
Amazon OpenSearch - Use Cases, Security/Observability, Serverless and Enhance...
Amazon OpenSearch - Use Cases, Security/Observability, Serverless and Enhance...Amazon OpenSearch - Use Cases, Security/Observability, Serverless and Enhance...
Amazon OpenSearch - Use Cases, Security/Observability, Serverless and Enhance...
 
How to backup, restore and archive your data on AWS
How to backup, restore and archive your data on AWSHow to backup, restore and archive your data on AWS
How to backup, restore and archive your data on AWS
 
AWS 마켓플레이스 성공 런칭을 위한 핵심 기술 (이경수, AWS 솔루션즈아키텍트) :: AWS TechShift 2018
AWS 마켓플레이스 성공 런칭을 위한 핵심 기술 (이경수, AWS 솔루션즈아키텍트) :: AWS TechShift 2018AWS 마켓플레이스 성공 런칭을 위한 핵심 기술 (이경수, AWS 솔루션즈아키텍트) :: AWS TechShift 2018
AWS 마켓플레이스 성공 런칭을 위한 핵심 기술 (이경수, AWS 솔루션즈아키텍트) :: AWS TechShift 2018
 
ABD201-Big Data Architectural Patterns and Best Practices on AWS
ABD201-Big Data Architectural Patterns and Best Practices on AWSABD201-Big Data Architectural Patterns and Best Practices on AWS
ABD201-Big Data Architectural Patterns and Best Practices on AWS
 

Ähnlich wie Deep Dive on S3 Glacier Deep Archive

Ähnlich wie Deep Dive on S3 Glacier Deep Archive (20)

Storing data long term with Amazon S3 Glacier Deep Archive - STG302 - Chicago...
Storing data long term with Amazon S3 Glacier Deep Archive - STG302 - Chicago...Storing data long term with Amazon S3 Glacier Deep Archive - STG302 - Chicago...
Storing data long term with Amazon S3 Glacier Deep Archive - STG302 - Chicago...
 
Storing data long term with Amazon S3 Glacier Deep Archive - STG301 - New Yor...
Storing data long term with Amazon S3 Glacier Deep Archive - STG301 - New Yor...Storing data long term with Amazon S3 Glacier Deep Archive - STG301 - New Yor...
Storing data long term with Amazon S3 Glacier Deep Archive - STG301 - New Yor...
 
Deep dive on Amazon S3 Glacier Deep Archive - STG301 - Santa Clara AWS Summit
Deep dive on Amazon S3 Glacier Deep Archive - STG301 - Santa Clara AWS SummitDeep dive on Amazon S3 Glacier Deep Archive - STG301 - Santa Clara AWS Summit
Deep dive on Amazon S3 Glacier Deep Archive - STG301 - Santa Clara AWS Summit
 
Compliance-Data-Archival
Compliance-Data-ArchivalCompliance-Data-Archival
Compliance-Data-Archival
 
Best Practices for Amazon S3 and Amazon Glacier (STG203-R2) - AWS re:Invent 2018
Best Practices for Amazon S3 and Amazon Glacier (STG203-R2) - AWS re:Invent 2018Best Practices for Amazon S3 and Amazon Glacier (STG203-R2) - AWS re:Invent 2018
Best Practices for Amazon S3 and Amazon Glacier (STG203-R2) - AWS re:Invent 2018
 
Protect & Manage Amazon S3 & Amazon Glacier Objects at Scale (STG316-R1) - AW...
Protect & Manage Amazon S3 & Amazon Glacier Objects at Scale (STG316-R1) - AW...Protect & Manage Amazon S3 & Amazon Glacier Objects at Scale (STG316-R1) - AW...
Protect & Manage Amazon S3 & Amazon Glacier Objects at Scale (STG316-R1) - AW...
 
February 2016 Webinar Series - Use AWS Cloud Storage as the Foundation for Hy...
February 2016 Webinar Series - Use AWS Cloud Storage as the Foundation for Hy...February 2016 Webinar Series - Use AWS Cloud Storage as the Foundation for Hy...
February 2016 Webinar Series - Use AWS Cloud Storage as the Foundation for Hy...
 
SRV301 Latest Updates & Best Practices for Amazon S3
 SRV301 Latest Updates & Best Practices for Amazon S3 SRV301 Latest Updates & Best Practices for Amazon S3
SRV301 Latest Updates & Best Practices for Amazon S3
 
What's New with Amazon S3, Amazon EFS, and Other AWS Storage Services - STG20...
What's New with Amazon S3, Amazon EFS, and Other AWS Storage Services - STG20...What's New with Amazon S3, Amazon EFS, and Other AWS Storage Services - STG20...
What's New with Amazon S3, Amazon EFS, and Other AWS Storage Services - STG20...
 
Object Storage: Amazon S3 and Amazon Glacier
Object Storage: Amazon S3 and Amazon GlacierObject Storage: Amazon S3 and Amazon Glacier
Object Storage: Amazon S3 and Amazon Glacier
 
Cost efficiencies and security best practices with Amazon S3 storage - STG301...
Cost efficiencies and security best practices with Amazon S3 storage - STG301...Cost efficiencies and security best practices with Amazon S3 storage - STG301...
Cost efficiencies and security best practices with Amazon S3 storage - STG301...
 
AWS S3 Cost Optimization
AWS S3 Cost OptimizationAWS S3 Cost Optimization
AWS S3 Cost Optimization
 
SRV208 S3 One Zone-IA and S3 Select GA
SRV208 S3 One Zone-IA and S3 Select GASRV208 S3 One Zone-IA and S3 Select GA
SRV208 S3 One Zone-IA and S3 Select GA
 
What’s new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...
What’s new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...What’s new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...
What’s new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...
 
Building a Data Lake on S3 for IoT Workloads
Building a Data Lake on S3 for IoT WorkloadsBuilding a Data Lake on S3 for IoT Workloads
Building a Data Lake on S3 for IoT Workloads
 
STG301_Deep Dive on Amazon S3 and Glacier Architecture
STG301_Deep Dive on Amazon S3 and Glacier ArchitectureSTG301_Deep Dive on Amazon S3 and Glacier Architecture
STG301_Deep Dive on Amazon S3 and Glacier Architecture
 
What’s new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...
What’s new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...What’s new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...
What’s new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...
 
How to Build a Data Lake in Amazon S3 & Amazon Glacier - AWS Online Tech Talks
How to Build a Data Lake in Amazon S3 & Amazon Glacier - AWS Online Tech TalksHow to Build a Data Lake in Amazon S3 & Amazon Glacier - AWS Online Tech Talks
How to Build a Data Lake in Amazon S3 & Amazon Glacier - AWS Online Tech Talks
 
What's new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...
What's new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...What's new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...
What's new with Amazon S3, Amazon EFS, and other AWS storage services - STG20...
 
Deep Dive on Amazon S3
Deep Dive on Amazon S3Deep Dive on Amazon S3
Deep Dive on Amazon S3
 

Mehr von Amazon Web Services

Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
Amazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Amazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
Amazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Amazon Web Services
 

Mehr von Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Deep Dive on S3 Glacier Deep Archive

  • 1. P U B L I C S E C T O R S U M M I T WASHINGTON, DC
  • 2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Storing data long term with Amazon Simple Storage Service Glacier (Amazon S3 Glacier) Deep Archive Rob Czarnecki Principal Product Manager, Amazon S3 Glacier 3 2 0 9 9 6
  • 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Agenda • Amazon Simple Storage Service (Amazon S3) storage classes • Using Amazon S3 Glacier Deep Archive • AWS Partner Network integrations • Economics • Questions
  • 4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Services I’ll Mention Here Amazon Simple Storage Service (Amazon S3) Amazon Simple Storage Service Glacier (Amazon S3 Glacier) AWS Command Line Interface (AWS CLI) Amazon Simple Queue Service (Amazon SQS) Amazon Simple Notification Service (Amazon SNS) AWS Lambda (Lambda)
  • 5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Amazon S3 storage classes Access frequencyFrequent Infrequent • Active data • Millisecond access • > 3 AZ • $0.023/GB-mo. • Data with changing access pattern • Millisecond access • > 3 AZ • $0.023 to $0.0125/GB- mo. + object monitoring • 30-day min. duration • Inactive data • Millisecond access • > 3 AZ • $0.0125/GB • $0.01 per GB retrieved • 30-day min. duration • 128KB min. object S3 Standard S3 Standard-IA S3 One Zone-IA S3 Glacier • Inactive data • Millisecond access • 1 AZ • $0.01/GB • $0.01 per GB retrieved • 30-day min. duration • 128KB min. object • Archive data • Minutes to hours • > 3 AZ • $0.004/GB-mo. • $0.0025–$0.03/GB retrieved • 90-day min. duration S3 Intelligent- Tiering S3 Glacier Deep Archive • Archive data • Hours access • > 3 AZ • $0.00099/GB-mo. • $0.0025–$0.02/GB retrieved • 180-day min. duration
  • 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Amazon S3 Glacier Deep Archive No tape to manage Designed for 99.999999999% durability Recover data in hours vs. days/weeks $0.00099 per GB- month Less than 1/4 the cost of Amazon S3 Glacier
  • 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Common use cases for Amazon S3 Backup & recovery Easy place to start Integrated with all major vendors Pay only for what you use Photo & video Media archives and workflows Tape replacement Cost optimization with Amazon S3 storage classes Data lakes & analytics Broad analytics and visualization integrations Streaming or batch data Scalable storage and parallel processing
  • 8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T What is Amazon S3 Glacier Deep Archive? Designed to reduce costs on data you access once per year or less API matches Amazon S3 Glacier • Asynchronous • --storage-class DEEP_ARCHIVE Three differences • Standard retrievals in 12 hours • Bulk retrievals in 48 hours • No expedited retrieval, no S3 Glacier Select • Minimum object duration is 180 days
  • 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T
  • 10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Storing data in S3 Glacier Deep Archive Copy another S3 objectDirect PUT to Glacier Deep Archive Store data in any Amazon S3 storage class AWS Management Console AWS CLI AWS SDK Put object storage_class: DEEP_ARCHIVE Copy object storage_class: DEEP_ARCHIVE
  • 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Storing data in Amazon S3 Glacier Deep Archive Cross-region replication to Amazon S3 Glacier Deep Archive Lifecycle S3 objects Store data in any storage class Define lifecycle policies— tags, object age S3 continuously evaluates policy, shifts data Define CRR policies, with remote storage class Data replicated as it’s uploaded
  • 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Cross-region replication Automatically replicate data to any other AWS Region Configure by bucket, prefix, or tag values Replicate data to Glacier Deep Archive Change the object owner in the destination region
  • 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Lifecycle management Create rules to automatically transition or expire your storage Configure by bucket, prefix, or tag values Lifecycle rules take action based on object age Example policy: Move all objects older than 30 days to Amazon S3 Standard-Infrequent Access Move all objects older than 90 days to Amazon S3 Glacier Deep Archive
  • 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Lifecycle management Create rules to automatically transition or expire your storage Configure by bucket, prefix, or tag values Lifecycle rules take action based on object age Example policy: Move all objects older than 180 days to Amazon S3 Glacier Move all objects older than 730 days to Amazon S3 Glacier Deep Archive
  • 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T AWS tape gateway Benefits On premise iSCSI VTL Tape gateway HTTPS Application AWS Storage Gateway service (S3 Glacier Deep Archive) • Drop-in replacement for physical tape infrastructure • Allows you to backup & archive data to AWS without changing your backup applications or workflows • Reduces your TCO compared to using physical tapes and offsite warehousing • Extensive compliance and security features: HIPAA, PCI, SOC (1, 2, 3), ISO (9001, 27001, 27017, 27018) NEW OR (S3 Glacier) Tape library (Amazon S3) Tape shelf
  • 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Multiple data transfer services AWS SnowballAWS Snowmobile AWS Direct Connect AWS Transfer for SFTP AWS DataSync
  • 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Accessing data in Amazon S3 Glacier Deep Archive Restore, followed by a get Issue a restore request for the object (select speed, duration of restore) Amazon S3 restores object into Amazon S3 Reduced Redundancy Storage; notification issued Issue a Get request for the object (millisecond access) Use restore notification to drive workflows through Amazon SNS, Amazon SQS, or Lambda functions $ aws s3api restore-object --bucket mybucket --key dir1/example.obj -- restore-request '{"Days":25,"GlacierJobParameters":{"Tier":"Standard"}}'
  • 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Amazon S3 Glacier restore – Architecture pattern Lambda function Instance Amazon S3 Initiate restore Container Lambda function Object restored Amazon SNS Amazon SQS
  • 19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Multiple restore speeds Retrieval tier S3 Glacier S3 Glacier Deep Archive Expedited 1–5 mins* not available Standard 3–5 hours within 12 hours Bulk 5–12 hours within 48 hours
  • 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Archive partner integrations
  • 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T
  • 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Typical workflow 250TB stored 50,000 objects (avg. size 25GB) 0.5% accessed monthly
  • 23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Long-term retention with standard retrieval costs Amazon S3 Glacier Deep Archive PUT requests $2.50 (50,000 @ $0.05/1k) Storage $3,041.28 (250TB @ $0.00099/GB-mo.) Standard retrieval requests $0.30 (250 @ $0.025/1k per mo.) Standard retrieval bytes $307.20 (1.25TB @ $0.0025/GB per mo.) Total cost (12 months) $3,351.28 Amazon S3 Glacier PUT requests $2.50 (50,000 @ $0.05/1k) Storage $12,288.00 (250TB @ $0.004/GB-mo.) Standard retrieval requests $0.15 (250 @ $0.025/1k per mo.) Standard retrieval bytes $153.60 (1.25TB @ $0.0025/GB per mo.) Total cost (12 months) $12,444.71
  • 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Long-term retention with bulk retrieval costs S3 Glacier Deep Archive PUT requests $2.50 (50,000 @ $0.05/1k) Storage $3,041.28 (250TB @ $0.00099/GB-mo.) Bulk retrieval requests $0.075 (250 @ $0.025/1k per mo.) Bulk retrieval bytes $38.40 (1.25TB @ $0.0025/GB per mo.) Total cost (12 months) $3,082.255 S3 Glacier PUT requests $2.50 (50,000 @ $0.05/1k) Storage $12,288.00 (250TB @ $0.004/GB-mo.) Bulk retrieval requests $0.075 (250 @ $0.025/1k per mo.) Bulk retrieval bytes $38.40 (1.25TB @ $0.0025/GB per mo.) Total cost (12 months) $12,328.975
  • 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Compare to on-premises tape libraries Opportunity cost Tape generation migrations Administration & operations Offsite storage & transportation Capital: servers, libraries, tapes Hosting (space, power, cooling, network)
  • 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T How do the storage classes compare? Access times
  • 27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Storage cost to retrieval cost S3 S3-IA S3 Glacier S3 Glacier Deep Archive -$0.0050 $0.0000 $0.0050 $0.0100 $0.0150 $0.0200 $0.0250 $0.0300 -$0.0100 $0.0000 $0.0100 $0.0200 $0.0300 $0.0400 $0.0500 $0.0600 $0.0700 StorageCosts(perGB-mo.) Standard Retrieval Costs
  • 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Economics: Other factors to consider Frequency of access Object size
  • 29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T
  • 30. Thank you! © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T Rob Czarnecki czarnr@amazon.com
  • 31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.P U B L I C S E C T O R S U M M I T