AWS Tech Talks: Armazenamento Híbrido na Nuvem

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.1
Melissa Ravanini
Arquiteta de Soluções
ravanini@amazon.com
Armazenamento na AWS

Agenda
• Introdução
• Conceitos básicos
• Armazenamento em blocos
• Armazenamento de objetos
• Armazenamento híbrido
• File System
• AWS Backup

© 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S U M M I T
Cenários híbridos de armazenamento na nuvem
TierizaçãoTransbordoBackup e Migração
Migrações em batch
Migrações recorrentes
Arquivamento
Melhor gestão dos
custos
Disaster Recovery
Capacidade sumplementar

S U M M I T
Por que mover para a Nuvem?
Componentes comuns para mover dados para a nuvem:
Pontos importantes na migração de dados:
Embora cada jornada de adoção da nuvem seja única, todas exigem uma etapa de
migração de dados, que pode ser prejudicada por bloqueadores, como:
Custo
efetivo
Produtividade
operacional
Agilidade
no Negócio
Resiliência
operacional
Acessos a
serviços
inovadores
Disrupção Velocidade Custos Canais Compatibilidad
e de nuvem

S U M M I T
Quadrante Mágico
Quadrante Mágico para Serviços
de Armazenamento em Nuvem
Pública, em todo o mundo –
2018
Posicionado para a visão
mais completa e mais alta
capacidade de execução em
cada relatório desde o início
em 2014
Magic Quadrant for Public Cloud Storage Services, July 2018 – Raj Bala, Julia Palmer
This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the
context of the entire document. The Gartner document is available upon request from Amazon Web Services.
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise
technology users to select only those vendors with the highest ratings or other designation. Gartner research
publications consist of the opinions of Gartner’s research organization and should not be construed as statements
of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any
warranties of merchantability or fitness for a particular purpose.

S U M M I T
Migração de dados: Cinco questões principais
1) Que tipo de dados são e para onde estão indo?
4) Quantos dados/tempo você tem?
2) Uma vez ou migração contínua?
3) Uma direção ou bidirecional?
5) E a WAN?
Amazon S3
Amazon EC2
Amazon
EFS
Amazon
EBS
Amazon
Aurora
Amazon
EMR
Amazon
Glacier
Amazon
RDS
AWS
IoT
Amazon
Redshift
Arquivos Volumes
de bloco
Bancos de
Dados
IoT Streams

S U M M I T
Conceitos básicos

Block vs File vs Object
Block Storage
Raw Storage
Data organized as an array of unrelated blocks
Host File System places data on disk
Ex: Hard Disks, Storage Area Network (SAN) Storage Arrays
File Storage
Unrelated data blocks managed by a file (serving) system
Native file system places data on disk
Ex: Network Attached Storage (NAS) Appliances, Windows File Servers
Object Storage
Stores Virtual containers that encapsulate the data, data attributes, metadata and Object IDs
API Access to data
Metadata Driven, Policy-based, etc.
Ex: Ceph, OpenStack Swift

Storage - Characteristics
Durability Availability Security Cost Scalability Performance Integration
Measure of
expected data
loss
Measure of
expected
downtime
Security
measures for
at-rest and in-
transit data
Amount per
storage unit,
e.g. $ / GB
Upward
flexibility,
storage size,
number of
users
Performance
metrics
(bandwidth)
Ability to
interact via API
or with other
services
Some of the ways we look at storage

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Montreal
Virginia
Ohio
GovCloud US West
California
Oregon
Sao Paulo
Dublim
Londres
Frankfurt
Paris
Mumbai
Singapura
Sydney
Tokyo
Seul
Beijing
Ningxia
Estocolmo
GovCloud US East
Cape Town (Em breve)
Milan (Em breve)
Hong Kong
Bahrain
Jacarta (Em breve)
Infraestrutura Global
22 REGIÕES
69 ZONAS DE
DISPONBILIDADE

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
REGION
~ 2ms latency
3
AZa
AZc
AZb
Redundant Tier-1 Internet &
Inter-Region Connectivity
TCTC
DC
DC
DC
DC
DCDC
DC
DC
DC

designed for
99.99%
durability
Two copies on one site
designed for
99.999%
durability
Copies on two sites
designed for
99.999999999%
durability
GlacierStandard IA
copies in three AZ
Understanding Durability

Availability vs Durability
% Availability Durability
99.999 5 minutes 15 seconds 1 in 100,000
99.9999 31 seconds 1 in 1,000,000
99.99999 3 seconds 1 in 10,000,000
99.999999999 300 uSeconds 1 in 100,000,000,000

Amazon FSx
for Lustre
More choice for more applications
Object storage
S3 Standard
S3 Glacier Deep Archive
S3 Intelligent-Tiering
S3 Glacier
S3 One Zone-IA
S3 Standard-IA
Block storage File storage
EFS Standard
EFS Infrequent Access
Amazon
EFS
AWS Storage
Gateway Family
Amazon S3
Amazon FSx
for Windows
File Server
Amazon EBS
Amazon EC2
Backup
General Purpose SSD
Provisioned IOPS SSD
Throughput-Optimized HDD
Cold HDD
FSx for Windows
FSx for Lustre

S U M M I T
Armazenamento de blocos

What is Amazon EBS?
• Block storage as a service
• Create, attach volumes through an API
• Service accessed over the network
EC2
instance
EBS
volume

Amazon EBS
Network attached block device
• Independent data lifecycle
• Multiple volumes per EC2 instance
• Only one EC2 instance at a time per volume
• Can be detached from an instance and attached to a different one
Raw block devices
• Unformatted block devices
• Ideal for databases, filesystems
Multiple Drive Types
• SSD (gp2 & io1) and HDD (st1 & sc1)

AWS EBS Features
Durable Secure
Low-latency SSD
Consistent I/O Performance
Stripe multiple volumes for
higher I/O performance
Identity and
Access Policies
Encryption
Scalable
Capacity when
you need it
Easily scale up
and down
Performance Backup
Designed for 99.999
reliability
Redundant storage
across multiple devices
within an AZ
Point-in-time Snapshots
Copy snapshots across AZ
and Regions

Amazon EBS volume types
HDDSSD
Provisioned IOPS
SSD
io1
General Purpose
SSD
gp2
Throughput Optimized HDD
st1 sc1
Cold HDD
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html

Amazon EBS use cases
HDDSSD
Relational Databases
MySQL, SQL Server,
PostgreSQL, SAP,
Oracle
NoSQL Databases
Cassandra, MongoDB,
CouchDB
Big Data , Analytics
Logging, Kafka,
Splunk, Hadoop, Data
Warehousing
File / Media
CIFS/NFS,
Transcoding,
Encoding, Rendering

Amazon EBS volume types: General Purpose SSD
gp2
Throughput: Up to 250 MiB/s
Latency: Single-digit ms
Capacity: 1 GiB to 16 TiB
Baseline: 100 to 16,000 IOPS; 3 IOPS per GiB
Burst: 3,000 IOPS (for volumes up to 1,000 GiB)
Great for boot volumes, low-latency applications, and bursty databases
General Purpose SSD

Amazon EBS volume types: Provisioned IOPS
io1
Baseline: 100–64,000 IOPS
Throughput: Up to 1,000 MiB/s
Latency: Single-digit ms
Capacity: 4 GiB to 16 TiB
Ideal for critical applications and databases with sustained IOPSProvisioned IOPS

EBS Snapshots
• Point-in-time snapshots of volume blocks
• Stored in Amazon S3 and accessed via EBS APIs
• Key Features:
• Immediate access to EBS volume data
• Resizing EBS volumes
• Sharing EBS Snapshots
• Copying EBS Snapshots across AWS regions

Internet
AWS Cloud
EBS Snapshots – Delta Block Changes
EC2 Availability Zone
EC2
Amazon S3
EBS
EC2 EC2
EBS EBS EBS EBS EBS
EBS Snapshot
EBS Snapshot
EBS Snapshot
Create Snapshot
Clone From
Snapshot

EBS Volume
How Do EBS Snapshots Work?
Time
Snapshot 1 Snapshot 2 Snapshot 3
S3
Block 1
Block 2
Block 3
Block 4
Chunk 1
Chunk 2
Chunk 3
Chunk 4
X

What is Amazon EC2 instance store?
• Local to instance
• Non-persistent data store
• Available on certain EC2
families
– Ex: d2, i3, r5d, m5d
• Data is not replicated (by
default)
• No snapshot support
• SSD or HDD
EC2 instances
Physical Host
Instance Store
or

Atachar um disco em múltiplas instâncias Novo!
https://aws.amazon.com/pt/about-aws/whats-new/2020/02/ebs-multi-attach-
available-provisioned-iops-ssd-volumes/
Instância Amazon
EC2
Volume
Provisioned IOPs
Instância Amazon
EC2

Lab1) Discos de Bloco - Amazon EBS:
https://go.aws/2O2iMXd
Atenção: não executar o passo 3. Limpe sua instância
https://dashboard.eventengine.run/

S U M M I T
Armazenamento de objetos

S U M M I T
Amazon
S3
Arquivos mestres de mídia
Registros de
mídia
Imagens e
registros
médicos
Dados de treinamento
de Machine learningDados de sensor
de IoT
Sequências de
DNA
Data
Lakes
Backups
duráveis
Preservação de
registros digitais
Dados sísmicos e
de reservatório
Gravações
caseiras
Pesquisa
meteorológica ou
ambiental
Origem de dados
para CDN
Registros de
transações
financeiras Hospedage
m de site
web
Imagens de vigilância
ou circuito de TV
interno
Sync de aplicativos
Dados de veículo
autônomo
Topografia da indústria de
Óleo e Gás
Registros de
Call Center
Mapas
Imagem geoespacial ou
lunarRegistros
de
Compliance
Dados de estudos
farmacêuticos
Conteúdo gerado
pelo usuárioArquivo
s de log
Analytics

Standard
Dados quentes Dados arquivadosDados acessados de forma infrequente
Infrequent Access Amazon Glacier
Opções de classes de armazenamento no S3

Permitem que você mova entre as camadas de armazenamento ou remova objetos de
acordo com suas regras de negócio
Políticas do ciclo de vida dos objetos no S3
Objetos no
Amazon S3
60 dias
AmazonGlacier
-
arquivamento
365 dias

S U M M I T
1 PB storage “raw”
800 TB storage utilizável
600 TB storage alocado
400 TB com dados
de aplicação
Utilização de Storage – Pague pelo que você usa
Storage Tradicional Storage na
AWS

Visualize o padrão de acesso a seus dados

S U M M I T
Amazon S3 Intelligent-Tiering
Tierização automática de dados com base em padrões de acesso
Novo!

S U M M I T
Amazon S3 Glacier Deep Archive
Storage mais barato para arquivamentos de longo prazo e preservação
de acervo digital
Totalmente
gerenciado sem o
fardo de gerenciar fita
$0.00099 por
GB-mês
Desenhado para
fornecer
99.999999999% de
durabilidade
Recupere
dados em
12 horas
Novo!

Your choice of object storage classes
Access FrequencyFrequent Infrequent
• Active, frequently
accessed data
• Milliseconds access
• > 3 AZ
• $0.0210/GB
• Data with changing
access patterns
• > 3 AZ
• $0.0210 to
$0.0125/GB
• Monitoring fee per
Obj.
• Min storage duration
• Infrequently
accessed data
• > 3 AZ
• $0.0125/GB
• Retrieval fee per GB
• Min object size
S3 Standard S3 Standard-IA S3 One Zone-IA S3 Glacier
• Re-creatable, less
accessed data
• 1 AZ
• $0.0100/GB
• Min object size
• Archive data
• Select minutes or
hours
• > 3 AZ
• $0.0040/GB
• Min object size
S3 Intelligent-Tiering
S3 Glacier
Deep Archive
• Archive data
• Select 12 or 48
hours
• > 3 AZ
• $0.00099/GB
• Min object size

Lab2) Armazenamento de Objetos - Amazon S3:

S U M M I T
Armazenamento híbrido

Many Options for Data Transfer
AWS
Direct Connect
Amazon
Kinesis
Firehose
AWS
Snowball
AWS
Snowmobile
AWS
Storage
Gateway
Amazon S3
Transfer
Acceleration
AWS
DataSync
AWS
Transfer
for SFTP
AWS
Snowball Edge
Amazon
Kinesis
Data Streams
Amazon Kinesis
Video Streams
AWS
Database
Migration
Service

S U M M I T
AWS Storage Gateway
Arquivos
existentes
Dados
novos
Data Centers ou
Escritórios
corporativos
AWS Storage Gateway
HTTPS
(Internet ou DX)
NFS
SMB
iSCSI
VTL
Serviço de armazenamento híbrido que permite que os aplicativos usem o
armazenamento AWS
On-Premises Gateway Service
Amazon S3
Amazon EBS
snapshots
Amazon Glacier

S U M M I T
A família AWS Storage Gateway
Gateway de Arquivos
Armazene e acesse objetos no Amazon S3 a partir de
aplicações baseadas em arquivos com cache local
Gateway de Volumes
Armazenamento de blocos on-premises com armazenamento na nuvem e
cache local, snapshots e clones do Amazon EBS
Gateway de Fitas
Substituição direta de infraestrutura de fita física com suporte de
armazenamento em nuvem e cache local
Três tipos de armazenamento: arquivo, bloco e fita

S U M M I T
Características principais do AWS Storage
Gateway
Protocolos de
armazenamento padrão
Conecte perfeitamente aplicações locais ao
armazenamento da AWS sem modificar as
aplicações
Cache totalmente
gerenciado
Dados usado frequentemente em cache
local para acessos com baixa latência
Transferência otimizada
de dados
Otimiza o upload de dados alterados e
downloads de dados solicitados
Integrado com AWS
Gerencie gateways locais no Console de
Gerenciamento da AWS e use seus dados em todo o
portfólio de aplicações e serviços da AWS

File gateway
On-premises file storage maintained as objects in Amazon S3
Customer Premises
File
Gateway
• Data stored and retrieved from your S3 buckets
• One-to-one mapping from files-to-objects
• File metadata stored in object metadata
• Bucket access managed by IAM role you own and manage
• Use S3 Lifecycle Policies, versioning, or CRR to manage data
GlacierS3
Standard
S3
Standard -
Infrequent
Access
HTTPS
NFS
v3 / v4.1
Application
Server

Application
Server
Volume gateway
On-premises volume storage backed by Amazon S3 with EBS snapshots
Block storage in S3 accessed via the volume gateway
Data compressed in-transit and at-rest
Backup on-premises volumes to EBS snapshots
Create on-premises volumes from EBS snapshots
Up to 1PB of total volume storage per gateway
Amazon EBS
snapshots
Storage Gateway
bucket in
Amazon S3
Customer Premises
Volume
Gateway
iSCSI HTTPS

What is Snowball?
Terabyte scale data transport
E-ink shipping
label
Ruggedized
case
“8.5G Impact”
All data encrypted
end-to-end
80 TB
10G network
Rain & dust
resistant
Tamper-resistant
case & electronics

Amazon Snowmobile
https://www.youtube.com/watch?v=8vQmTZTq7nw

Lab3) Storage Híbrido - AWS Storage Gateway:

S U M M I T
Data center do cliente Nuvem da AWS
AWS Storage
Gateway
AWS Storage
Gateway - OVA
Amazon Simple
Storage Service
(S3)
Usuário
Disco em D:/ HTTPS
Cache local
gerenciado

PERFORMÁTICO
Amazon EFS
Sistema de arquivos Linux nativo da nuvem, escalável e elástico
ACESSO
COMPARTILHADO
ALTAMENTE
DISPONÍVEL E
DURÁVEL
ELÁSTICO e
ESCALÁVEL
SEGURO E
COMPATÍVEL
CLASSES DE
STORAGE
N o v o !

EFS Infrequent Access
Classe de armazenamento mais barata para arquivos menos
acessados
Políticas de ciclo de vida do EFS
85% de economia para arquivos
acessados <1 vez por mês
EFS Standard EFS IA
Arquivos
frequentemente
acessados
Arquivos
infrequentemente
acessados

TCO: Faça-o-seu-próprio vs Amazon EFS
500GB em Alta Disponibilidade
Compute Inter-AZ Bandwidth Storage
$350/mês
$30/mês
$205/mês
$150/mês
$585/mês

Example use cases
Big Data Analytics
Media Workflow Processing
Web Serving
Content Management
Home Directories

EFS – Mounting
EFS
EC2EC2 EC2 EC2EC2
EFS DNS Name
availability-zone.file-system-id.efs.aws-region.amazonaws.com
Mount on machine
sudo mount -t nfs4 mount-target-DNS:/ ~/efs-mount-point
EC2

Performance and Throughput
• Two performance modes: General Purpose and Max I/O.
• General Purpose - lowest latency per file system operation and can achieve
this for random or sequential IO patterns.
• Max I/O - higher levels of aggregate throughput and operations per
second, ideal for highly parallelized applications that can scale out to
thousands of Amazon EC2 instances
• Two throughput modes: Bursting and Provisioned
• Bursting - scales with the size of the file system (50 KB/s per GB), and
bursts as needed to support the spiky nature of many file-based
workloads.
• Provisioned - designed to support applications that require higher
dedicated throughput than the default Bursting mode and can be
configured independently of the amount of data stored on the file system.

EFS Lifecycle Management
• EFS offers both Standard and Infrequent Access (IA)
storage classes
• With Lifecycle Management enabled, EFS automatically
moves files not accessed for 30 days from the Standard
storage class to the EFS IA storage class

FSx for Windows
Integrated with
AWS
Fully managed Windows
file systems …
… built on Windows
Server

Native Windows compatibility and features
Native Windows
compatibility
DFS
Namespaces
and
DFS Replication
Integrates with
Microsoft AD
and supports
Windows ACLs
AD
NTFS
Windows Server
Native SMB
2.0 to 3.1.1
SMB

AWS Backup launched in January 2019 to centrally
manage and automate backups across AWS services
AWS EBS
AWS RDSAWS
Storage
Gateway
AWS
Elastic
Filesystem
AWS
DynamoDB

AWS Backup provides centralized backup,
compliance, and automation across AWS Services
Automated lifecycle
and retention
management
Operate across
multiple services
Centralized
monitoring and
security
Manage and monitor backup
jobs and IAM policies / roles
Simplify data retention for
different backup tiers
Create plans to cover
more than one service

AWS Backup supports the following features
Lifecycle backups
to cold storage
Automated backup
scheduling
Policy and tag-based
backup of resources
Centralized backup activity
monitoring and logs
Automated backup
retention
management
Backup access
policies
Backup encryption
(independent of source
resource)

Automated backup schedules
Automated retention management
Centralized backup monitoring/logging
Lifecycle to cold storage
Independent backup encryption
Since launch, AWS Backup protects 5 key AWS
services
✓ ✓ ✓ ✓ ✓
✓ ✓ ✓ ✓ ✓
✓ ✓ ✓ ✓ ✓
✓
✓
Amazon Elastic
Block Store
Amazon Elastic
File System
Amazon
RDS
Amazon
DynamoDB
AWS Storage
Gateway

Any Questions?

Responda à pesquisa de satisfação para ganhar
$30 em créditos AWS

AWS Tech Talks: Armazenamento Híbrido na Nuvem

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie AWS Tech Talks: Armazenamento Híbrido na Nuvem

Ähnlich wie AWS Tech Talks: Armazenamento Híbrido na Nuvem (20)

Mehr von Amazon Web Services LATAM

Mehr von Amazon Web Services LATAM (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

AWS Tech Talks: Armazenamento Híbrido na Nuvem