Adam's slides from his talk at the CloudStack European User group meetup, March 13, London. To provide tighter integration between the S3 compatible object store and CloudStack, Cloudian has developed a connector to allow users and their applications to utilize the object store directly from within the CloudStack platform in a single sign-on manner with self-service provisioning. Additionally, CloudStack templates and snapshots are centrally stored within the object store and managed through the CloudStack service. The object store offers protection of these templates and snapshots across data centres using replication or erasure coding.
4. Data fuels the transformation to digital enterprise
More connected people, apps, and things generating more data in many forms
Human data
Sensor data
Business data
faster growth
than traditional
business data
10x
90% of the world’s data was created in the last 2 years.
10. S3 Functionality
S3 is a modern storage protocol combining
• Connectivity
• Access control
• Data management
• Reporting & Notifications
• Advanced Functionality
• Tiering
• Encryption
• Billing
• Monitoring
• Notifications
• Replication
• Time limited access
…. that can be used across the internet.
Standard
Storage
Protocols
S3 API
12
12. 14
Why S3 Compatibility Matters
1. Cloud storage standard established at Amazon; world’s largest object
storage environment
2. MSPs worldwide adopting S3 for cloud storage services
3. Hundreds of applications now support S3, many more in development
S3 Compatibility = Investment Protection
13. Unstructured Data Consolidation – additional use cases
Backup
File
NAS Offload
Media Asset Mgmt
Sync and Share
AI / Machine Learning
S3 API
16. 18
HyperStore Node Hardware
CPU HDD Drives Network SSD Drives
• CPUs – CPU Clock speed is preferred over
number of cores, but of course the more cores
the better.
• Memory – More memory is required as a
node has more drives/capacity
• OS/Meta-data disks - More capacity is
required for meta-data storage as a node has
more drives/capacity
• Network Interface – For better performance it
is highly recommended to deploy nodes with 2
x 10GbE interfaces as a minimum, more ports
can be utilized by bonding ports together for
resilience and greater bandwidth capability.
RAM
20. Peer-to-peer system = no SPOF
Distributed Everything = Data , Metadata, Configuration
User Defined
Location Affinity
DC1
DC2
Add Node <-> Auto Rebalance
Server <-> vNodes <-> Disks
Distributed & Elastic Geo Cluster
21. 23
Support for Multiple Configurations
Storage MSP
On-Premises
Apps
Multi-Site
Hybrid
Apps
Stand alone, on-premise storage Replicate or migrate to the cloud for capacity, data
protection, content distribution
Fully S3-compatible storage servicesReplicate or distribute data across sites
SITE A
SITE B SITE C
27. 29
Versioning and WORM support
• Accidental deletion
• Ransomware
• Rouge admins
• Malicious attacks
Enabled at the Bucket Level, versioning can provide a
safeguard against:
When using WORM objects in the bucket cannot
be deleted, overwritten or modified from any
source or by any command until the retention
period has expired.
Once locked, the bucket lock policy cannot be
deleted from the bucket.
The bucket can only be deleted when the retention
period for all objects in the bucket has expired and
the objects have been deleted
30. 32
Encryption
Object
S3 bucket
SSL
Object
Master key
Data keys
Server side Encryption key (regular SSE)
• Server manages master key and generates
per-object key that’s stored in object metadata.
• Can be integrated with an external key
management system.
Object
Per-object key
Encrypt S3 bucket
SSL
Customer-provided encryption key
(SSE-C)
• Encryption key is never stored.
• Customer must use same key on PUT
and GET.
Object
Encrypted
Object & key S3 bucket
Master key
AWS SDK
Envelope keys
Client-side Encryption
• Client provides and manages master keys.
• With AWS SDK, dynamic “envelope” key
generated and used to encrypt object and key.
• Encrypted envelope key is sent and stored as
object metadata, and checked on retrieval.
KMS Support for SSE
• Supported with Gemalto KeySecure
37. NFS Support
• NFS Gateway support is offered via HyperFile
• Allows access to same object via S3 or NFS
• Suitable use cases :
• VM templates/snapshots storage
• Backup workloads (reading/writing large files)
• Media Repository Sharing
S3
NFS
39
38. CloudStack Integration – SSO
•Integrated Management GUI
•Integrated provisioning
• Store OS images & snapshots
• Single Sign-on
40
39. •Open standards approach – full S3 compatibility
•Store templates, ISO images and snapshots on a multi-tenant hybrid object
storage platform
•Share data between user projects and virtual machine instances
•Built in advanced data protection features including:
•Versioning
•Erasure Coding
•Replication
•DR with multi DC support, multiple regions and tunable consistency settings
41
Key Benefits