Yair Hershko - Building Software Defined Storage Cloud Using OpenStack
1. Zadara Storage Confidential
Building a
Software Defined Storage Cloud
using OpenStack
Yair Hershko
VP Engineering
yair@zadarastorage.com
Not For Distribution
2. Zadara Storage Confidential
Agenda
• About Zadara Storage
• Traditional Storage Systems
• Software Defined Storage
• SDS Architecture using OpenStack
Not For Distribution
3. Zadara Storage Confidential
About Zadara Storage
• Founded early 2011. Raised $7M to date.
• Team has worked together on Enterprise Storage Systems
&Virtualization for over 12 years (StoreAge, LSI)
• Primary Block & File Enterprise Storage-as-a-Service
• Available at AWS & Equinix
• Adopted by Dimension Data and in evaluation by 20+ clouds
• Customers include 3 Fortune 50 and many Fortune 500
Not For Distribution
4. Zadara Storage Confidential
Example:
SAN Array
Categories
Monolithic
Multi Node
Dual Controller
Traditional Enterprise Storage
Criteria for selection
Performance
Latency
Throughout
IOPs
Capacity
Redundancy & Availability
Data Management services
Business Continuity
Data Migration
Capacity Efficiency
Not For Distribution
5. Zadara Storage Confidential
Example:
SAN Array
Downside
Costly
Min 24 month lock-in
Up front costs (CapEx)
Not easily modified
Not Multi-Tenant
Single Management & Control
Single performance & uptime characteristics
No chargeback per Tenant usage
Flexibility
Traditional Enterprise Storage
Not For Distribution
6. Zadara Storage Confidential
CPU 32GB
Memory
Data Controller
CPU
CPU
CPU
CPU 32GB
Memory
Data Controller
CPU
CPU
CPU
NIC NIC NIC NICNIC NIC NIC NIC
Scale-out with Monolithic & Dual-Control Systems
• Define storage capacity & QoS to meet business requirement
Not For Distribution
7. Zadara Storage Confidential
CPU 32GB
Memory
Data Controller
CPU
CPU
CPU
CPU 32GB
Memory
Data Controller
CPU
CPU
CPU
CPU 32GB
Memory
Data Controller
CPU
CPU
CPU
CPU 32GB
Memory
NIC
Data Controller
CPU
CPU
CPU
NIC NIC NICNIC NIC NIC NIC
NIC NIC NIC NICNIC NIC NIC NIC
Scale-out with Monolithic & Dual-Control Systems
• Define storage capacity & QoS to meet business requirement
• Business expands. Need more storage
Not For Distribution
8. Zadara Storage Confidential
CPU 32GB
Memory
Data Controller
CPU
CPU
CPU
CPU 32GB
Memory
Data Controller
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU 32GB
Memory
Data Controller
CPU
CPU
CPU
CPU 32GB
Memory
NIC
Data Controller
CPU
CPU
CPU
CPU
CPU
CPU 256GB
MemoryCPU
CPU
CPU
CPU
CPU
NIC NIC NICNIC NIC NIC NIC
NIC NIC NIC NICNIC NIC NIC NIC NIC NIC NIC NIC
256GB
Memory
Data Controller Data Controller
NIC NIC NIC NIC NICNIC
Scale-out with Monolithic & Dual-Control Systems
• Define Storage Capacity & QoS to meet business requirement
• Business expands. Need more storage
• Business changes. Need different storage
Not For Distribution
9. Zadara Storage Confidential
Data Silos with Monolithic & Dual-Control Systems
• Business is dynamic. Keep adding Storage Silos
• Difficult to manage.
• Eventually leading to low storage utilization and efficiency
Not For Distribution
Not For Distribution
10. Zadara Storage Confidential
Multi-node Scale-out Architecture
`
NIC NIC NIC NICNIC NIC NIC NIC
CPU
mem
CPU CPU CPU CPU CPU CPU CPU
mem mem mem mem mem mem
`
NIC NIC NIC NICNIC NIC NIC NIC
CPU
mem
CPU CPU CPU CPU CPU CPU CPU
mem mem mem mem mem mem
`
NIC NIC NIC NICNIC NIC NIC NIC
CPU
mem
CPU CPU CPU CPU CPU CPU CPU
mem mem mem mem mem mem
Pros
• Independent storage nodes.
• Easy to manage Scale-out
• Survive the loss of a storage node
Cons
•Single management & control
•Single QoS
•Chargeback
Not For Distribution
12. Zadara Storage Confidential
What is Software-Defined Storage?
• Abstraction of storage services and capabilities from
the underlying physical storage resources
• Hardware Agnostic
• Evolution of storage virtualization
• Native API integration
Not For Distribution
13. Zadara Storage Confidential
SDS for the Cloud
1. Single-Tenant experience in a multi-tenant
environment
– QoS (performance & data management services)
– Fault isolation
– Secure
– Control
– Chargeback
2. Storage management at scale
3. Agility and Flexibility
Not For Distribution
14. Zadara Storage Confidential
The Challenge
• Architect a Storage-as-a-Service which is
– Defined by Software
– Enterprise-level Block, File and Object
– Cloud Scale & Economics
– Provides a Single-Tenant Experience in a Multi-
Tenant Environment
Not For Distribution
15. Zadara Storage Confidential
The Solution
`
NIC NIC NIC NICNIC NIC NIC NIC
CPU
mem
CPU CPU CPU CPU CPU CPU CPU
mem mem mem mem mem mem
`
NIC NIC NIC NICNIC NIC NIC NIC
CPU
mem
CPU CPU CPU CPU CPU CPU CPU
mem mem mem mem mem mem
`
NIC NIC NIC NICNIC NIC NIC NIC
CPU
mem
CPU CPU CPU CPU CPU CPU CPU
mem mem mem mem mem mem
• Commodity Hardware
• Cloud Orchestration stack to pool and manage *ALL* storage resources
(drives, memory, CPUs, NICs)
• Virtual Private Storage Array (VPSA) Per Tenant
Not For Distribution
16. Zadara Storage Confidential
Virtual Private Storage Array (VPSA)
`
NIC NIC NIC NICNIC NIC NIC NIC
CPU
mem
CPU CPU CPU CPU CPU CPU CPU
mem mem mem mem mem mem
`
NIC NIC NIC NICNIC NIC NIC NIC
CPU
mem
CPU CPU CPU CPU CPU CPU CPU
mem mem mem mem mem mem
`
NIC NIC NIC NICNIC NIC NIC NIC
CPU
mem
CPU CPU CPU CPU CPU CPU CPU
mem mem mem mem mem mem
Virtual Controller Virtual Controller
Define Storage QoS
Provision and Spawn Virtual Controllers
Provision Drives
Setup Networking (FE, BE, MGM)
Not For Distribution
21. Zadara Storage Confidential
Open Stack Logical Architecture
Vsa-api
vsa-scheduler
vsa-volume
driver
vsa
database
Not For Distribution
22. Zadara Storage Confidential
Virtual Controller Elasticity
• Auto scale-up & scale-down
– Meter & monitor resources utilization
• CPU, Memory, Network bandwidth, Disk IO Wait, Disk capacity
– Fully automated API
– No downtime
• Virtual Controller online migration
– Load Balancing
– Maintenance
Not For Distribution
23. Zadara Storage Confidential
VPSA Key Value proposition
• High Consistent Block & File Performance
• Highly Available & Redundant
• User Control - Rest API and GUI
• Thin Provisioning
• Unlimited instant Snapshots
• Instant clones
• Remote Replication (DR)
Not For Distribution
24. Zadara Storage Confidential
Private
Consistent QoS
Data Management
Control & Privacy
Flexibility
Automation
Cloud Orchestration
Resource management
Evolving Cloud Ecosystem
Cloud Scale
Resource metering & chargeback
Automation
Traditional SAN/NAS Benefits + Cloud Elasticity and Ecosystem
Enterprise Storage-as-a-Service
Not For Distribution
25. Zadara Storage Confidential
THANK YOU
Yair Hershko
VP Engineering
+972 (4) 688-6262 Office
+972 (52) 474-5020 Mobile
yair@zadarastorage.com