This was presented at the GlusterFS Community Meetup on 21st Nov. 2014

1. GlusterFS &
Openstack Storage
GlusterFS Openstack
Integration Team
GlusterFS Meetup
Nov. 2014

2. Openstack architecture
11/21/14 GlusterFS Meetup 2

3. Openstack components
11/21/14 GlusterFS Meetup 3
Manila
(incubated)

4. OpenStack Swift +
GlusterFS
Swift On File
Prashanth Pai
ppai@redhat.com
IRC: ppai
11/21/14 GlusterFS Meetup 4

5. What is OpenStack Swift ?
Swift is a highly available, distributed, eventually consistent object store.
● Object is { Data + Meta-data + Identifier }
● Alternative to Amazon S3
● Decoupled/Independent from OpenStack
● Unstructured data
● Scales horizontally
But you can't..
● Mount it
● Have file hierarchies
Hierarchy
Account -> Containers -> Objects
11/21/14 GlusterFS Meetup 5

6. All operations are carried out by sending HTTP requests.
Each object’s access path consists of exactly three elements:
/account/container/object
GET
'X-Auth-Token: blah_blah'
http://127.0.0.1:8080/v1/AUTH_test/c1/a/b/c/d/z
Proxy-server
Account
Container
11/21/14 GlusterFS Meetup 6
Object
Request
method Request
header

7. Swift On File
● Allows objects created using Swift API to be accessed as files and vice-versa
● Map URL directly to file path
● Deployed as a Storage Policy
● Leverage distributed filesystem features
● Extend an existing Swift cluster you might already have
● Provide object interface to existing GlusterFS cluster
Example Usecases
Video transcoding, Analytics, Scientific Collaboration
11/21/14 GlusterFS Meetup 7

8. GlusterFS Cinder
Integration
Bharat Kumar K
Bharat.kobagana@redhat.com
IRC: BharatK
11/21/14 GlusterFS Meetup 8

9. What is Cinder?
● Cinder is the code name for the open source project to develop OpenStack Block
Storage.
● It is the Block-based storage component of OpenStack cloud computing.
● Cinder volumes provides persistent storage to guest virtual machine (aka instances),
that are managed by OpenStack compute.
● Block storage is a fundamental requirement for virtual infrastructure.
● It is the foundation for storing virtual machines and data used by those machine.
● It provisions storage in the form of block devices know as Cinder volumes.
● Cinder provides APIs such as create/delete volume, backup/restore volume,
snapshot/clone volume and many more. These APIs are implemented by the storage
backends that wish to support themselves as part of Cinder
11/21/14 GlusterFS Meetup 9

10. Cinder architecture
11/21/14 GlusterFS Meetup 10

11. What are Cinder Backends?
● Backend is a storage system which implement the APIs provided by
Cinder. Backend typically exports a storage pool which is used by
Cinder to provision new storage (aka cinder volumes). Eg: GlusterFS
volume, Ceph RBD pool etc.
● Cinder by default uses LVM (Logical Volume Manager) as its
backend.
● There are open-source storage solutions from GlusterFS and Ceph
that provide Cinder support using native GlusterFS protocol and Ceph
RADOS Block Device (RBD) respectively.
● Many storage array suppliers now provide Cinder block device
support. These include EMC, Hitachi Data Systems, HP, IBM and
NetApp.
● Backend is configured in cinder via the cinder.conf file
11/21/14 GlusterFS Meetup 11

12. Cinder - GlusterFS
● Available upstream
● 1 Cinder volume == 1 GlusterFS file
● Support raw and qcow2 format cinder volumes
● GlusterFS backed cinder volumes can be attached to Nova instances in 2
ways
● Fuse
● Libgfapi
● Supports most of the Cinder APIs (including libgfapi based snapshots)
● cinder.conf - glusterfs_shares_config lists the glusterfs volumes to use in
hostname:/volname format
● More info on configuring Cinder w/ GlusterFS is available @
http://docs.openstack.org/admin-guide-cloud/content/glusterfs_backend.html
● Support cinder multi-backend feature
11/21/14 GlusterFS Meetup 12

13. GlusterFS as Cinder backend
Storage Backend
GlusterFS
11/21/14 GlusterFS Meetup 13

14. Cinder – GlusterFS contd.
● TODOs
● Manage/Unmanage support – WIP
● Cinder - GlusterFS upstream CI – WIP
● backup/restore for qcow2 and disks with snaps
● Support for cinder volume replication
● Perf improvements - WIP
11/21/14 GlusterFS Meetup 14

15. Gluster Manila
Integration
Ramana Raja
Deepak C Shetty
rraja@redhat.com
deepakcs@redhat.com
IRC: rraja, deepakcs
11/21/14 GlusterFS Meetup 15

16. What is Manila
● Shared FileSystem as a service
● Incubated since openstack Juno
● Provision file shares to Nova (openstack compute) instance(s)
● Strives to provide an API for management of shared filesystems with
support for multiple protocols and backend implementations
● NFS and CIFS primarily supported
● Other protocols are encouraged too (eg: glusterfs)
● Supports Multi-tenancy
● Enables public cloud usecase
● Has framework to support storage backends that don't support
multi-tenancy natively
11/21/14 GlusterFS Meetup 16

17. Manila usecase
11/21/14 GlusterFS Meetup 17

18. Manila - GlusterFS
Two approaches
1. GlusterFS native driver
– protocol: 'glusterfs'
– access type: 'cert' based
2. GlusterFS (NFS/CIFS) driver
– protocol: NFSv3 (now), NFSv3/v4 and CIFS (later)
– access type: IP based
– mediated by GlusterNFS server (now),
NFS-Ganesha & SMB server (later)
11/21/14 GlusterFS Meetup 18

19. Manila access types
● IP
● Access control using IP address
● Takes IP as an argument
● Typically used in controlling access to NFS shares
● User
● Access control using user name
● Takes user name as argument
● Typically used in controlling access to CIFS shares
● Cert
● Access control using SSL certificates
● Takes SSL Certificate's CN (common name) as argument
● Certificate setup (aka trust setup) between client and server is out of band
● Currently implemented by GlusterFS native driver ('glusterfs' protocol)
11/21/14 GlusterFS Meetup 19

20. GlusterFS Native Driver
● Supports Certificate based access type of Manila
● Provision shares that use the 'glusterfs' protocol
● Instances directly talk with GlusterFS storage backend
● No service VM needed
● Secure access
● Only tenants with the right certificate will be able to access the
share
● Multi-tenant
● Separation using tenant specific certificates
● Supports certificate chaining and cipher lists
11/21/14 GlusterFS Meetup 20

21. GlusterFS Native Driver contd.
● Available upstream
● 1 Manila share == 1 GlusterFS volume
● Pre-requisites
– GlusterFS volume(s) setup with Cert based access enabled
– Instance should have server signed client certificates pre-loaded
– Manila.conf – Provide list of glusterfs volume(s) to work with
● TODOs
● Add documentation
● Snapshot support
● Dynamic creation of glusterfs volumes
● Data shredding as part of gluster volume delete
● Create share from snapshot
11/21/14 GlusterFS Meetup 21

22. Current GlusterFS Driver (NFS)
● Flat Network Driver
● A Manila Share maps to a subdirectory within
a GlusterFS volume
(Share layout may be changed in the future.)
● Shares exported by GlusterNFS (v3) residing
in the GlusterFS storage pool
11/21/14 GlusterFS Meetup 22

23. GlusterFS Driver (Ganesha)
● WIP: Flat Network Ganesha + GlusterFS driver
- NFSv3 and v4
- Modular architecture of Ganesha driver allows easy
integration of GlusterFS, and other drivers
● Future: Network Segmented Ganesha +
GlusterFS driver
- will be a fully open source, multi-tenant Manila stack
with a powerful distributed FS backend
11/21/14 GlusterFS Meetup 23

24. GlusterFS Ganesha Driver (WIP)
Flat Network Model
Tenant 1 Nodes Tenant 2 Nodes
Generic Ganesha
Driver
GlusterFS Driver
NFS-Ganesha Server
Gluster FSAL
Storage Backend
GlusterFS
Network
11/21/14 GlusterFS Meetup 24

25. GlusterFS Ganesha Driver (Future)
Network-segmented Model
Network 1
Storage Backend
GlusterFS
Tenant 1 Nodes
Ganesha
Generic Driver
Service VM
NFS-Ganesha Server
Gluster FSAL
NFS-Ganesha Server
11/21/14 GlusterFS Meetup 25
Network 2
GlusterFS Driver
Ganesha
Generic Driver
GlusterFS Driver
Service VM
Gluster FSAL

26. Summary
Deepak C Shetty
deepakcs@redhat.com
IRC: deepakcs
11/21/14 GlusterFS Meetup 26

27. Openstack + GlusterFS – Current Integration
Nova Nodes
(GlusterFS as
ephemeral storage)
Swift
Objects
Fuse/libgfapi
Block Images Files
Cinder
Data
Glance
Data
Swift API
Swift
Data
Storage
Server
(GlusterFS)
Storage
Server
(GlusterFS)
Storage
Server
(GlusterFS)
KVM
KVM
…
KVM
Manila
Data
Storage
Server
(GlusterFS)
Objects
11/21/14 GlusterFS Meetup 27

28. Hyper converged storage using GlusterFS - WIP
KVM
KVM
…
KVM
Fuse/libgfapi
Block Images
Cinder Data Glance Data
Compute +
File
Manila Data
Hadoop
11/21/14 GlusterFS Meetup 28
Swift
Objects
Object
Swift API
Swift Data
Storage
(GlusterFS)
Compute +
Storage
(GlusterFS)
Compute +
Storage
(GlusterFS)
Compute +
Storage
(GlusterFS)
Sahara Data
Compute +
Storage
(GlusterFS)

29. Thank You
Questions ?
11/21/14 GlusterFS Meetup 29