OpenStack Austin Summit vBrownBag session on storage service chaining

1. Storage Is Not Virtualized Enough
Part 2: Let’s do service chaining !
Howard Huang@Huawei
Accela Zhao@EMC

2. Bio
Accela Zhao: Technologist at EMC OCTO (Office of CTO), 3 year experience in
Openstack and distributed storage, active Cinder contributor, former presenter
in Tokyo summit.
Howard Huang: Standard Engineer and open source community operation
manager from Huawei, involved in various open source projects like
OpenStack, OPNFV, Kubernetes, Hyperledger

3. Outline
❏ Motivation
❏ Overview of OpenStack Storage Function Chaining
❏ Future Plans

4. Motivation - 1: Flashback on part 1

5. Motivation - 2: Why do people need to chain
things
You know, the chain thing
- Salvatore Tessio

6. Motivation - 2: Why do people need to chain
things
➢ Peter deployed Ceph Block, deduplication and compression services in three
VMs. (Dedup and Comp might come from ZFS ZVOL)
➢ Now he wants to have a storage service that offers a compressed and
deduped Ceph Block, for his compute service.
➢ Naturally he could just chain the three storage service VMs he’s got together.
➢ How does he chain it ?

7. Outline
❏ Motivation
❏ Overview of OpenStack Storage Function Chaining
❏ OpenStack Network Function Chaining Reference
❏ OpenStack Storage Function Chaining Design
❏ Potential benefits
❏ Future Plans

8. OpenStack Neutron SFC

9. Outline
❏ Motivation
❏ Overview of OpenStack Storage Function Chaining
❏ OpenStack Network Function Chaining Reference
❏ OpenStack Storage Function Chaining Design
❏ Potential benefits
❏ Future Plans

10. OpenStack Storage Function Chaining Design
➢ General Ideas:
○ Utilize as many existing OpenStack Storage Services as possible
○ Borrow Ideas from Networking SFC
○ Minimal changes to the current modules
○ Try Block first
➢ Challenges:
○ Cinder attaches physical arrays for the VM, how do we make Cinder attach a VM for another
VM ?
○ Cinder does not dynamically add or remove a storage backend
○ Networking
○ Performance

11. OpenStack Storage Function Chaining Design
➢ Start discussion like a community:
○ http://www.meetup.com/Shanghai-SFV-Meetup/
➢ Initial thoughts discussed during the meetups:
○ Use LVM
■ Passed. Although LVM could store the data each time, you still need a way to link them
○ Use iSCSI
■ Passed. We will need static config, and no O/M functionality like Cinder involved.
○ Divide up separate LUNs on the array for each VM
■ Passed. Again, difficult and no OpenStack module involved to help

12. We need a module that could help couple these VMs together
OpenStack Storage Function Chaining Design

13. OpenStack Storage Function Chaining Design
Coupler
Mapping
Table
Policy
Consumer
Coupler Cinder Plugin
Allocator
● Coupler provides a virtual backend
service
○ Mapping Table : Provide a mapping for
storage service VM
○ Policy Consumer : Analyze policies injected
into Coupler
○ Allocator : Allocating fake IQNs so that for
each /dev/vdX they know which chain they
belong to
○ Cinder Plugin : expose as a “normal”
storage backend driver

14. OpenStack Storage Function Chaining Design

15. OpenStack Storage Function Chaining Design
● Coupler allows VMs to act as an virtualized storage backend, and provide volume mounts to another
● Multiple VMs can be chained flexibly to compose a new storage service
● Ceph can also be natively hosted inside Openstack. So as other storage backends
● Network part is tricky …

16. Outline
❏ Motivation
❏ Overview of OpenStack Storage Function Chaining
❏ OpenStack Network Function Chaining Reference
❏ OpenStack Storage Function Chaining Design
❏ Potential benefits
❏ Future Plans

17. Storage Service Chaining Potential Benefits
Decompose storage systems into
fine-grained functions, virtualized
into VMs.
For one type of function, there will
be VMs providing diverse algorithm
implementation.
The policy chains VMs to provide
customized storage services.
VMs comprise pools. They scale-out
to serve more users.
Users have the freedom to choose
what feature they want.

18. Storage Service Chaining Potential Benefits (2)

19. ● We could finally enable OpenStack to use a service that OpenStack itself
provided, just like k8se finally got something that would make storage
backends running as services provided by OpenStack, provided to other
OpenStack services, like what k8s already could do.
Storage Service Chaining Potential Benefits

20. Outline
❏ Motivation
❏ Overview of OpenStack Storage Function Chaining
❏ Future Plans

21. Future Plan
1. Try to code it out, and fix some of the challenges.
a. Coupler development : https://github.com/openstack/coupler
b. New use cases or requirements : https://github.com/OpenSFV/specs
c. Write the coupler driver for Cinder
2. File and Object ?
3. Standardization opportunities.
4. Possible collaboration with Kolla
5. Policy module study
6. Demo it out.

22. Thank You!
Contact us: accelazh@gmail, zhipengh512@gmail.com