Agenda: What is Software Defined Storage? What is Ceph? What is Rook? Storage for Kubernetes Storage Classes Storage on Kubernetes Operator Pattern Custom Resource Definition Rook Operator Rook architecture Ceph on Kubernetes with Rook Demo Rook Framework for Storage solutions How to Get Involved?

1. Introduction to Rook
Rohan Gupta
GSoC 2018 with CNCF
https://rook.io/
https://github.com/rook/rook

2. Agenda
● What is Software Defined Storage?
● What is Ceph?
● What is Rook?
● Storage for Kubernetes
● Storage Classes
● Storage on Kubernetes
● Operator Pattern
● Custom Resource Definition
● Rook Operator
● Rook architecture
● Ceph on Kubernetes with Rook
● Demo
● Rook Framework for Storage solutions
● How to Get involved?

3. What is Software Defined Storage (SDS)
● An abstraction layer that hides the complexities of the underlying storage.
● Provides mechanism for storage provisioning.
● E.g. NFS, Ceph, Gluster etc.

4. Why SDS?
● Operational - Manage provisioning process and data independent of
underlying hardware
● Physical - Abstract consumed logical storage from underlying physical
storage
● Policy - Automation of policy driven both external (users) and internal
(platform)
● Day 2 Operations - Maintenance is inherently different

5. What is CEPH?
● Ceph is an open-source, massively scalable, software-defined storage
system
● It provides object, block and file system storage in a single platform.
● It runs on commodity hardware—saving you costs, giving you flexibility.

6. Ceph Architecture

7. Components of CEPH
● Monitors
● Managers
● Ceph OSDs
● MDSs

8. How will the cluster look like?

9. ● Cloud-Native Storage Orchestrator
● Extends Kubernetes with custom types and controllers
● Automates deployment, bootstrapping, configuration,
provisioning, scaling, upgrading, migration, disaster recovery,
monitoring, and resource management
● Framework for many storage providers and solutions
● Open Source (Apache 2.0)
● Hosted by the Cloud-Native Computing Foundation (CNCF)
What is Rook?

10. Storage for Kubernetes
● Volume plugins allow external storage solutions to provide
storage to your apps

11. Limitations
● Not portable: requires these services to be accessible
● Deployment burden of external solutions
● Vendor lock-in due to using provider managed services

12. Storage Classes
● SC provides dynamic provisioning of volumes in kubernetes.
● Allow admins to define abstractions for the underlying storage platform
● Eliminates the need for cluster administrators to pre-provision storage

13. How does it look like?
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: myStorageClass
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2

14. How to get volume from a SC?
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage: 8Gi
storageClassName: myStorageClass

15. Storage ON Kubernetes
● Kubernetes can
manage our storage
solution
● Highly portable
applications (including
storage dependencies)
● Dedicated K8s storage
cluster also possible

16. Portable Abstractions
● Concept that describes the general data need of your
application
● Database, message queue, cache, object store, etc.
● Works across environments and providers
○ RDS in Amazon
○ Cloud SQL in Google
● Choose the best environment for your code
○ cost, features, resiliency, compliance

17. Operator Pattern
● Codifies domain expertise to deploy and manage an
application
○ Automates actions a human would normally do
● Control loop that reconciles user’s desired state and the
actual system state
○ Observe - discover current actual state of cluster
○ Analyze - determine differences from desired state
○ Act - perform operations to drive actual towards desired

18. ● Teaches Kubernetes about new first-class objects
● Custom Resource Definition (CRDs) are arbitrary types that
extend the Kubernetes API
○ look just like any other built-in object (e.g. Pod)
○ Enabled native kubectl experience
● A means for user to describe their desired state
Custom Resource Definitions (CRDs)

19. ● Implements the Operator Pattern for storage solutions
● Defines desired state for the storage cluster
○ Storage Cluster, Pool, Object Store, etc.
● The Operator runs reconciliation loops
○ Watches for changes in desired state
○ Watches for changes in the cluster
○ Applies changes to the cluster to make it match desired
Rook Operators

20. Rook Operators
● The Operators leverages the full power of K8S
○ Services, ReplicaSets, DaemonSets, Secrets, …
● Contain all the logic to manage storage systems at scale
○ Handle stateful upgrades
○ Handle rebalancing the cluster
○ Handle health and monitoring tasks
● Not on the data path – can be offline for minutes

21. Rook Architecture
Rook
Operators
Kubernetes
API
New Objects:
Storage
Clusters
Storage Pools
Object Store
File Store
Objects:
Deployments
DaemonSets
Pods
Services
StorageClass / PV / PVC
ClusterRole
Namespace
Config Maps
Kubelet
Rook Flex Volume
Plugin
Daemons
kubectl
New Objects:
Volume
Attachments
Management & Health API
Rook
Agen
t

22. Ceph on Kubernetes with Rook
MON
MON
MON
OSD OSD OSD
OSD
MGR

23. Demo
Deploying a Ceph cluster with a
Stateful Application

24. Rook Framework for Storage Solutions
● Rook is more than just a collection of Operators and CRDs
● Framework for storage providers to integrate their solutions
into cloud-native environments
○ Storage resource normalization
○ Operator patterns/plumbing
○ Common policies, specs, logic
○ Testing effort
● Ceph, CockroachDB, Minio, Nexenta, and more...

25. ● Contribute to Rook
○ https://github.com/rook/rook
○ https://rook.io/
● Slack - https://rook-io.slack.com/
● Twitter - @rook_io
● Forums - https://groups.google.com/forum/#!forum/rook-dev
● Community Meetings
How to get involved?

26. Questions?
https://github.com/rook/rook
https://rook.io/

27. Thank you!
https://github.com/rook/rook
https://rook.io/