Slides of my presentation "Ceph and Storage Management with openATTIC" held at FrOSCon in St. Augustin (Germany) on 2016-08-21. A video recording can be found here: https://media.ccc.de/v/froscon2016-1755-open_source_storage_management_mit_openattic

1. Ceph and Storage Management
with openATTIC
FrOSCon 2016, St. Augustin, Germany
2016-08-21
Lenz Grimmer <lenz@openattic.org>

2. 2
openATTIC – Our Vision
 Develop an open source alternative to proprietary
storage management systems
 „Traditional” unified storage (NAS/SAN)
 Support Ceph for scale-out scenarios
 Backed with commercial support and services

3. 3
openATTIC – Notable Recent Changes
 Removed Enterprise/Community Edition split
 Now fully under the GPLv2
 Removed requirement for CLA
 DCA (Signed-off-by) is all that's needed
 Public Jira Bug Tracker (Issue tracking & roadmap)
 Public pull requests / code reviews on BitBucket
 Development/Release Branch
 Monthly releases / nighly snapshot builds
 Entire code base (Backend/WebUI/Test/Docs) in one
branch

4. 4
openATTIC – What Sets us Apart?
 Focus on data center storage management
 Support both SAN and NAS functionality without limitations
 Fully Open Source (GPLv2)
 No arbitrary functional restrictions
 Low entrance barrier for adoption
 Based on Linux / OSS tools
 Multiple Linux distributions (Debian/Ubuntu/Red
Hat/SUSE)
 Well-established technology stack (e.g. drivers, hardware
support)
 Broad user base

5. 5
 Modern Web UI
 RESTful API (Software-
Defined Storage)
 Unified Storage
 NAS (NFS, CIFS, HTTP)
 SAN (iSCSI, Fibre Channel)
 LVM, XFS, ZFS, Btrfs,
ext3/4
 Volume mirroring (DRBD®)
 Multi-node support
 Monitoring (Nagios/Icinga)
built-in
 Ceph management and
monitoring (WIP)
 Development sponsored
by it-novum
openATTIC – Open Source Storage Management

6. 6
 Backend
 Python (Django)
 Django REST Framework
(RESTful API)
 Linux tools for storage
management, e.g. LVM, LIO,
filesystem utilities, DRBD,
etc.
 Nagios/Icinga &
PNP4Nagios (Monitoring
and Graphing)
 Web Frontend
 AngularJS (JS framework)
 Bootstrap (HTML, CSS, and
JS framework)
 Uses REST API exclusively
 Automated Test Suites
 Python Unit Tests
 REST Backend Tests
(Gatling)
 WebUI Tests
(Protractor/Jasmine)
openATTIC – Components

7. 7
openATTIC – High Level Architecture

8. 8
openATTIC – Storage Development Roadmap
 Add Disk and Storage Pool Management to the API &
WebUI
 Creating/Modifying LVM Volume Groups / MD RAID setups
 Creating/Modifying Btrfs/ZFS Pools (incl. RAID setups)
 Automatic discovery of disks/pools (via udev)
 Monitoring Disk health (SMART)
 Manage HW RAID controllers
 Add DRBD volume mirroring support to the WebUI
 Extend SAN functionality (more iSCSI/FC features)
 Public Roadmap on the openATTIC Jira/Wiki to solicit
community feedback

9. 9
openATTIC – Ceph Management Challenges
 Managing and monitoring Ceph is complex
 Many tools exist (e.g. Calamari, Intel VSM, ceph-dash)
 Limited functionality, unclear roadmaps
 Finding the best approach for managing Ceph

10. 10
openATTIC – Ceph Management Goals
 Create a management & monitoring GUI tool
 A tool that administrators actually want to use
 That scales without becoming overwhelming
 Still should allow changes to be made elsewhere, without
becoming inconsistent

11. 11
openATTIC – Ceph Management Implementation
 Which Ceph Management API?
 How to manage a distributed system?
 How to monitor the cluster‘s health/performance?
 How to perform remote management tasks?
 How to monitor cluster nodes in a scalable way?

12. 12
Django REST Framework
Django Models
 Client (GUI)
REST API

DB
(PostgreSQL)
nodb
Python librados / librbddjango.db.models
openATTIC – “NoDB” Django models

13. 13
openATTIC – Current Ceph Development Status
 “NoDB” and “TaskQueue” backend architecture in place
 Pool management (view/create/delete)
 OSD management (view)
 RBD management (view/create/delete/map)
 Cluster health/performance & pool monitoring
 CRUSH map editor
 Support for managing multiple Ceph clusters

14. 14
openATTIC – Ceph Management REST API
https://wiki.openattic.org/display/OP/openATTIC+Ceph+REST+API+overview

15. 15
openATTIC – Ceph Development Roadmap
 Ceph Cluster Status Dashboard incl. Performance Graphs
 Extend Pool Management
 OSD Monitoring/Management
 RBD Management/Monitoring
 CephFS Management
 RGW Management (users, buckets keys)
 Deployment, remote configuration of Ceph nodes (via
Salt)
 Remote node monitoring (via collectd)
 Public Roadmap on the openATTIC Wiki:

16. 16
openATTIC – Storage Management Dashboard

17. 17
openATTIC – Volume Management

18. 18
openATTIC – API Recorder

19. 19
openATTIC – Ceph Pool List

20. 20
openATTIC – Ceph OSD List

21. 21
openATTIC – Ceph RBD list

22. 22
openATTIC – Create Ceph RBD

23. 23
openATTIC – Ceph CRUSH map editing

24. 24
 www.openattic.org
 demo.openattic.org
 blog.openattic.org
 docs.openattic.org
 bitbucket.org/openattic
 tracker.openattic.org
 @openATTIC
 openATTIC
openATTIC – Resources

25. 25
Questions / Discussion

26. 26
Thank you!