State of Gluster Performance with Manoj Pillai

1. State of Gluster
Performance
Manoj Pillai - mpillai@redhat.com
Red Hat Perfomance and Scale Engineering

2. Agenda
● Motivation
● Gluster performance by workload class
● EC for video storage use-cases
● Work in Progress

3. Motivation
● Understand gluster performance characteristics, for
different workload classes and workload parameters
● Identify areas for improvement

4. Configuration
● Systems:
○ 12 drives per server
○ Replica-2 results with single 12-disk RAID-6 device
on each server
○ EC results with brick on each drive
○ Upto 6 servers, 4 clients in these tests
○ 1 10GbE link each
● Software versions
○ Glusterfs-3.8.2-1.el7.x86_64, RHEL-7
● Standard tuning
○ lookup-optimize, client-io-threads: on
○ event-threads=4

5. Sequential I/O: Replica
Volume
● With sufficient number of requests concurrently in flight,
close to network line speed on both writes and reads.

6. Sequential I/O: Disperse
Volume
● Similar to dist-rep, but larger gap in throughput between
low thread count and high thread count.

7. Seq I/O: Comparison of
Volume Types
● Replica-2 performs better for low thread count,
especially on writes.
● Disperse excels for writes with higher no. of threads.

8. Seq Write: Comparison with
kNFS
● kNFS better on single-thread tests, gluster does well with
higher number of threads.

9. Seq Read: Comparison with
kNFS
● kNFS dominates on single-thread test, hits line speed

10. Random Read: Comparison
with kNFS
● “1-cached” is single thread test where server-side cache
has not been dropped.
○ Good test of latency in the I/O path

11. Random Write: Comparison
with kNFS
● All of these tests bottleneck on disk
○ RAID-6 not a good choice for random writes.

12. Random Read: Comparison of
Volume Types

13. Random Write: Comparison
of Volume Types
● Replica-2 suffers possibly because of RAID-6 poor
performance on random writes.

14. Small File Read: Comparison
of Volume Types
● Disperse at a clear disadvantage in these tests

15. Small File Create:
Comparison of Volume Types
● Small file performance still under investigation

16. EC for Video Storage
Use-Cases
● Lot of interest seen in this area
● Significantly lower cost for EC, good performance

17. Work In Progress
● Rebalance Performance
○ Crawl, migration and throttle settings
● Small-file Performance
○ Performance degradation with increasing number of
files
● Tiering performance for small files
○ Small file performance with EC cold tier

18. Theme
Thanks!!