5. ARCHITECTURAL COMPONENTS
RGW
A web services
gateway for object
storage, compatible
with S3 and Swift
LIBRADOS
A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby, PHP)
RADOS
A software-based, reliable, autonomous, distributed object store comprised of
self-healing, self-managing, intelligent storage nodes and lightweight monitors
RBD
A reliable, fully-
distributed block
device with cloud
platform integration
CEPHFS
A distributed file
system with POSIX
semantics and scale-
out metadata
APP HOST/VM CLIENT
6. ARCHITECTURAL COMPONENTS
RGW
A web services
gateway for object
storage, compatible
with S3 and Swift
LIBRADOS
A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby, PHP)
RADOS
A software-based, reliable, autonomous, distributed object store comprised of
self-healing, self-managing, intelligent storage nodes and lightweight monitors
RBD
A reliable, fully-
distributed block
device with cloud
platform integration
CEPHFS
A distributed file
system with POSIX
semantics and scale-
out metadata
APP HOST/VM CLIENT
12. RBD FORMAT
• Format 1
• Deprecated
• Supported by all versions of Ceph
• No reason to use it in greenfield environment
• Format 2
• New, default, format
• Support snapshot and clone
13. RBD ODER
• The chunk / striping boundary for block device
• Default is 4MB -> 22
• 4MB = 222
• Used default during our testing
14. RBD: Fancy Striping
• Only available to QEMU / librbd
• Finer striping for parallelization of small writes across order
• Helps with some HDD workloads
• Used default during our testing
15. TCP_NO_DELAY
• Disables Nagel congestion control algorithm
• Important for latency sensitive workloads
• Good for maximizing IOPS -> MySQL
• Default in QEMU
• Default in KRBD
• Added in mainline 4.2
• Backported to RHEL 7.2 3.10-236+
21. QEMU: Timers
• Block storage benchmark too – fio
• Very frequent access to CPU timing registers
• Accesses need to be emulated
• Can block main QEMU event loop with concurrent high IO load
22. QEMU: Timers
• Block storage benchmark too – fio
• Very frequent access to CPU timing registers
• Accesses need to be emulated
• Can block main QEMU event loop with concurrent high IO load