SlideShare ist ein Scribd-Unternehmen logo
1 von 50
Downloaden Sie, um offline zu lesen
November 8–13, 2015 | Washington, D.C.
www.usenix.org/lisa15 #lisa15
GlusterFSGlusterFS
A Scale-out Software Defined Storage
Rajesh Joseph
Poornima Gurusiddaiah
05/17/16
AGENDA
Introduction
GlusterFS Concepts
Architecture
Additional Features
Installation
Setup and Configuration
Future Work
Q&A
05/17/16
GlusterFS
Open-source general purpose scale-out
distributed file system
Aggregates storage exports over
network interconnect to provide a single
unified namespace
Layered on disk file systems that support
extended attributes
No meta-data server
Modular Architecture for Scale
and Functionality
Heterogeneous commodity
hardware
Scalable to petabytes & beyond
05/17/16
GlusterFS Concepts
05/17/16
Server
Intel/AMD x86 64-bit processor
Disk: 8GB minimum using direct-
attached-storage, RAID, Amazon EBS,
and FC/Infiniband/iSCSI SAN disk
backends using SATA/SAS/FC disks
Memory: 1GB minimum
Logical Volume Manager
LVM2 with thin provisioning
Networking
Gigabit Ethernet
10 Gigabit Ethernet
InfiniBand (OFED 1.5.2 or later)
Filesystem
POSIX w/ Extended Attributes (EXT4,
XFS, BTRFS, …)
A node is server capable of hosting GlusterFS bricks
Node
05/17/16
Trusted Storage Pool
Also known as Cluster
Trusted Storage Pool is formed by invitation – “probe”
Members can be dynamically added and removed from the pool
Only nodes in a Trusted Storage Pool can participate in volume creation
A collection of storage servers (Node)
Node 2Node 1 Node 3
05/17/16
Bricks
A brick is the combination of a node and an export directory – e.g.
hostname:/dir
Layered on posix compliant file-system (e.g. XFS, ext4)
Each brick inherits limits of the underlying filesystem
It is recommended to use an independent thinly provisioned LVM as brick
Thin provisioning is needed by Snapshot feature
A unit of storage used as a capacity building block
05/17/16
Bricks
Storage devices
Thin LV
Volume GroupBrick1 Brick2
Thin Pool
mount
Thin LV
Thin Pool
mount
05/17/16
GlusterFS Volume
Node hosting these bricks should be part of a single Trusted
Storage Pool
One or more volumes can be hosted on the same node
A volume is a logical collection of one or more bricks
05/17/16
GlusterFS Volume
GlusterFS Volume
Storage Node Storage Node Storage Node
Brick Brick Brick Brick
05/17/16
GlusterFS Volume
Volume1
Storage Node Storage Node Storage Node
Brick Brick Brick BrickBrick Brick
Volume2
05/17/16
Translators
Translators can be stacked together for achieving desired
functionality
Translators are deployment agnostic – can be loaded in either the
client or server stacks
Building block of GlusterFS process
05/17/16
Translators
GigE, 10GigE – TCP/IP / Inifiniband - RDMA
Client Client Client Client
Network
VFS
Distribute
Replicate Replicate
Server
POSIX
Server
POSIX
Server
POSIX
Server
POSIX
I/O Cache
Read Ahead
Bricks
Client
05/17/16
GlusterFS Processes
glusterd
Management daemon
One instance on each GlusterFS node
glusterfsd
GlusterFS brick daemon
One process per brick on each node
Managed by glusterd
gluster
Gluster console manager (CLI)
glusterfs
Node services
One process for each service
NFS Server, Self heal, Quota, ...
mount.glusterfs
FUSE native client mount extension
05/17/16
Accessing Volume
Volume can be mounted on local file-system
Following protocols supported for accessing volume
GlusterFS Native client
Filesystem in Userspace (FUSE)
libgfapi flexible abstracted storage
Samba, NFS-Ganesha, QEMU
NFS
NFS Ganesha
Gluster NFSv3
SMB / CIFS
Windows and Linux
Gluster For OpenStack
Object-based access via OpenStack Swift
05/17/16
Fuse Native
Network Interconnect
NFS
Samba
libgfapi
NFSGanesha
AppQEMU
05/17/16
GlusterFS Native Client (FUSE)
FUSE kernel module allows the filesystem to be built and operated entirely in userspace
Specify mount to any GlusterFS server
Native Client fetches volfile from mount server, then communicates directly with all nodes to
access data
05/17/16
Accessing Volume – Fuse client
Gluster Volume
Storage Node Storage Node Storage Node
Brick Brick Brick Brick
mount Client
05/17/16
libgfapi
Userspace library for accessing GlusterFS volume
File-system like API
No FUSE, No copies, No context switches
Samba, QEMU, NFS-Ganesha integrated with libgfapi
05/17/16
Libgfapi - Advantage
Storage NodeClient
Userspace
Kernel
Userspace
Kernel
App
/mnt/gmount
/dev/fuseKernel VFS
Net Net
Glusterfsd
Brick
XFS
GlusterFS
Fuse Library
05/17/16
Libgfapi - Advantage
Storage NodeClient
Userspace
Kernel
Userspace
Kernel
Net Net
Glusterfsd
Brick
XFS
App
libgfapi
05/17/16
Gluster NFS
Supports NFS v3 protocol
Mount to any server or use load-balancer
GlusterFS NFS server uses Network Lock Manager (NLM) to synchronize locks across clients
Load balancing should be managed externally
05/17/16
Accessing Volume – Gluster NFS
Gluster Volume
Storage Node Storage Node Storage Node
Brick Brick Brick Brick
mount
Client
NFSNFSNFS
IP Load Balancer
05/17/16
SMB / CIFS
Samba VFS plugin for GlusterFS
Uses libgfapi
Must be setup on each server you wish to connect
CTDB is required for Samba clustering
05/17/16
Accessing Volume – SMB
Gluster Volume
Storage Node Storage Node Storage Node
Brick Brick Brick Brick
mount
Client
SMBDSMBDSMBD
IP Load Balancer
05/17/16
Volume Types
Type of a volume is specified at the time of volume creation
Volume type determines how and where data is placed
Basic Volume Types
Distributed
Replicated
Dispersed
Striped
Sharded
Advance Volume Types
Distributed Replicated
Distributed Dispersed
Distributed Striped
Distributed Striped Replicated
05/17/16
Distributed Volume
Distributes files across various bricks of the volume
Similar to file-level RAID-0
Scaling but no high availability
Uses Davies-Meyer hash algorithm
A 32-bit hash space is divided into N ranges for N bricks
Removes the need for an external meta data server
05/17/16
Distribute Volume
Storage Node Storage Node Storage Node
Brick Brick Brick
Client
File1File2
[0, a] [a + 1, b] [b + 1, c]
File1 Hash = x, Where 0 <= x <= aFile2 Hash = y, Where b < y <= c
05/17/16
Replicate Volume
Synchronous replication of all directory and file updates
Similar to file-level RAID-1
High availability but no scaling
Replica pairs are decided at the volume creation time
Its recommended to host each brick of replica set on different node
05/17/16
Replicate Volume
Storage Node
Brick
Client
File1
Storage Node
Brick
05/17/16
Disperse Volume
Erasure Coded volume
High availability with less number of bricks
Store m disk of data on k disk (k > m)
n (k-m) redundant disks
Following setup is the most tested configuration
6 bricks with redundancy level 2 (4 +2)
11 bricks with redundancy level 3 (8 +3)
12 bricks with redundancy level 4 (8 + 4)
05/17/16
Disperse Volume
Storage Node
Brick
Client
File1
Storage Node
Brick
Storage Node
Brick
Data Part
Parity / Redundancy Part
05/17/16
Striped Volume
Individual files split among bricks (sparse files)
Similar to block-level RAID-0
Recommended only when very large files greater than the size of the disks are present
Chunks are files with holes – this helps in maintaining offset consistency
05/17/16
Striped Volume
Storage Node
Brick
Client
File1
Storage Node
Brick
Storage Node
Brick
05/17/16
Sharded (Stripe – 2.0) Volume
Individual files split among bricks
Each part split is a different file in the backend
File name of each part derived from GFID (UUID) of actual file
Unlike Stripe file rename will not have any impact on these parts.
Unlike other volume types shard is a volume option which can be set on any volume
Recommended only when very large files greater than the size of the disks are present
05/17/16
Sharded Volume
Storage Node
Brick
Client
File1
Storage Node
Brick
Storage Node
Brick
GFID1.1 GFID1.2
GFID1.3
05/17/16
Distribute Replicate Volume
Distribute files across replicated bricks
Number of bricks must be a multiple of the replica count
Ordering of bricks in volume definition matters
Scaling and high availability
Most preferred model of deployment
05/17/16
Distribute Replicate Volume
Storage Node
Brick
Storage Node
Brick
Storage Node
Brick
Client
File1File2
Storage Node
Brick
Replica Pairs
Replica Pairs
05/17/16
Performance
Different SLA than a local file-system
Uneven performance distribution between nodes and clients
Various performance options available (e.g., read-ahead, md-cache, etc)
Based on workload use various options
Don't forget to attend detailed session on performance by Jeff Darcy
Topic  : Evaluating Distributed File System Performance (GlusterFS and Ceph)
When   : Friday, November 13, 2015 ­ 9:00am­10:30am
Detail : https://www.usenix.org/conference/lisa15/conference­program/presentation/darcy
05/17/16
Additional Features
05/17/16
Additional Features
Geo Replication
Volume Snapshot
Quota
Bit-rot
Compression
Encryption at rest
Trash / Recycle-bin
pNFS with NFS-Ganesha
Data Tiering
User Serviceable Snapshots
Server side Quorum
Client side Quorum
And many more ...
05/17/16
Geo-Replication
Mirrors data across geographically distributed trusted storage pools
Provides back-ups of data for disaster recovery
Master-Slave model
Asynchronous replication
Provides an incremental continuous replication service over
Local Area Networks (LANs)
Wide Area Network (WANs)
Internet
05/17/16
Geo-Replication
Trusted Storage Pool Trusted Storage Pool
Network
LAN/WAN/Internet
Master Volume Slave Volume
05/17/16
GlusterFS Snapshot
Volume level crash consistent snapshots
LVM2 based
Operates only on thin-provisioned volumes
Snapshots themselves are thin-provisioned snapshot volumes
A GlusterFS volume snapshot consists of snapshots of the individual bricks making up the
volume
Snapshot of a GlusterFS volume is also a GlusterFS volume
05/17/16
Thinly Provisioned LVM2 Volume & Snapshot
Storage devices
SnapshotThin LV
Thin Pool
Volume Group
05/17/16
Gluster Volume Snapshot
Gluster Volume
Storage Node Storage Node
Brick1 Brick2
Brick3 Brick4
Brick1_s1 Brick2_s1
Brick4_s1Brick3_s1
Snapshot volume
Barrier
05/17/16
Data Tiering
Data classification and placement of data
Currently based on access rate
Tiered volume can be created by attaching a tier
Bricks in original volume are “cold” bricks
Bricks in attached tier are “hot” bricks
Hot bricks are normally hosted on faster disks e.g. SSDs
Tier can be attached or detached using volume commands
05/17/16
Tiered Volume
Storage Node
Brick
Storage Node
Brick
Storage Node
Brick
Client
Storage Node
BrickBrick
Cold Tier Hot Tier
TT
05/17/16
Tiered Volume
Replica
Tier
Distribute
Replica Replica
Brick Brick Brick Brick Brick Brick Brick
Hot Tier
Cold Tier
05/17/16
Future Work
New Style Replication (NSR)
Scalability improvements (1000+ Nodes)
Multi-protocol support
Multi-master Geo-Replication
Caching Improvements
Btrfs Support
And many more…

Weitere ähnliche Inhalte

Was ist angesagt?

Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...
Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...
Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...xKinAnx
 
ceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-shortceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-shortNAVER D2
 
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseApplication Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseVictoriaMetrics
 
Prometheus Monitoring Mixins
Prometheus Monitoring MixinsPrometheus Monitoring Mixins
Prometheus Monitoring MixinsGrafana Labs
 
The Linux Block Layer - Built for Fast Storage
The Linux Block Layer - Built for Fast StorageThe Linux Block Layer - Built for Fast Storage
The Linux Block Layer - Built for Fast StorageKernel TLV
 
The Google Chubby lock service for loosely-coupled distributed systems
The Google Chubby lock service for loosely-coupled distributed systemsThe Google Chubby lock service for loosely-coupled distributed systems
The Google Chubby lock service for loosely-coupled distributed systemsRomain Jacotin
 
Seastore: Next Generation Backing Store for Ceph
Seastore: Next Generation Backing Store for CephSeastore: Next Generation Backing Store for Ceph
Seastore: Next Generation Backing Store for CephScyllaDB
 
Linux Memory Management with CMA (Contiguous Memory Allocator)
Linux Memory Management with CMA (Contiguous Memory Allocator)Linux Memory Management with CMA (Contiguous Memory Allocator)
Linux Memory Management with CMA (Contiguous Memory Allocator)Pankaj Suryawanshi
 
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019Storage 101: Rook and Ceph - Open Infrastructure Denver 2019
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019Sean Cohen
 
Linux memory-management-kamal
Linux memory-management-kamalLinux memory-management-kamal
Linux memory-management-kamalKamal Maiti
 
Slab Allocator in Linux Kernel
Slab Allocator in Linux KernelSlab Allocator in Linux Kernel
Slab Allocator in Linux KernelAdrian Huang
 
The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization OpportunitiesThe Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization OpportunitiesDatabricks
 
IBM Spectrum Scale Overview november 2015
IBM Spectrum Scale Overview november 2015IBM Spectrum Scale Overview november 2015
IBM Spectrum Scale Overview november 2015Doug O'Flaherty
 
Using Wildcards with rsyslog's File Monitor imfile
Using Wildcards with rsyslog's File Monitor imfileUsing Wildcards with rsyslog's File Monitor imfile
Using Wildcards with rsyslog's File Monitor imfileRainer Gerhards
 
Cosco: An Efficient Facebook-Scale Shuffle Service
Cosco: An Efficient Facebook-Scale Shuffle ServiceCosco: An Efficient Facebook-Scale Shuffle Service
Cosco: An Efficient Facebook-Scale Shuffle ServiceDatabricks
 
ClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovAltinity Ltd
 
Cloudera Impala Internals
Cloudera Impala InternalsCloudera Impala Internals
Cloudera Impala InternalsDavid Groozman
 
The Anatomy Of The Google Architecture Fina Lv1.1
The Anatomy Of The Google Architecture Fina Lv1.1The Anatomy Of The Google Architecture Fina Lv1.1
The Anatomy Of The Google Architecture Fina Lv1.1Hassy Veldstra
 
Introducing the Apache Flink Kubernetes Operator
Introducing the Apache Flink Kubernetes OperatorIntroducing the Apache Flink Kubernetes Operator
Introducing the Apache Flink Kubernetes OperatorFlink Forward
 

Was ist angesagt? (20)

Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...
Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...
Ibm spectrum scale fundamentals workshop for americas part 4 spectrum scale_r...
 
ceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-shortceph optimization on ssd ilsoo byun-short
ceph optimization on ssd ilsoo byun-short
 
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouseApplication Monitoring using Open Source: VictoriaMetrics - ClickHouse
Application Monitoring using Open Source: VictoriaMetrics - ClickHouse
 
Prometheus Monitoring Mixins
Prometheus Monitoring MixinsPrometheus Monitoring Mixins
Prometheus Monitoring Mixins
 
The Linux Block Layer - Built for Fast Storage
The Linux Block Layer - Built for Fast StorageThe Linux Block Layer - Built for Fast Storage
The Linux Block Layer - Built for Fast Storage
 
The Google Chubby lock service for loosely-coupled distributed systems
The Google Chubby lock service for loosely-coupled distributed systemsThe Google Chubby lock service for loosely-coupled distributed systems
The Google Chubby lock service for loosely-coupled distributed systems
 
Seastore: Next Generation Backing Store for Ceph
Seastore: Next Generation Backing Store for CephSeastore: Next Generation Backing Store for Ceph
Seastore: Next Generation Backing Store for Ceph
 
Linux Memory Management with CMA (Contiguous Memory Allocator)
Linux Memory Management with CMA (Contiguous Memory Allocator)Linux Memory Management with CMA (Contiguous Memory Allocator)
Linux Memory Management with CMA (Contiguous Memory Allocator)
 
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019Storage 101: Rook and Ceph - Open Infrastructure Denver 2019
Storage 101: Rook and Ceph - Open Infrastructure Denver 2019
 
Linux memory-management-kamal
Linux memory-management-kamalLinux memory-management-kamal
Linux memory-management-kamal
 
Slab Allocator in Linux Kernel
Slab Allocator in Linux KernelSlab Allocator in Linux Kernel
Slab Allocator in Linux Kernel
 
The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization OpportunitiesThe Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
 
IBM Spectrum Scale Overview november 2015
IBM Spectrum Scale Overview november 2015IBM Spectrum Scale Overview november 2015
IBM Spectrum Scale Overview november 2015
 
Using Wildcards with rsyslog's File Monitor imfile
Using Wildcards with rsyslog's File Monitor imfileUsing Wildcards with rsyslog's File Monitor imfile
Using Wildcards with rsyslog's File Monitor imfile
 
Nfs
NfsNfs
Nfs
 
Cosco: An Efficient Facebook-Scale Shuffle Service
Cosco: An Efficient Facebook-Scale Shuffle ServiceCosco: An Efficient Facebook-Scale Shuffle Service
Cosco: An Efficient Facebook-Scale Shuffle Service
 
ClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei MilovidovClickHouse Deep Dive, by Aleksei Milovidov
ClickHouse Deep Dive, by Aleksei Milovidov
 
Cloudera Impala Internals
Cloudera Impala InternalsCloudera Impala Internals
Cloudera Impala Internals
 
The Anatomy Of The Google Architecture Fina Lv1.1
The Anatomy Of The Google Architecture Fina Lv1.1The Anatomy Of The Google Architecture Fina Lv1.1
The Anatomy Of The Google Architecture Fina Lv1.1
 
Introducing the Apache Flink Kubernetes Operator
Introducing the Apache Flink Kubernetes OperatorIntroducing the Apache Flink Kubernetes Operator
Introducing the Apache Flink Kubernetes Operator
 

Andere mochten auch

Introduction to highly_availablenfs_server_on_scale-out_storage_systems_based...
Introduction to highly_availablenfs_server_on_scale-out_storage_systems_based...Introduction to highly_availablenfs_server_on_scale-out_storage_systems_based...
Introduction to highly_availablenfs_server_on_scale-out_storage_systems_based...Gluster.org
 
Scale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_glusterScale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_glusterGluster.org
 
Gluster fs current_features_and_roadmap
Gluster fs current_features_and_roadmapGluster fs current_features_and_roadmap
Gluster fs current_features_and_roadmapGluster.org
 
Smb gluster devmar2013
Smb gluster devmar2013Smb gluster devmar2013
Smb gluster devmar2013Gluster.org
 
Dustin Black - Red Hat Storage Server Administration Deep Dive
Dustin Black - Red Hat Storage Server Administration Deep DiveDustin Black - Red Hat Storage Server Administration Deep Dive
Dustin Black - Red Hat Storage Server Administration Deep DiveGluster.org
 
Hands On Gluster with Jeff Darcy
Hands On Gluster with Jeff DarcyHands On Gluster with Jeff Darcy
Hands On Gluster with Jeff DarcyGluster.org
 
Persistent Storage in Openshift using GlusterFS
Persistent Storage in Openshift using GlusterFSPersistent Storage in Openshift using GlusterFS
Persistent Storage in Openshift using GlusterFSHumble Chirammal
 
Finding the Right Balance: Security vs. Performance with Network Storage Systems
Finding the Right Balance: Security vs. Performance with Network Storage SystemsFinding the Right Balance: Security vs. Performance with Network Storage Systems
Finding the Right Balance: Security vs. Performance with Network Storage SystemsArun Olappamanna Vasudevan
 
Software Defined storage
Software Defined storageSoftware Defined storage
Software Defined storageKirillos Akram
 
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013Gluster.org
 
Scale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_glusterScale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_glusterGluster.org
 
Award winning scale-up and scale-out storage for Xen
Award winning scale-up and scale-out storage for XenAward winning scale-up and scale-out storage for Xen
Award winning scale-up and scale-out storage for XenGlusterFS
 
How to Install Gluster Storage Platform
How to Install Gluster Storage PlatformHow to Install Gluster Storage Platform
How to Install Gluster Storage PlatformGlusterFS
 
Gluster Storage Platform Installation Guide
Gluster Storage Platform Installation GuideGluster Storage Platform Installation Guide
Gluster Storage Platform Installation GuideGlusterFS
 
Gluster fs architecture_future_directions_tlv
Gluster fs architecture_future_directions_tlvGluster fs architecture_future_directions_tlv
Gluster fs architecture_future_directions_tlvSahina Bose
 
Introduction to GlusterFS Webinar - September 2011
Introduction to GlusterFS Webinar - September 2011Introduction to GlusterFS Webinar - September 2011
Introduction to GlusterFS Webinar - September 2011GlusterFS
 
Petascale Cloud Storage with GlusterFS
Petascale Cloud Storage with GlusterFSPetascale Cloud Storage with GlusterFS
Petascale Cloud Storage with GlusterFSThe Linux Foundation
 
Distributed Shared Memory Systems
Distributed Shared Memory SystemsDistributed Shared Memory Systems
Distributed Shared Memory SystemsAnkit Gupta
 
Red Hat Gluster Storage : GlusterFS
Red Hat Gluster Storage : GlusterFSRed Hat Gluster Storage : GlusterFS
Red Hat Gluster Storage : GlusterFSbipin kunal
 

Andere mochten auch (20)

Introduction to highly_availablenfs_server_on_scale-out_storage_systems_based...
Introduction to highly_availablenfs_server_on_scale-out_storage_systems_based...Introduction to highly_availablenfs_server_on_scale-out_storage_systems_based...
Introduction to highly_availablenfs_server_on_scale-out_storage_systems_based...
 
Scale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_glusterScale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_gluster
 
Gdeploy 2.0
Gdeploy 2.0Gdeploy 2.0
Gdeploy 2.0
 
Gluster fs current_features_and_roadmap
Gluster fs current_features_and_roadmapGluster fs current_features_and_roadmap
Gluster fs current_features_and_roadmap
 
Smb gluster devmar2013
Smb gluster devmar2013Smb gluster devmar2013
Smb gluster devmar2013
 
Dustin Black - Red Hat Storage Server Administration Deep Dive
Dustin Black - Red Hat Storage Server Administration Deep DiveDustin Black - Red Hat Storage Server Administration Deep Dive
Dustin Black - Red Hat Storage Server Administration Deep Dive
 
Hands On Gluster with Jeff Darcy
Hands On Gluster with Jeff DarcyHands On Gluster with Jeff Darcy
Hands On Gluster with Jeff Darcy
 
Persistent Storage in Openshift using GlusterFS
Persistent Storage in Openshift using GlusterFSPersistent Storage in Openshift using GlusterFS
Persistent Storage in Openshift using GlusterFS
 
Finding the Right Balance: Security vs. Performance with Network Storage Systems
Finding the Right Balance: Security vs. Performance with Network Storage SystemsFinding the Right Balance: Security vs. Performance with Network Storage Systems
Finding the Right Balance: Security vs. Performance with Network Storage Systems
 
Software Defined storage
Software Defined storageSoftware Defined storage
Software Defined storage
 
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013
Integrating gluster fs,_qemu_and_ovirt-vijay_bellur-linuxcon_eu_2013
 
Scale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_glusterScale out backups-with_bareos_and_gluster
Scale out backups-with_bareos_and_gluster
 
Award winning scale-up and scale-out storage for Xen
Award winning scale-up and scale-out storage for XenAward winning scale-up and scale-out storage for Xen
Award winning scale-up and scale-out storage for Xen
 
How to Install Gluster Storage Platform
How to Install Gluster Storage PlatformHow to Install Gluster Storage Platform
How to Install Gluster Storage Platform
 
Gluster Storage Platform Installation Guide
Gluster Storage Platform Installation GuideGluster Storage Platform Installation Guide
Gluster Storage Platform Installation Guide
 
Gluster fs architecture_future_directions_tlv
Gluster fs architecture_future_directions_tlvGluster fs architecture_future_directions_tlv
Gluster fs architecture_future_directions_tlv
 
Introduction to GlusterFS Webinar - September 2011
Introduction to GlusterFS Webinar - September 2011Introduction to GlusterFS Webinar - September 2011
Introduction to GlusterFS Webinar - September 2011
 
Petascale Cloud Storage with GlusterFS
Petascale Cloud Storage with GlusterFSPetascale Cloud Storage with GlusterFS
Petascale Cloud Storage with GlusterFS
 
Distributed Shared Memory Systems
Distributed Shared Memory SystemsDistributed Shared Memory Systems
Distributed Shared Memory Systems
 
Red Hat Gluster Storage : GlusterFS
Red Hat Gluster Storage : GlusterFSRed Hat Gluster Storage : GlusterFS
Red Hat Gluster Storage : GlusterFS
 

Ähnlich wie Lisa 2015-gluster fs-introduction

GlusterFs: a scalable file system for today's and tomorrow's big data
GlusterFs: a scalable file system for today's and tomorrow's big dataGlusterFs: a scalable file system for today's and tomorrow's big data
GlusterFs: a scalable file system for today's and tomorrow's big dataRoberto Franchini
 
GlusterFS : un file system open source per i big data di oggi e domani - Robe...
GlusterFS : un file system open source per i big data di oggi e domani - Robe...GlusterFS : un file system open source per i big data di oggi e domani - Robe...
GlusterFS : un file system open source per i big data di oggi e domani - Robe...Codemotion
 
GlusterFS Architecture - June 30, 2011 Meetup
GlusterFS Architecture - June 30, 2011 MeetupGlusterFS Architecture - June 30, 2011 Meetup
GlusterFS Architecture - June 30, 2011 MeetupGlusterFS
 
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013Gluster.org
 
Gpfs introandsetup
Gpfs introandsetupGpfs introandsetup
Gpfs introandsetupasihan
 
Gluster fs hadoop_fifth-elephant
Gluster fs hadoop_fifth-elephantGluster fs hadoop_fifth-elephant
Gluster fs hadoop_fifth-elephantGluster.org
 
GlusterFS Talk for CentOS Dojo Bangalore
GlusterFS Talk for CentOS Dojo BangaloreGlusterFS Talk for CentOS Dojo Bangalore
GlusterFS Talk for CentOS Dojo BangaloreRaghavendra Talur
 
IBM Platform Computing Elastic Storage
IBM Platform Computing  Elastic StorageIBM Platform Computing  Elastic Storage
IBM Platform Computing Elastic StoragePatrick Bouillaud
 
Gluster intro-tdose
Gluster intro-tdoseGluster intro-tdose
Gluster intro-tdoseGluster.org
 
Gluster intro-tdose
Gluster intro-tdoseGluster intro-tdose
Gluster intro-tdoseGluster.org
 
Wheeler w 0450_linux_file_systems1
Wheeler w 0450_linux_file_systems1Wheeler w 0450_linux_file_systems1
Wheeler w 0450_linux_file_systems1sprdd
 
Wheeler w 0450_linux_file_systems1
Wheeler w 0450_linux_file_systems1Wheeler w 0450_linux_file_systems1
Wheeler w 0450_linux_file_systems1sprdd
 
Spectrum Scale Unified File and Object with WAN Caching
Spectrum Scale Unified File and Object with WAN CachingSpectrum Scale Unified File and Object with WAN Caching
Spectrum Scale Unified File and Object with WAN CachingSandeep Patil
 
Software Defined Analytics with File and Object Access Plus Geographically Di...
Software Defined Analytics with File and Object Access Plus Geographically Di...Software Defined Analytics with File and Object Access Plus Geographically Di...
Software Defined Analytics with File and Object Access Plus Geographically Di...Trishali Nayar
 
Performance characterization in large distributed file system with gluster fs
Performance characterization in large distributed file system with gluster fsPerformance characterization in large distributed file system with gluster fs
Performance characterization in large distributed file system with gluster fsNeependra Khare
 
2018 Infortrend EonStor GSe Pro Family Introduction
2018 Infortrend EonStor GSe Pro Family Introduction2018 Infortrend EonStor GSe Pro Family Introduction
2018 Infortrend EonStor GSe Pro Family Introductioninfortrendgroup
 
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GR
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GRGlusterFS Presentation FOSSCOMM2013 HUA, Athens, GR
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GRTheophanis Kontogiannis
 

Ähnlich wie Lisa 2015-gluster fs-introduction (20)

GlusterFs: a scalable file system for today's and tomorrow's big data
GlusterFs: a scalable file system for today's and tomorrow's big dataGlusterFs: a scalable file system for today's and tomorrow's big data
GlusterFs: a scalable file system for today's and tomorrow's big data
 
GlusterFS : un file system open source per i big data di oggi e domani - Robe...
GlusterFS : un file system open source per i big data di oggi e domani - Robe...GlusterFS : un file system open source per i big data di oggi e domani - Robe...
GlusterFS : un file system open source per i big data di oggi e domani - Robe...
 
GlusterFS Architecture - June 30, 2011 Meetup
GlusterFS Architecture - June 30, 2011 MeetupGlusterFS Architecture - June 30, 2011 Meetup
GlusterFS Architecture - June 30, 2011 Meetup
 
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013
Gluster fs architecture_&amp;_roadmap-vijay_bellur-linuxcon_eu_2013
 
Gpfs introandsetup
Gpfs introandsetupGpfs introandsetup
Gpfs introandsetup
 
Gluster fs hadoop_fifth-elephant
Gluster fs hadoop_fifth-elephantGluster fs hadoop_fifth-elephant
Gluster fs hadoop_fifth-elephant
 
GlusterFS Talk for CentOS Dojo Bangalore
GlusterFS Talk for CentOS Dojo BangaloreGlusterFS Talk for CentOS Dojo Bangalore
GlusterFS Talk for CentOS Dojo Bangalore
 
IBM Platform Computing Elastic Storage
IBM Platform Computing  Elastic StorageIBM Platform Computing  Elastic Storage
IBM Platform Computing Elastic Storage
 
Gluster intro-tdose
Gluster intro-tdoseGluster intro-tdose
Gluster intro-tdose
 
Gluster intro-tdose
Gluster intro-tdoseGluster intro-tdose
Gluster intro-tdose
 
GlusterFS And Big Data
GlusterFS And Big DataGlusterFS And Big Data
GlusterFS And Big Data
 
Hadoop
HadoopHadoop
Hadoop
 
Wheeler w 0450_linux_file_systems1
Wheeler w 0450_linux_file_systems1Wheeler w 0450_linux_file_systems1
Wheeler w 0450_linux_file_systems1
 
Wheeler w 0450_linux_file_systems1
Wheeler w 0450_linux_file_systems1Wheeler w 0450_linux_file_systems1
Wheeler w 0450_linux_file_systems1
 
Spectrum Scale Unified File and Object with WAN Caching
Spectrum Scale Unified File and Object with WAN CachingSpectrum Scale Unified File and Object with WAN Caching
Spectrum Scale Unified File and Object with WAN Caching
 
Software Defined Analytics with File and Object Access Plus Geographically Di...
Software Defined Analytics with File and Object Access Plus Geographically Di...Software Defined Analytics with File and Object Access Plus Geographically Di...
Software Defined Analytics with File and Object Access Plus Geographically Di...
 
Performance characterization in large distributed file system with gluster fs
Performance characterization in large distributed file system with gluster fsPerformance characterization in large distributed file system with gluster fs
Performance characterization in large distributed file system with gluster fs
 
2018 Infortrend EonStor GSe Pro Family Introduction
2018 Infortrend EonStor GSe Pro Family Introduction2018 Infortrend EonStor GSe Pro Family Introduction
2018 Infortrend EonStor GSe Pro Family Introduction
 
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GR
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GRGlusterFS Presentation FOSSCOMM2013 HUA, Athens, GR
GlusterFS Presentation FOSSCOMM2013 HUA, Athens, GR
 
Ch18 system administration
Ch18 system administration Ch18 system administration
Ch18 system administration
 

Mehr von Gluster.org

Automating Gluster @ Facebook - Shreyas Siravara
Automating Gluster @ Facebook - Shreyas SiravaraAutomating Gluster @ Facebook - Shreyas Siravara
Automating Gluster @ Facebook - Shreyas SiravaraGluster.org
 
nfusr: a new userspace NFS client based on libnfs - Shreyas Siravara
nfusr: a new userspace NFS client based on libnfs - Shreyas Siravaranfusr: a new userspace NFS client based on libnfs - Shreyas Siravara
nfusr: a new userspace NFS client based on libnfs - Shreyas SiravaraGluster.org
 
Facebook’s upstream approach to GlusterFS - David Hasson
Facebook’s upstream approach to GlusterFS  - David HassonFacebook’s upstream approach to GlusterFS  - David Hasson
Facebook’s upstream approach to GlusterFS - David HassonGluster.org
 
Throttling Traffic at Facebook Scale
Throttling Traffic at Facebook ScaleThrottling Traffic at Facebook Scale
Throttling Traffic at Facebook ScaleGluster.org
 
GlusterFS w/ Tiered XFS
GlusterFS w/ Tiered XFS  GlusterFS w/ Tiered XFS
GlusterFS w/ Tiered XFS Gluster.org
 
Gluster Metrics: why they are crucial for running stable deployments of all s...
Gluster Metrics: why they are crucial for running stable deployments of all s...Gluster Metrics: why they are crucial for running stable deployments of all s...
Gluster Metrics: why they are crucial for running stable deployments of all s...Gluster.org
 
Up and Running with Glusto & Glusto-Tests in 5 Minutes (or less)
Up and Running with Glusto & Glusto-Tests in 5 Minutes (or less)Up and Running with Glusto & Glusto-Tests in 5 Minutes (or less)
Up and Running with Glusto & Glusto-Tests in 5 Minutes (or less)Gluster.org
 
Data Reduction for Gluster with VDO
Data Reduction for Gluster with VDOData Reduction for Gluster with VDO
Data Reduction for Gluster with VDOGluster.org
 
Releases: What are contributors responsible for
Releases: What are contributors responsible forReleases: What are contributors responsible for
Releases: What are contributors responsible forGluster.org
 
RIO Distribution: Reconstructing the onion - Shyamsundar Ranganathan
RIO Distribution: Reconstructing the onion - Shyamsundar RanganathanRIO Distribution: Reconstructing the onion - Shyamsundar Ranganathan
RIO Distribution: Reconstructing the onion - Shyamsundar RanganathanGluster.org
 
Gluster and Kubernetes
Gluster and KubernetesGluster and Kubernetes
Gluster and KubernetesGluster.org
 
Native Clients, more the merrier with GFProxy!
Native Clients, more the merrier with GFProxy!Native Clients, more the merrier with GFProxy!
Native Clients, more the merrier with GFProxy!Gluster.org
 
Gluster: a SWOT Analysis
Gluster: a SWOT Analysis Gluster: a SWOT Analysis
Gluster: a SWOT Analysis Gluster.org
 
GlusterD-2.0: What's Happening? - Kaushal Madappa
GlusterD-2.0: What's Happening? - Kaushal MadappaGlusterD-2.0: What's Happening? - Kaushal Madappa
GlusterD-2.0: What's Happening? - Kaushal MadappaGluster.org
 
Scalability and Performance of CNS 3.6
Scalability and Performance of CNS 3.6Scalability and Performance of CNS 3.6
Scalability and Performance of CNS 3.6Gluster.org
 
What Makes Us Fail
What Makes Us FailWhat Makes Us Fail
What Makes Us FailGluster.org
 
Gluster as Native Storage for Containers - past, present and future
Gluster as Native Storage for Containers - past, present and futureGluster as Native Storage for Containers - past, present and future
Gluster as Native Storage for Containers - past, present and futureGluster.org
 
Heketi Functionality into Glusterd2
Heketi Functionality into Glusterd2Heketi Functionality into Glusterd2
Heketi Functionality into Glusterd2Gluster.org
 
Architecture of the High Availability Solution for Ganesha and Samba with Kal...
Architecture of the High Availability Solution for Ganesha and Samba with Kal...Architecture of the High Availability Solution for Ganesha and Samba with Kal...
Architecture of the High Availability Solution for Ganesha and Samba with Kal...Gluster.org
 
Challenges with Gluster and Persistent Memory with Dan Lambright
Challenges with Gluster and Persistent Memory with Dan LambrightChallenges with Gluster and Persistent Memory with Dan Lambright
Challenges with Gluster and Persistent Memory with Dan LambrightGluster.org
 

Mehr von Gluster.org (20)

Automating Gluster @ Facebook - Shreyas Siravara
Automating Gluster @ Facebook - Shreyas SiravaraAutomating Gluster @ Facebook - Shreyas Siravara
Automating Gluster @ Facebook - Shreyas Siravara
 
nfusr: a new userspace NFS client based on libnfs - Shreyas Siravara
nfusr: a new userspace NFS client based on libnfs - Shreyas Siravaranfusr: a new userspace NFS client based on libnfs - Shreyas Siravara
nfusr: a new userspace NFS client based on libnfs - Shreyas Siravara
 
Facebook’s upstream approach to GlusterFS - David Hasson
Facebook’s upstream approach to GlusterFS  - David HassonFacebook’s upstream approach to GlusterFS  - David Hasson
Facebook’s upstream approach to GlusterFS - David Hasson
 
Throttling Traffic at Facebook Scale
Throttling Traffic at Facebook ScaleThrottling Traffic at Facebook Scale
Throttling Traffic at Facebook Scale
 
GlusterFS w/ Tiered XFS
GlusterFS w/ Tiered XFS  GlusterFS w/ Tiered XFS
GlusterFS w/ Tiered XFS
 
Gluster Metrics: why they are crucial for running stable deployments of all s...
Gluster Metrics: why they are crucial for running stable deployments of all s...Gluster Metrics: why they are crucial for running stable deployments of all s...
Gluster Metrics: why they are crucial for running stable deployments of all s...
 
Up and Running with Glusto & Glusto-Tests in 5 Minutes (or less)
Up and Running with Glusto & Glusto-Tests in 5 Minutes (or less)Up and Running with Glusto & Glusto-Tests in 5 Minutes (or less)
Up and Running with Glusto & Glusto-Tests in 5 Minutes (or less)
 
Data Reduction for Gluster with VDO
Data Reduction for Gluster with VDOData Reduction for Gluster with VDO
Data Reduction for Gluster with VDO
 
Releases: What are contributors responsible for
Releases: What are contributors responsible forReleases: What are contributors responsible for
Releases: What are contributors responsible for
 
RIO Distribution: Reconstructing the onion - Shyamsundar Ranganathan
RIO Distribution: Reconstructing the onion - Shyamsundar RanganathanRIO Distribution: Reconstructing the onion - Shyamsundar Ranganathan
RIO Distribution: Reconstructing the onion - Shyamsundar Ranganathan
 
Gluster and Kubernetes
Gluster and KubernetesGluster and Kubernetes
Gluster and Kubernetes
 
Native Clients, more the merrier with GFProxy!
Native Clients, more the merrier with GFProxy!Native Clients, more the merrier with GFProxy!
Native Clients, more the merrier with GFProxy!
 
Gluster: a SWOT Analysis
Gluster: a SWOT Analysis Gluster: a SWOT Analysis
Gluster: a SWOT Analysis
 
GlusterD-2.0: What's Happening? - Kaushal Madappa
GlusterD-2.0: What's Happening? - Kaushal MadappaGlusterD-2.0: What's Happening? - Kaushal Madappa
GlusterD-2.0: What's Happening? - Kaushal Madappa
 
Scalability and Performance of CNS 3.6
Scalability and Performance of CNS 3.6Scalability and Performance of CNS 3.6
Scalability and Performance of CNS 3.6
 
What Makes Us Fail
What Makes Us FailWhat Makes Us Fail
What Makes Us Fail
 
Gluster as Native Storage for Containers - past, present and future
Gluster as Native Storage for Containers - past, present and futureGluster as Native Storage for Containers - past, present and future
Gluster as Native Storage for Containers - past, present and future
 
Heketi Functionality into Glusterd2
Heketi Functionality into Glusterd2Heketi Functionality into Glusterd2
Heketi Functionality into Glusterd2
 
Architecture of the High Availability Solution for Ganesha and Samba with Kal...
Architecture of the High Availability Solution for Ganesha and Samba with Kal...Architecture of the High Availability Solution for Ganesha and Samba with Kal...
Architecture of the High Availability Solution for Ganesha and Samba with Kal...
 
Challenges with Gluster and Persistent Memory with Dan Lambright
Challenges with Gluster and Persistent Memory with Dan LambrightChallenges with Gluster and Persistent Memory with Dan Lambright
Challenges with Gluster and Persistent Memory with Dan Lambright
 

Kürzlich hochgeladen

KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 

Kürzlich hochgeladen (20)

KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 

Lisa 2015-gluster fs-introduction

  • 1. November 8–13, 2015 | Washington, D.C. www.usenix.org/lisa15 #lisa15 GlusterFSGlusterFS A Scale-out Software Defined Storage Rajesh Joseph Poornima Gurusiddaiah
  • 3. 05/17/16 GlusterFS Open-source general purpose scale-out distributed file system Aggregates storage exports over network interconnect to provide a single unified namespace Layered on disk file systems that support extended attributes No meta-data server Modular Architecture for Scale and Functionality Heterogeneous commodity hardware Scalable to petabytes & beyond
  • 5. 05/17/16 Server Intel/AMD x86 64-bit processor Disk: 8GB minimum using direct- attached-storage, RAID, Amazon EBS, and FC/Infiniband/iSCSI SAN disk backends using SATA/SAS/FC disks Memory: 1GB minimum Logical Volume Manager LVM2 with thin provisioning Networking Gigabit Ethernet 10 Gigabit Ethernet InfiniBand (OFED 1.5.2 or later) Filesystem POSIX w/ Extended Attributes (EXT4, XFS, BTRFS, …) A node is server capable of hosting GlusterFS bricks Node
  • 6. 05/17/16 Trusted Storage Pool Also known as Cluster Trusted Storage Pool is formed by invitation – “probe” Members can be dynamically added and removed from the pool Only nodes in a Trusted Storage Pool can participate in volume creation A collection of storage servers (Node) Node 2Node 1 Node 3
  • 7. 05/17/16 Bricks A brick is the combination of a node and an export directory – e.g. hostname:/dir Layered on posix compliant file-system (e.g. XFS, ext4) Each brick inherits limits of the underlying filesystem It is recommended to use an independent thinly provisioned LVM as brick Thin provisioning is needed by Snapshot feature A unit of storage used as a capacity building block
  • 8. 05/17/16 Bricks Storage devices Thin LV Volume GroupBrick1 Brick2 Thin Pool mount Thin LV Thin Pool mount
  • 9. 05/17/16 GlusterFS Volume Node hosting these bricks should be part of a single Trusted Storage Pool One or more volumes can be hosted on the same node A volume is a logical collection of one or more bricks
  • 10. 05/17/16 GlusterFS Volume GlusterFS Volume Storage Node Storage Node Storage Node Brick Brick Brick Brick
  • 11. 05/17/16 GlusterFS Volume Volume1 Storage Node Storage Node Storage Node Brick Brick Brick BrickBrick Brick Volume2
  • 12. 05/17/16 Translators Translators can be stacked together for achieving desired functionality Translators are deployment agnostic – can be loaded in either the client or server stacks Building block of GlusterFS process
  • 13. 05/17/16 Translators GigE, 10GigE – TCP/IP / Inifiniband - RDMA Client Client Client Client Network VFS Distribute Replicate Replicate Server POSIX Server POSIX Server POSIX Server POSIX I/O Cache Read Ahead Bricks Client
  • 14. 05/17/16 GlusterFS Processes glusterd Management daemon One instance on each GlusterFS node glusterfsd GlusterFS brick daemon One process per brick on each node Managed by glusterd gluster Gluster console manager (CLI) glusterfs Node services One process for each service NFS Server, Self heal, Quota, ... mount.glusterfs FUSE native client mount extension
  • 15. 05/17/16 Accessing Volume Volume can be mounted on local file-system Following protocols supported for accessing volume GlusterFS Native client Filesystem in Userspace (FUSE) libgfapi flexible abstracted storage Samba, NFS-Ganesha, QEMU NFS NFS Ganesha Gluster NFSv3 SMB / CIFS Windows and Linux Gluster For OpenStack Object-based access via OpenStack Swift
  • 17. 05/17/16 GlusterFS Native Client (FUSE) FUSE kernel module allows the filesystem to be built and operated entirely in userspace Specify mount to any GlusterFS server Native Client fetches volfile from mount server, then communicates directly with all nodes to access data
  • 18. 05/17/16 Accessing Volume – Fuse client Gluster Volume Storage Node Storage Node Storage Node Brick Brick Brick Brick mount Client
  • 19. 05/17/16 libgfapi Userspace library for accessing GlusterFS volume File-system like API No FUSE, No copies, No context switches Samba, QEMU, NFS-Ganesha integrated with libgfapi
  • 20. 05/17/16 Libgfapi - Advantage Storage NodeClient Userspace Kernel Userspace Kernel App /mnt/gmount /dev/fuseKernel VFS Net Net Glusterfsd Brick XFS GlusterFS Fuse Library
  • 21. 05/17/16 Libgfapi - Advantage Storage NodeClient Userspace Kernel Userspace Kernel Net Net Glusterfsd Brick XFS App libgfapi
  • 22. 05/17/16 Gluster NFS Supports NFS v3 protocol Mount to any server or use load-balancer GlusterFS NFS server uses Network Lock Manager (NLM) to synchronize locks across clients Load balancing should be managed externally
  • 23. 05/17/16 Accessing Volume – Gluster NFS Gluster Volume Storage Node Storage Node Storage Node Brick Brick Brick Brick mount Client NFSNFSNFS IP Load Balancer
  • 24. 05/17/16 SMB / CIFS Samba VFS plugin for GlusterFS Uses libgfapi Must be setup on each server you wish to connect CTDB is required for Samba clustering
  • 25. 05/17/16 Accessing Volume – SMB Gluster Volume Storage Node Storage Node Storage Node Brick Brick Brick Brick mount Client SMBDSMBDSMBD IP Load Balancer
  • 26. 05/17/16 Volume Types Type of a volume is specified at the time of volume creation Volume type determines how and where data is placed Basic Volume Types Distributed Replicated Dispersed Striped Sharded Advance Volume Types Distributed Replicated Distributed Dispersed Distributed Striped Distributed Striped Replicated
  • 27. 05/17/16 Distributed Volume Distributes files across various bricks of the volume Similar to file-level RAID-0 Scaling but no high availability Uses Davies-Meyer hash algorithm A 32-bit hash space is divided into N ranges for N bricks Removes the need for an external meta data server
  • 28. 05/17/16 Distribute Volume Storage Node Storage Node Storage Node Brick Brick Brick Client File1File2 [0, a] [a + 1, b] [b + 1, c] File1 Hash = x, Where 0 <= x <= aFile2 Hash = y, Where b < y <= c
  • 29. 05/17/16 Replicate Volume Synchronous replication of all directory and file updates Similar to file-level RAID-1 High availability but no scaling Replica pairs are decided at the volume creation time Its recommended to host each brick of replica set on different node
  • 31. 05/17/16 Disperse Volume Erasure Coded volume High availability with less number of bricks Store m disk of data on k disk (k > m) n (k-m) redundant disks Following setup is the most tested configuration 6 bricks with redundancy level 2 (4 +2) 11 bricks with redundancy level 3 (8 +3) 12 bricks with redundancy level 4 (8 + 4)
  • 32. 05/17/16 Disperse Volume Storage Node Brick Client File1 Storage Node Brick Storage Node Brick Data Part Parity / Redundancy Part
  • 33. 05/17/16 Striped Volume Individual files split among bricks (sparse files) Similar to block-level RAID-0 Recommended only when very large files greater than the size of the disks are present Chunks are files with holes – this helps in maintaining offset consistency
  • 35. 05/17/16 Sharded (Stripe – 2.0) Volume Individual files split among bricks Each part split is a different file in the backend File name of each part derived from GFID (UUID) of actual file Unlike Stripe file rename will not have any impact on these parts. Unlike other volume types shard is a volume option which can be set on any volume Recommended only when very large files greater than the size of the disks are present
  • 36. 05/17/16 Sharded Volume Storage Node Brick Client File1 Storage Node Brick Storage Node Brick GFID1.1 GFID1.2 GFID1.3
  • 37. 05/17/16 Distribute Replicate Volume Distribute files across replicated bricks Number of bricks must be a multiple of the replica count Ordering of bricks in volume definition matters Scaling and high availability Most preferred model of deployment
  • 38. 05/17/16 Distribute Replicate Volume Storage Node Brick Storage Node Brick Storage Node Brick Client File1File2 Storage Node Brick Replica Pairs Replica Pairs
  • 39. 05/17/16 Performance Different SLA than a local file-system Uneven performance distribution between nodes and clients Various performance options available (e.g., read-ahead, md-cache, etc) Based on workload use various options Don't forget to attend detailed session on performance by Jeff Darcy Topic  : Evaluating Distributed File System Performance (GlusterFS and Ceph) When   : Friday, November 13, 2015 ­ 9:00am­10:30am Detail : https://www.usenix.org/conference/lisa15/conference­program/presentation/darcy
  • 41. 05/17/16 Additional Features Geo Replication Volume Snapshot Quota Bit-rot Compression Encryption at rest Trash / Recycle-bin pNFS with NFS-Ganesha Data Tiering User Serviceable Snapshots Server side Quorum Client side Quorum And many more ...
  • 42. 05/17/16 Geo-Replication Mirrors data across geographically distributed trusted storage pools Provides back-ups of data for disaster recovery Master-Slave model Asynchronous replication Provides an incremental continuous replication service over Local Area Networks (LANs) Wide Area Network (WANs) Internet
  • 43. 05/17/16 Geo-Replication Trusted Storage Pool Trusted Storage Pool Network LAN/WAN/Internet Master Volume Slave Volume
  • 44. 05/17/16 GlusterFS Snapshot Volume level crash consistent snapshots LVM2 based Operates only on thin-provisioned volumes Snapshots themselves are thin-provisioned snapshot volumes A GlusterFS volume snapshot consists of snapshots of the individual bricks making up the volume Snapshot of a GlusterFS volume is also a GlusterFS volume
  • 45. 05/17/16 Thinly Provisioned LVM2 Volume & Snapshot Storage devices SnapshotThin LV Thin Pool Volume Group
  • 46. 05/17/16 Gluster Volume Snapshot Gluster Volume Storage Node Storage Node Brick1 Brick2 Brick3 Brick4 Brick1_s1 Brick2_s1 Brick4_s1Brick3_s1 Snapshot volume Barrier
  • 47. 05/17/16 Data Tiering Data classification and placement of data Currently based on access rate Tiered volume can be created by attaching a tier Bricks in original volume are “cold” bricks Bricks in attached tier are “hot” bricks Hot bricks are normally hosted on faster disks e.g. SSDs Tier can be attached or detached using volume commands
  • 48. 05/17/16 Tiered Volume Storage Node Brick Storage Node Brick Storage Node Brick Client Storage Node BrickBrick Cold Tier Hot Tier TT
  • 49. 05/17/16 Tiered Volume Replica Tier Distribute Replica Replica Brick Brick Brick Brick Brick Brick Brick Hot Tier Cold Tier
  • 50. 05/17/16 Future Work New Style Replication (NSR) Scalability improvements (1000+ Nodes) Multi-protocol support Multi-master Geo-Replication Caching Improvements Btrfs Support And many more…

Hinweis der Redaktion

  1. If Snapshot feature is not needed one can opt for regular LVM or non-LVM setup. Bricks and other volume metadata constitute the backend. It is not recommended to do any direct operation (e.g. I/O) on the backend. Always use the mount point to modify the file and use gluster commands to modify other volume metadata
  2. Data transformations such as compression or de-duplication would benefit from sharding because portions of the file may be processed rather than exclusively at whole-file granularity. The cache tiering feature would benefit from sharding. Currently large files must be migrated in full between tiers, even if only a small portion of the file is accessed. With sharding, only the shard accessed would need to be migrated.