SlideShare ist ein Scribd-Unternehmen logo
1 von 34
Downloaden Sie, um offline zu lesen
The Future of GlusterFS and Gluster.org
                                 John Mark Walker
                         GlusterFS Community Guy
                                     Red Hat, Inc.
                                  January 25, 2012
The Roots of GlusterFS
       ●   Distributed storage solutions difficult to find
       ●   Decided to write their own
       ●   No filesystem experts – Pro & Con
       ●   Applied lessons from microkernel architecture
             –     GNU Hurd




January 26, 2012          The Future of Gluster.org - John Mark Walker
The Roots of GlusterFS
●   All storage solutions were either
    ●   Too expensive. or...
    ●   Not scalable, or…
    ●   Single purpose, or…
    ●   Don’t support legacy apps, or…
    ●   Don't support new apps, or...
    ●   Do some combo of the above, but not very well
The Roots of GlusterFS
●   The challenge:
    ●   Create a storage system that was…
        –   Scalable
        –   Seamlessly integrated in the data center
        –   Future-proof
●   The solution: GlusterFS
    ●   Scalable, with DHT
    ●   POSIX-compliant
    ●   Stackable
    ●   User-space
GlusterFS Client Architecture
●   Creating a file system in user space
    ●   Utilizes fuse module
        –   Kernel goes through fuse, which hands off to glusterd


            glusterd                  Applications




                       Linux kernel
                       Fuse Ext4 …       …
No Centralized Metadata
   Client A         Client B         Client C




  Server X         Server Y         Server Z




Extended Attr.   Extended Attr.   Extended Attr.

    Files            Files            Files
What is a Translator?
●   Add/remove layers         FUSE Interface Layer

●   Reorder layers             Performance Layer
●   Move layers between        Distribution Layer
    client and server
                               Replication Layer
●   Implement new layers
                                 Protocol Layer
    ●   e.g. encryption
●   Replace old layers       Local Filesystem Later

    ●   e.g. replication
Some Features
●   Distributed, replicated and/or striped volumes
●   Global namespace
●   High availability
●   Geo-replication
●   Rebalancing
●   Remove or replace bricks
●   Self healing
●   volume profile and top metrics
No one ever expects the Red Hat acquisition
Red Hat Invests in GlusterFS
●   Unstructured data volume to grow 44x by 2020
●   Cloud and virtualization are driving scale-out
    storage growth
●   Scale-out storage shipments to exceed 63,000
    PB by 2015 (74% CAGR)
●   40% of core cloud spend related to storage
●   GlusterFS-based solutions up to 50% less than
    other storage systems
Red Hat Invests in GlusterFS
●   GlusterFS adds to the
                                           JBoss
    Red Hat stack
    ●   Complements other
                                           RHEL
        Red Hat offerings
    ●   Many integration points
                                           Bare
                                  RHEV               Clouds
●   More engineers                         Metal

    hacking on GlusterFS
                                  GlusterFS Unified Storage
    than ever before
Red Hat Invests in GlusterFS
●   Acceleration of community investment
    ●   GlusterFS needs to be “bigger than Red Hat”
    ●   Transformation of GlusterFS from product to project
        – From “open core” to upstream
    ●   More resources for engineering and community
        outreach
    ●   Red Hat's success rests on economies of scale
        –   Critical mass of users and developers
Join a Winning Team

                       “Join me, and
                       together, we can
                       rule the galaxy...”

● We're hiring hackers and engineers
● Looking for community collaborators

    ●   ISVs, students, IT professionals, fans, et
        al.
The Immediate Future
The Gluster Community
Global adoption   ●   300,000+ downloads
                      ●   ~35,000 /month
                      ●   >300% increase Y/Y
                  ●   1000+ deployments
                      ●   45 countries
                  ●   2,000+ registered
                      users
                      ●   Mailing lists, Forums, etc.
The Gluster Community
●   Why are we changing?
    ●   Only 1 non-Red Hat core contributor
        –   There were 2, but he acquired us
    ●   Want to be the software standard for distributed
        storage
    ●   Want to be more inclusive, more community-driven


    Goal: create global ecosystem that
    supports ISVs, service providers and more
Towards “Real” Open Source
●   GlusterFS, prior to acquisition
    ●   “Open Core”
    ●   Tied directly to Gluster products
        –   No differentiation
    ●   Very little outside collaboration
    ●   Contributors had to assign copyright to Gluster
        –   Discouraged would-be contributors
Towards “Real” Open Source
“Open Core”
 ●   All engineering controlled by
     project/product sponsor

 ●   No innovation outside of
                                     Commercial Product
     core engineering team

 ●   All open source features
     also in commercial product

 ●   Many features in
                                     Open Source
     Commercial product not in          Code
     open source code
Towards “Real” Open Source
“Real” Open Source
 ●   Many points of collaboration
     and innovation in open
     source project
                                         Open Source Code

 ●   Engineering team from
     multiple sources

 ●   Project and product do not
     completely overlap

 ●   Commercial products are        Commercial Products
     hardened, more secure and
     thoroughly tested
Towards “Real” Open Source
“Real” Open Source
 ●   Enables more innovation on
     the fringes
                                    Fedora Linux
 ●   Engineering team from
     multiple sources

 ●   Open source project is
     “upstream” from commercial
     product

 ●   “Downstream” products are    RHEL
     hardened, more secure and
     thoroughly tested
Towards “Real” Open Source
“Real” Open Source
 ●   Enables more innovation on
     the fringes
                                           GlusterFS
 ●   Engineering team from
     multiple sources

 ●   Open source project is
     “upstream” from commercial
     product

 ●   “Downstream” products are    Red Hat Storage
     hardened, more secure and
     thoroughly tested
Project Roadmaps
What's New in GlusterFS 3.3
●   New features
    ●   Unified File & Object access
    ●   Hadoop / HDFS compatibility
●   New Volume Type
    ●   Replicated + striped (+ distributed) volumes
●   Enhancements to Distributed volumes (DHT translator)
    ●   Rebalance can migrate open files
    ●   Remove-brick can migrate data to remaining bricks
●   Enhancements to Replicated volumes (AFR translator)
    ●   Change replica count on an active volume, add replication to distribute-only volumes
    ●   Granular locking – Much faster self-healing for large files
    ●   Proactive self-heal process starts without FS stat
    ●   Round-trip reduction for lower latency
    ●   Quorum enforcement - avoid split brain scenarios

    GlusterFS 3.3 ETA in Q2/Q3 2012
File and Object Storage
●   Traditional SAN / NAS support either
    file or block storage
●   New storage methodologies
    implement RESTful APIs over HTTP
●   Demand for unifying the storage
    infrastructure increasing
●   Treats files as objects and volumes
    as buckets
●   Available now in 3.3 betas
●   Soon to be backported to 3.2.x
●   Contributing to OpenStack project
    ●   Re-factored Swift API
Technology Integrations

GlusterFS used as VM storage system
                                                Mobile Apps. Web Clients. Enterprise Software Ecosystem
 ●   Pause and re-start VM’s, even on another                     API Layer
     hypervisor

                                                                    …
 ●   HA and DR for VM’s                                            Compute
 ●   Faster VM deployment
 ●   V-motion –like capability                                  Unified File &
                                                                Object Storage
Shared storage ISOs and appliances
 ●   oVirt / RHEV
 ●   CloudStack
 ●   OpenStack                                        OpenStack Imaging Services




     Goal: The standard for cloud storage
HDFS/Hadoop Compatibility
●   HDFS compatibility library
    ●   Simultaneous file and object access within Hadoop

●   Benefits
    ●   Legacy app access to MapReduce applications
    ●   Enables data storage consolidation

●   Simplify and unify storage deployments
●   Provide users with file level access to data
●   Enable legacy applications to access data via NFS
    ●   Analytic apps can access data without modification
The Gluster Community
●   What is changing?
    ●   HekaFS / CloudFS being folded into Gluster project
        –   HekaFS == GlusterFS + multi-tenancy and SSL for auth
            and data encryption
        –   HekaFS.org
        –   ETA ~9 months
What else?
GlusterFS Advisory Board
●   Advisory board
    ●   Consists of industry and community leaders from Facebook, Citrix,
        Fedora, and OpenStack
        –   Richard Wareing, Storage Engineer, Facebook
        –   Jeff Darcy, Filesystem Engineer, Red Hat; Founder, HekaFS Project
        –   AB Periasamy, Co-Founder, GlusterFS project
        –   Ewan Mellor, Xen Engineer, Citrix; Member, OpenStack project
        –   David Nalley, CloudStack Community Mgr; Fedora Advisory Board
        –   Louis Zuckerman, Sr. System Administrator, Picture Marketing
        –   Joe Julian, Sr. System Administrator, Ed Wyse Beauty Products
        –   Greg DeKoenigsberg, Community VP, Eucalyptus; co-founder, Fedora
        –   John Mark Walker, Gluster.org Community Guy (Chair)
Gluster.org Web Site
●   Services for users and developers
    ●   Developer section with comprehensive docs
    ●   Collaborative project hosting
    ●   Continuing development of end user documentation
        and interactive tools
●   Published roadmaps
    ●   Transparent feature development
GlusterFS Downloads
●   Where's the code?
    ●   GlusterFS 3.3
        –   Simultaneous file + object
        –   HDFS compatibility
        –   Improved self-healing + VM hosting
             ●   Granular locking
        –   Beta 3 due Feb/Mar 2012
        –   http://download.gluster.org/pub/gluster/glusterfs
Gluster.org Services
●   Gluster.org
    ●   Portal into all things GlusterFS
●   Community.gluster.org
    ●   Self-support site; Q&A; HOWTOs; tutorials
●   Patch review, CI
    ●   review.gluster.com
●   #gluster
    ●   IRC channel on Freenode
Development Process
●   Source code
    ●   Hosted at github.com/gluster
●   Bugs and Feature Requests
    ●   Bugzilla.redhat.com – select GlusterFS from menu
●   Patches
    ●   Submit via Gerritt at review.gluster.com
●   See Development Work Flow doc:
    ●   gluster.org/community/documentation/index.php/Development_Work_Flow
Thank You
●   GlusterFS contacts
    ●   Gluster.org/interact/mailinglists
    ●   @RedHatStorage & @GlusterOrg
    ●   #gluster on Freenode
●   My contact info
    ●   johnmark@redhat.com
    ●
        Twitter & identi.ca: @johnmark

Weitere ähnliche Inhalte

Was ist angesagt?

Room 1 - 2 - Nguyễn Văn Thắng & Dzung Nguyen - Proxmox VE và ZFS over iscsi
Room 1 - 2 - Nguyễn Văn Thắng & Dzung Nguyen - Proxmox VE và ZFS over iscsiRoom 1 - 2 - Nguyễn Văn Thắng & Dzung Nguyen - Proxmox VE và ZFS over iscsi
Room 1 - 2 - Nguyễn Văn Thắng & Dzung Nguyen - Proxmox VE và ZFS over iscsiVietnam Open Infrastructure User Group
 
Why kubernetes matters
Why kubernetes mattersWhy kubernetes matters
Why kubernetes mattersPlatform9
 
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...Vietnam Open Infrastructure User Group
 
Monitoring, Logging and Tracing on Kubernetes
Monitoring, Logging and Tracing on KubernetesMonitoring, Logging and Tracing on Kubernetes
Monitoring, Logging and Tracing on KubernetesMartin Etmajer
 
NVMe Over Fabrics Support in Linux
NVMe Over Fabrics Support in LinuxNVMe Over Fabrics Support in Linux
NVMe Over Fabrics Support in LinuxLF Events
 
BPF - in-kernel virtual machine
BPF - in-kernel virtual machineBPF - in-kernel virtual machine
BPF - in-kernel virtual machineAlexei Starovoitov
 
Dockers and kubernetes
Dockers and kubernetesDockers and kubernetes
Dockers and kubernetesDr Ganesh Iyer
 
Cassandra における SSD の活用
Cassandra における SSD の活用Cassandra における SSD の活用
Cassandra における SSD の活用Yuji Ito
 
Memory management in Linux
Memory management in LinuxMemory management in Linux
Memory management in LinuxRaghu Udiyar
 
Introduction to Docker storage, volume and image
Introduction to Docker storage, volume and imageIntroduction to Docker storage, volume and image
Introduction to Docker storage, volume and imageejlp12
 
VMware vSAN - Novosco, June 2017
VMware vSAN - Novosco, June 2017VMware vSAN - Novosco, June 2017
VMware vSAN - Novosco, June 2017Novosco
 
Kubernetes dealing with storage and persistence
Kubernetes  dealing with storage and persistenceKubernetes  dealing with storage and persistence
Kubernetes dealing with storage and persistenceJanakiram MSV
 
Git Series. Episode 3. Git Flow and Github-Flow
Git Series. Episode 3. Git Flow and Github-FlowGit Series. Episode 3. Git Flow and Github-Flow
Git Series. Episode 3. Git Flow and Github-FlowMikhail Melnik
 
Dealing with Merge Conflicts in Git
Dealing with Merge Conflicts in GitDealing with Merge Conflicts in Git
Dealing with Merge Conflicts in Gitgittower
 
Trunk based development for Beginners
Trunk based development for BeginnersTrunk based development for Beginners
Trunk based development for BeginnersNebulaworks
 

Was ist angesagt? (20)

Room 1 - 2 - Nguyễn Văn Thắng & Dzung Nguyen - Proxmox VE và ZFS over iscsi
Room 1 - 2 - Nguyễn Văn Thắng & Dzung Nguyen - Proxmox VE và ZFS over iscsiRoom 1 - 2 - Nguyễn Văn Thắng & Dzung Nguyen - Proxmox VE và ZFS over iscsi
Room 1 - 2 - Nguyễn Văn Thắng & Dzung Nguyen - Proxmox VE và ZFS over iscsi
 
Why kubernetes matters
Why kubernetes mattersWhy kubernetes matters
Why kubernetes matters
 
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
Room 3 - 6 - Nguyễn Văn Thắng & Dzung Nguyen - Ứng dụng openzfs làm lưu trữ t...
 
ZFS
ZFSZFS
ZFS
 
Monitoring, Logging and Tracing on Kubernetes
Monitoring, Logging and Tracing on KubernetesMonitoring, Logging and Tracing on Kubernetes
Monitoring, Logging and Tracing on Kubernetes
 
NVMe Over Fabrics Support in Linux
NVMe Over Fabrics Support in LinuxNVMe Over Fabrics Support in Linux
NVMe Over Fabrics Support in Linux
 
BPF - in-kernel virtual machine
BPF - in-kernel virtual machineBPF - in-kernel virtual machine
BPF - in-kernel virtual machine
 
Dockers and kubernetes
Dockers and kubernetesDockers and kubernetes
Dockers and kubernetes
 
GIT INTRODUCTION
GIT INTRODUCTIONGIT INTRODUCTION
GIT INTRODUCTION
 
Cassandra における SSD の活用
Cassandra における SSD の活用Cassandra における SSD の活用
Cassandra における SSD の活用
 
Git
GitGit
Git
 
Memory management in Linux
Memory management in LinuxMemory management in Linux
Memory management in Linux
 
4. linux file systems
4. linux file systems4. linux file systems
4. linux file systems
 
Introduction to Docker storage, volume and image
Introduction to Docker storage, volume and imageIntroduction to Docker storage, volume and image
Introduction to Docker storage, volume and image
 
VMware vSAN - Novosco, June 2017
VMware vSAN - Novosco, June 2017VMware vSAN - Novosco, June 2017
VMware vSAN - Novosco, June 2017
 
Kubernetes dealing with storage and persistence
Kubernetes  dealing with storage and persistenceKubernetes  dealing with storage and persistence
Kubernetes dealing with storage and persistence
 
Backup using rsync
Backup using rsyncBackup using rsync
Backup using rsync
 
Git Series. Episode 3. Git Flow and Github-Flow
Git Series. Episode 3. Git Flow and Github-FlowGit Series. Episode 3. Git Flow and Github-Flow
Git Series. Episode 3. Git Flow and Github-Flow
 
Dealing with Merge Conflicts in Git
Dealing with Merge Conflicts in GitDealing with Merge Conflicts in Git
Dealing with Merge Conflicts in Git
 
Trunk based development for Beginners
Trunk based development for BeginnersTrunk based development for Beginners
Trunk based development for Beginners
 

Andere mochten auch

Red Hat Storage - Introduction to GlusterFS
Red Hat Storage - Introduction to GlusterFSRed Hat Storage - Introduction to GlusterFS
Red Hat Storage - Introduction to GlusterFSGlusterFS
 
BitRot (The silent corruption of data on disk ) detection in GlusterFS (Glust...
BitRot (The silent corruption of data on disk ) detection in GlusterFS (Glust...BitRot (The silent corruption of data on disk ) detection in GlusterFS (Glust...
BitRot (The silent corruption of data on disk ) detection in GlusterFS (Glust...Gaurav Kumar Garg
 
Gluster fs architecture_future_directions_tlv
Gluster fs architecture_future_directions_tlvGluster fs architecture_future_directions_tlv
Gluster fs architecture_future_directions_tlvSahina Bose
 
Red Hat Storage for Mere Mortals
Red Hat Storage for Mere MortalsRed Hat Storage for Mere Mortals
Red Hat Storage for Mere MortalsRed_Hat_Storage
 
Ceph Block Devices: A Deep Dive
Ceph Block Devices:  A Deep DiveCeph Block Devices:  A Deep Dive
Ceph Block Devices: A Deep DiveRed_Hat_Storage
 
Red Hat Gluster Storage Performance
Red Hat Gluster Storage PerformanceRed Hat Gluster Storage Performance
Red Hat Gluster Storage PerformanceRed_Hat_Storage
 

Andere mochten auch (7)

Red Hat Storage - Introduction to GlusterFS
Red Hat Storage - Introduction to GlusterFSRed Hat Storage - Introduction to GlusterFS
Red Hat Storage - Introduction to GlusterFS
 
BitRot (The silent corruption of data on disk ) detection in GlusterFS (Glust...
BitRot (The silent corruption of data on disk ) detection in GlusterFS (Glust...BitRot (The silent corruption of data on disk ) detection in GlusterFS (Glust...
BitRot (The silent corruption of data on disk ) detection in GlusterFS (Glust...
 
Erasure Code in Ceph
Erasure Code in CephErasure Code in Ceph
Erasure Code in Ceph
 
Gluster fs architecture_future_directions_tlv
Gluster fs architecture_future_directions_tlvGluster fs architecture_future_directions_tlv
Gluster fs architecture_future_directions_tlv
 
Red Hat Storage for Mere Mortals
Red Hat Storage for Mere MortalsRed Hat Storage for Mere Mortals
Red Hat Storage for Mere Mortals
 
Ceph Block Devices: A Deep Dive
Ceph Block Devices:  A Deep DiveCeph Block Devices:  A Deep Dive
Ceph Block Devices: A Deep Dive
 
Red Hat Gluster Storage Performance
Red Hat Gluster Storage PerformanceRed Hat Gluster Storage Performance
Red Hat Gluster Storage Performance
 

Ähnlich wie The Future of GlusterFS and Gluster.org

The Future of GlusterFS and Gluster.org
The Future of GlusterFS and Gluster.orgThe Future of GlusterFS and Gluster.org
The Future of GlusterFS and Gluster.orgJohn Mark Walker
 
Ceph: A decade in the making and still going strong
Ceph: A decade in the making and still going strongCeph: A decade in the making and still going strong
Ceph: A decade in the making and still going strongPatrick McGarry
 
Ceph Day New York: Ceph: one decade in
Ceph Day New York: Ceph: one decade inCeph Day New York: Ceph: one decade in
Ceph Day New York: Ceph: one decade inCeph Community
 
Orchestrating stateful applications with PKS and Portworx
Orchestrating stateful applications with PKS and PortworxOrchestrating stateful applications with PKS and Portworx
Orchestrating stateful applications with PKS and PortworxVMware Tanzu
 
Orchestrating Stateful Applications with PKS and Portworx
Orchestrating Stateful Applications with PKS and PortworxOrchestrating Stateful Applications with PKS and Portworx
Orchestrating Stateful Applications with PKS and PortworxCarlos Andrés García
 
Build Your Own PaaS, Just like Red Hat's OpenShift from LinuxCon 2013 New Orl...
Build Your Own PaaS, Just like Red Hat's OpenShift from LinuxCon 2013 New Orl...Build Your Own PaaS, Just like Red Hat's OpenShift from LinuxCon 2013 New Orl...
Build Your Own PaaS, Just like Red Hat's OpenShift from LinuxCon 2013 New Orl...OpenShift Origin
 
Introduction into Ceph storage for OpenStack
Introduction into Ceph storage for OpenStackIntroduction into Ceph storage for OpenStack
Introduction into Ceph storage for OpenStackOpenStack_Online
 
London Ceph Day Keynote: Building Tomorrow's Ceph
London Ceph Day Keynote: Building Tomorrow's Ceph London Ceph Day Keynote: Building Tomorrow's Ceph
London Ceph Day Keynote: Building Tomorrow's Ceph Ceph Community
 
Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph
Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph
Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph Ceph Community
 
Ceph Day NYC: Building Tomorrow's Ceph
Ceph Day NYC: Building Tomorrow's CephCeph Day NYC: Building Tomorrow's Ceph
Ceph Day NYC: Building Tomorrow's CephCeph Community
 
Ceph Day SF 2015 - Keynote
Ceph Day SF 2015 - Keynote Ceph Day SF 2015 - Keynote
Ceph Day SF 2015 - Keynote Ceph Community
 
A Tight Ship: How Containers and SDS Optimize the Enterprise
 A Tight Ship: How Containers and SDS Optimize the Enterprise A Tight Ship: How Containers and SDS Optimize the Enterprise
A Tight Ship: How Containers and SDS Optimize the EnterpriseEric Kavanagh
 
Alluxio 2.0 Deep Dive – Simplifying data access for cloud workloads
Alluxio 2.0 Deep Dive – Simplifying data access for cloud workloadsAlluxio 2.0 Deep Dive – Simplifying data access for cloud workloads
Alluxio 2.0 Deep Dive – Simplifying data access for cloud workloadsAlluxio, Inc.
 
Accelerate Analytics and ML in the Hybrid Cloud Era
Accelerate Analytics and ML in the Hybrid Cloud EraAccelerate Analytics and ML in the Hybrid Cloud Era
Accelerate Analytics and ML in the Hybrid Cloud EraAlluxio, Inc.
 
Transforming Application Delivery with PaaS and Linux Containers
Transforming Application Delivery with PaaS and Linux ContainersTransforming Application Delivery with PaaS and Linux Containers
Transforming Application Delivery with PaaS and Linux ContainersGiovanni Galloro
 
Vmware Serengeti - Based on Infochimps Ironfan
Vmware Serengeti - Based on Infochimps IronfanVmware Serengeti - Based on Infochimps Ironfan
Vmware Serengeti - Based on Infochimps IronfanJim Kaskade
 
How @twitterhadoop chose google cloud
How @twitterhadoop chose google cloudHow @twitterhadoop chose google cloud
How @twitterhadoop chose google cloudlohitvijayarenu
 
The Next Generation Datacenter
The Next Generation DatacenterThe Next Generation Datacenter
The Next Generation DatacenterRed Hat Events
 
Moving data to the cloud BY CESAR ROJAS from Pivotal
Moving data to the cloud BY CESAR ROJAS from PivotalMoving data to the cloud BY CESAR ROJAS from Pivotal
Moving data to the cloud BY CESAR ROJAS from PivotalVMware Tanzu Korea
 

Ähnlich wie The Future of GlusterFS and Gluster.org (20)

vBACD - Distributed Petabyte-Scale Cloud Storage with GlusterFS - 2/28
vBACD - Distributed Petabyte-Scale Cloud Storage with GlusterFS - 2/28vBACD - Distributed Petabyte-Scale Cloud Storage with GlusterFS - 2/28
vBACD - Distributed Petabyte-Scale Cloud Storage with GlusterFS - 2/28
 
The Future of GlusterFS and Gluster.org
The Future of GlusterFS and Gluster.orgThe Future of GlusterFS and Gluster.org
The Future of GlusterFS and Gluster.org
 
Ceph: A decade in the making and still going strong
Ceph: A decade in the making and still going strongCeph: A decade in the making and still going strong
Ceph: A decade in the making and still going strong
 
Ceph Day New York: Ceph: one decade in
Ceph Day New York: Ceph: one decade inCeph Day New York: Ceph: one decade in
Ceph Day New York: Ceph: one decade in
 
Orchestrating stateful applications with PKS and Portworx
Orchestrating stateful applications with PKS and PortworxOrchestrating stateful applications with PKS and Portworx
Orchestrating stateful applications with PKS and Portworx
 
Orchestrating Stateful Applications with PKS and Portworx
Orchestrating Stateful Applications with PKS and PortworxOrchestrating Stateful Applications with PKS and Portworx
Orchestrating Stateful Applications with PKS and Portworx
 
Build Your Own PaaS, Just like Red Hat's OpenShift from LinuxCon 2013 New Orl...
Build Your Own PaaS, Just like Red Hat's OpenShift from LinuxCon 2013 New Orl...Build Your Own PaaS, Just like Red Hat's OpenShift from LinuxCon 2013 New Orl...
Build Your Own PaaS, Just like Red Hat's OpenShift from LinuxCon 2013 New Orl...
 
Introduction into Ceph storage for OpenStack
Introduction into Ceph storage for OpenStackIntroduction into Ceph storage for OpenStack
Introduction into Ceph storage for OpenStack
 
London Ceph Day Keynote: Building Tomorrow's Ceph
London Ceph Day Keynote: Building Tomorrow's Ceph London Ceph Day Keynote: Building Tomorrow's Ceph
London Ceph Day Keynote: Building Tomorrow's Ceph
 
Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph
Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph
Ceph Day Santa Clara: Keynote: Building Tomorrow's Ceph
 
Ceph Day NYC: Building Tomorrow's Ceph
Ceph Day NYC: Building Tomorrow's CephCeph Day NYC: Building Tomorrow's Ceph
Ceph Day NYC: Building Tomorrow's Ceph
 
Ceph Day SF 2015 - Keynote
Ceph Day SF 2015 - Keynote Ceph Day SF 2015 - Keynote
Ceph Day SF 2015 - Keynote
 
A Tight Ship: How Containers and SDS Optimize the Enterprise
 A Tight Ship: How Containers and SDS Optimize the Enterprise A Tight Ship: How Containers and SDS Optimize the Enterprise
A Tight Ship: How Containers and SDS Optimize the Enterprise
 
Alluxio 2.0 Deep Dive – Simplifying data access for cloud workloads
Alluxio 2.0 Deep Dive – Simplifying data access for cloud workloadsAlluxio 2.0 Deep Dive – Simplifying data access for cloud workloads
Alluxio 2.0 Deep Dive – Simplifying data access for cloud workloads
 
Accelerate Analytics and ML in the Hybrid Cloud Era
Accelerate Analytics and ML in the Hybrid Cloud EraAccelerate Analytics and ML in the Hybrid Cloud Era
Accelerate Analytics and ML in the Hybrid Cloud Era
 
Transforming Application Delivery with PaaS and Linux Containers
Transforming Application Delivery with PaaS and Linux ContainersTransforming Application Delivery with PaaS and Linux Containers
Transforming Application Delivery with PaaS and Linux Containers
 
Vmware Serengeti - Based on Infochimps Ironfan
Vmware Serengeti - Based on Infochimps IronfanVmware Serengeti - Based on Infochimps Ironfan
Vmware Serengeti - Based on Infochimps Ironfan
 
How @twitterhadoop chose google cloud
How @twitterhadoop chose google cloudHow @twitterhadoop chose google cloud
How @twitterhadoop chose google cloud
 
The Next Generation Datacenter
The Next Generation DatacenterThe Next Generation Datacenter
The Next Generation Datacenter
 
Moving data to the cloud BY CESAR ROJAS from Pivotal
Moving data to the cloud BY CESAR ROJAS from PivotalMoving data to the cloud BY CESAR ROJAS from Pivotal
Moving data to the cloud BY CESAR ROJAS from Pivotal
 

Mehr von John Mark Walker

OSEN SF Meetup - Business of Open Source
OSEN SF Meetup - Business of Open SourceOSEN SF Meetup - Business of Open Source
OSEN SF Meetup - Business of Open SourceJohn Mark Walker
 
Product Development in the Age of Cloud Native
Product Development in the Age of Cloud NativeProduct Development in the Age of Cloud Native
Product Development in the Age of Cloud NativeJohn Mark Walker
 
Hybrid Cloud Management with ManageIQ
Hybrid Cloud Management with ManageIQHybrid Cloud Management with ManageIQ
Hybrid Cloud Management with ManageIQJohn Mark Walker
 
The Secrets to Open Source Innovation
The Secrets to Open Source InnovationThe Secrets to Open Source Innovation
The Secrets to Open Source InnovationJohn Mark Walker
 
Gluster: where weve been - a history
Gluster: where weve been - a historyGluster: where weve been - a history
Gluster: where weve been - a historyJohn Mark Walker
 
Open Source and Cloud - The Two Great Tastes...
Open Source and Cloud - The Two Great Tastes...Open Source and Cloud - The Two Great Tastes...
Open Source and Cloud - The Two Great Tastes...John Mark Walker
 
Intro to Open Cloud Initiative
Intro to Open Cloud InitiativeIntro to Open Cloud Initiative
Intro to Open Cloud InitiativeJohn Mark Walker
 
FOSS vs. Web Services Lightning Talk: Is FOSS Necessary?
FOSS vs. Web Services Lightning Talk: Is FOSS Necessary?FOSS vs. Web Services Lightning Talk: Is FOSS Necessary?
FOSS vs. Web Services Lightning Talk: Is FOSS Necessary?John Mark Walker
 
Building Vibrant Open Source Communities
Building Vibrant Open Source CommunitiesBuilding Vibrant Open Source Communities
Building Vibrant Open Source CommunitiesJohn Mark Walker
 

Mehr von John Mark Walker (12)

OSEN SF Meetup - Business of Open Source
OSEN SF Meetup - Business of Open SourceOSEN SF Meetup - Business of Open Source
OSEN SF Meetup - Business of Open Source
 
Product Development in the Age of Cloud Native
Product Development in the Age of Cloud NativeProduct Development in the Age of Cloud Native
Product Development in the Age of Cloud Native
 
From project to product
From project to productFrom project to product
From project to product
 
Hybrid Cloud Management with ManageIQ
Hybrid Cloud Management with ManageIQHybrid Cloud Management with ManageIQ
Hybrid Cloud Management with ManageIQ
 
The Secrets to Open Source Innovation
The Secrets to Open Source InnovationThe Secrets to Open Source Innovation
The Secrets to Open Source Innovation
 
Gluster: where weve been - a history
Gluster: where weve been - a historyGluster: where weve been - a history
Gluster: where weve been - a history
 
Gluster 3.3 deep dive
Gluster 3.3 deep diveGluster 3.3 deep dive
Gluster 3.3 deep dive
 
Open Source and Cloud - The Two Great Tastes...
Open Source and Cloud - The Two Great Tastes...Open Source and Cloud - The Two Great Tastes...
Open Source and Cloud - The Two Great Tastes...
 
GlusterFS Community Preso
GlusterFS Community PresoGlusterFS Community Preso
GlusterFS Community Preso
 
Intro to Open Cloud Initiative
Intro to Open Cloud InitiativeIntro to Open Cloud Initiative
Intro to Open Cloud Initiative
 
FOSS vs. Web Services Lightning Talk: Is FOSS Necessary?
FOSS vs. Web Services Lightning Talk: Is FOSS Necessary?FOSS vs. Web Services Lightning Talk: Is FOSS Necessary?
FOSS vs. Web Services Lightning Talk: Is FOSS Necessary?
 
Building Vibrant Open Source Communities
Building Vibrant Open Source CommunitiesBuilding Vibrant Open Source Communities
Building Vibrant Open Source Communities
 

Kürzlich hochgeladen

Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your QueriesExploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your QueriesSanjay Willie
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 

Kürzlich hochgeladen (20)

Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your QueriesExploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
Exploring ChatGPT Prompt Hacks To Maximally Optimise Your Queries
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 

The Future of GlusterFS and Gluster.org

  • 1. The Future of GlusterFS and Gluster.org John Mark Walker GlusterFS Community Guy Red Hat, Inc. January 25, 2012
  • 2. The Roots of GlusterFS ● Distributed storage solutions difficult to find ● Decided to write their own ● No filesystem experts – Pro & Con ● Applied lessons from microkernel architecture – GNU Hurd January 26, 2012 The Future of Gluster.org - John Mark Walker
  • 3. The Roots of GlusterFS ● All storage solutions were either ● Too expensive. or... ● Not scalable, or… ● Single purpose, or… ● Don’t support legacy apps, or… ● Don't support new apps, or... ● Do some combo of the above, but not very well
  • 4. The Roots of GlusterFS ● The challenge: ● Create a storage system that was… – Scalable – Seamlessly integrated in the data center – Future-proof ● The solution: GlusterFS ● Scalable, with DHT ● POSIX-compliant ● Stackable ● User-space
  • 5. GlusterFS Client Architecture ● Creating a file system in user space ● Utilizes fuse module – Kernel goes through fuse, which hands off to glusterd glusterd Applications Linux kernel Fuse Ext4 … …
  • 6. No Centralized Metadata Client A Client B Client C Server X Server Y Server Z Extended Attr. Extended Attr. Extended Attr. Files Files Files
  • 7. What is a Translator? ● Add/remove layers FUSE Interface Layer ● Reorder layers Performance Layer ● Move layers between Distribution Layer client and server Replication Layer ● Implement new layers Protocol Layer ● e.g. encryption ● Replace old layers Local Filesystem Later ● e.g. replication
  • 8. Some Features ● Distributed, replicated and/or striped volumes ● Global namespace ● High availability ● Geo-replication ● Rebalancing ● Remove or replace bricks ● Self healing ● volume profile and top metrics
  • 9. No one ever expects the Red Hat acquisition
  • 10. Red Hat Invests in GlusterFS ● Unstructured data volume to grow 44x by 2020 ● Cloud and virtualization are driving scale-out storage growth ● Scale-out storage shipments to exceed 63,000 PB by 2015 (74% CAGR) ● 40% of core cloud spend related to storage ● GlusterFS-based solutions up to 50% less than other storage systems
  • 11. Red Hat Invests in GlusterFS ● GlusterFS adds to the JBoss Red Hat stack ● Complements other RHEL Red Hat offerings ● Many integration points Bare RHEV Clouds ● More engineers Metal hacking on GlusterFS GlusterFS Unified Storage than ever before
  • 12. Red Hat Invests in GlusterFS ● Acceleration of community investment ● GlusterFS needs to be “bigger than Red Hat” ● Transformation of GlusterFS from product to project – From “open core” to upstream ● More resources for engineering and community outreach ● Red Hat's success rests on economies of scale – Critical mass of users and developers
  • 13. Join a Winning Team “Join me, and together, we can rule the galaxy...” ● We're hiring hackers and engineers ● Looking for community collaborators ● ISVs, students, IT professionals, fans, et al.
  • 15. The Gluster Community Global adoption ● 300,000+ downloads ● ~35,000 /month ● >300% increase Y/Y ● 1000+ deployments ● 45 countries ● 2,000+ registered users ● Mailing lists, Forums, etc.
  • 16. The Gluster Community ● Why are we changing? ● Only 1 non-Red Hat core contributor – There were 2, but he acquired us ● Want to be the software standard for distributed storage ● Want to be more inclusive, more community-driven Goal: create global ecosystem that supports ISVs, service providers and more
  • 17. Towards “Real” Open Source ● GlusterFS, prior to acquisition ● “Open Core” ● Tied directly to Gluster products – No differentiation ● Very little outside collaboration ● Contributors had to assign copyright to Gluster – Discouraged would-be contributors
  • 18. Towards “Real” Open Source “Open Core” ● All engineering controlled by project/product sponsor ● No innovation outside of Commercial Product core engineering team ● All open source features also in commercial product ● Many features in Open Source Commercial product not in Code open source code
  • 19. Towards “Real” Open Source “Real” Open Source ● Many points of collaboration and innovation in open source project Open Source Code ● Engineering team from multiple sources ● Project and product do not completely overlap ● Commercial products are Commercial Products hardened, more secure and thoroughly tested
  • 20. Towards “Real” Open Source “Real” Open Source ● Enables more innovation on the fringes Fedora Linux ● Engineering team from multiple sources ● Open source project is “upstream” from commercial product ● “Downstream” products are RHEL hardened, more secure and thoroughly tested
  • 21. Towards “Real” Open Source “Real” Open Source ● Enables more innovation on the fringes GlusterFS ● Engineering team from multiple sources ● Open source project is “upstream” from commercial product ● “Downstream” products are Red Hat Storage hardened, more secure and thoroughly tested
  • 23. What's New in GlusterFS 3.3 ● New features ● Unified File & Object access ● Hadoop / HDFS compatibility ● New Volume Type ● Replicated + striped (+ distributed) volumes ● Enhancements to Distributed volumes (DHT translator) ● Rebalance can migrate open files ● Remove-brick can migrate data to remaining bricks ● Enhancements to Replicated volumes (AFR translator) ● Change replica count on an active volume, add replication to distribute-only volumes ● Granular locking – Much faster self-healing for large files ● Proactive self-heal process starts without FS stat ● Round-trip reduction for lower latency ● Quorum enforcement - avoid split brain scenarios GlusterFS 3.3 ETA in Q2/Q3 2012
  • 24. File and Object Storage ● Traditional SAN / NAS support either file or block storage ● New storage methodologies implement RESTful APIs over HTTP ● Demand for unifying the storage infrastructure increasing ● Treats files as objects and volumes as buckets ● Available now in 3.3 betas ● Soon to be backported to 3.2.x ● Contributing to OpenStack project ● Re-factored Swift API
  • 25. Technology Integrations GlusterFS used as VM storage system Mobile Apps. Web Clients. Enterprise Software Ecosystem ● Pause and re-start VM’s, even on another API Layer hypervisor … ● HA and DR for VM’s Compute ● Faster VM deployment ● V-motion –like capability Unified File & Object Storage Shared storage ISOs and appliances ● oVirt / RHEV ● CloudStack ● OpenStack OpenStack Imaging Services Goal: The standard for cloud storage
  • 26. HDFS/Hadoop Compatibility ● HDFS compatibility library ● Simultaneous file and object access within Hadoop ● Benefits ● Legacy app access to MapReduce applications ● Enables data storage consolidation ● Simplify and unify storage deployments ● Provide users with file level access to data ● Enable legacy applications to access data via NFS ● Analytic apps can access data without modification
  • 27. The Gluster Community ● What is changing? ● HekaFS / CloudFS being folded into Gluster project – HekaFS == GlusterFS + multi-tenancy and SSL for auth and data encryption – HekaFS.org – ETA ~9 months
  • 29. GlusterFS Advisory Board ● Advisory board ● Consists of industry and community leaders from Facebook, Citrix, Fedora, and OpenStack – Richard Wareing, Storage Engineer, Facebook – Jeff Darcy, Filesystem Engineer, Red Hat; Founder, HekaFS Project – AB Periasamy, Co-Founder, GlusterFS project – Ewan Mellor, Xen Engineer, Citrix; Member, OpenStack project – David Nalley, CloudStack Community Mgr; Fedora Advisory Board – Louis Zuckerman, Sr. System Administrator, Picture Marketing – Joe Julian, Sr. System Administrator, Ed Wyse Beauty Products – Greg DeKoenigsberg, Community VP, Eucalyptus; co-founder, Fedora – John Mark Walker, Gluster.org Community Guy (Chair)
  • 30. Gluster.org Web Site ● Services for users and developers ● Developer section with comprehensive docs ● Collaborative project hosting ● Continuing development of end user documentation and interactive tools ● Published roadmaps ● Transparent feature development
  • 31. GlusterFS Downloads ● Where's the code? ● GlusterFS 3.3 – Simultaneous file + object – HDFS compatibility – Improved self-healing + VM hosting ● Granular locking – Beta 3 due Feb/Mar 2012 – http://download.gluster.org/pub/gluster/glusterfs
  • 32. Gluster.org Services ● Gluster.org ● Portal into all things GlusterFS ● Community.gluster.org ● Self-support site; Q&A; HOWTOs; tutorials ● Patch review, CI ● review.gluster.com ● #gluster ● IRC channel on Freenode
  • 33. Development Process ● Source code ● Hosted at github.com/gluster ● Bugs and Feature Requests ● Bugzilla.redhat.com – select GlusterFS from menu ● Patches ● Submit via Gerritt at review.gluster.com ● See Development Work Flow doc: ● gluster.org/community/documentation/index.php/Development_Work_Flow
  • 34. Thank You ● GlusterFS contacts ● Gluster.org/interact/mailinglists ● @RedHatStorage & @GlusterOrg ● #gluster on Freenode ● My contact info ● johnmark@redhat.com ● Twitter & identi.ca: @johnmark