SlideShare ist ein Scribd-Unternehmen logo
1 von 12
Downloaden Sie, um offline zu lesen
Island: Local Storage plugin for Cinder
                        在这里写上你的标题
                 Presented at OpenStack Summit, San Diego


                                  副标题文字副标题文字



                                  Hui Cheng/RongZhe Zhu
                           freedomhui@gmail.com | zrzhit@gmail.com



                                                作者名字 / 日
                                                2012-09-22
                                                期
Content


 1. Why
     00       写上你的文字你的文字

 2. Design
     01

 3. Snapshot
     02

 4. Discussions
     03

        04

        05




https://blueprints.launchpad.net/cinder/+spec/local-storage-volume
Why

            AWS EBS
              写上你的文字你的文字
                                                     Island
       00
Network bottleneck       High durability
       01
 Re-mirroring storm                    =
                      Incremental snapshot

      02                                      The high performance of
 Low performance      Snapshots store in S3
                                              12 disks RAID10
       03
                                              Incremental snapshot
     Our Environment
      04
                                              Snapshots store in swift
       05     写上你的文字你的文字

                      Every x86 server have
 1000Mb Network
                      12 disks
Design – Island in Cinder                               KEY
                                                                         REST
                       Users
                                                                         AMQP
         Nova API              Volume API                                SQL
                                                                         POSIX


                                            =
                Nova             cinder     Cinder
                RPC               RPC         DB
  Nova
   DB
                                                                                Swift API


           Compute             Volume                Snapshot in Swift
           Manager             Manager

         Host
                                  Island

           KVM
                                                                                Swift Storage
           Guest
                               images

Island Design Idea= Local Storage + QCOW2 Image + Dependent Snapshot + Swift
Design – Incremental Snapshot in Swift


     00
    VM           VM          VM
                                               Snapshots are saved incrementally.
                                               Each subsequent snapshot will only
     01                                        save modified data block to swift. A

     02
 image           image       image
                                         =
                                        Host   table of contents points to the latest
                                               versions of each data block. Each
                                               data block have a reference counter.
     03

     04

     05 snap                     snap                snap
             1                     2                   3

                                                                       Swift


1        2        3      4              1      2         2     3
create_volume
                                               delete_volume
     Design – Island Architecture              create_volume_from_snapshot
                                               create_snapshot
                                               delete_snapshot
create_vdisk 写上你的文字你的文字                        attach_volume
        00
delete_vdisk                                   detach_volume
create_snapshot    IslandDriver                ……
        01
delete_snapshot
read_vdisk
        02
write_vdisk
                           =              get_snapshot
                                          put_snapshot
get_update_blocks          Snapshot
        03
read_vdisk_meta
                                          delete_snapshot
                                                             get_object
      04                                                     put_object
                                                             delete_object
      05                                                     set_object_meta
                Vdisk                    ObjectStorage
                                                             get_object_meta



       QCOW2      VDMK           Swift           NFS        Glusterfs
Design - Process


     00
     Create volume
     Do nothing
     01
     Create volume from snapshot
     02
     Do nothing
                                 =
     03
     Delete volume
     Do nothing
     04
     Delete snapshot
      05
     Island delete snapshot in swift
Design - Process          Create the
                         image of the
                           volume

       写上你的文字你的文字
Attach volume
       00

     01 volume
       old                 Check the      new volume form snapshot

     02                      =
                            volume


     03
     Download the last                              Download
       snapshot and                               snapshot and
     04wirte it to the    new volume              wirte it to the
           image                                      image
     05




                            Update
                         volume[‘host’]
Design - Process


 Create snapshot
     00                          Detach volume

       01
                             Create a new snapshot DB item, set
 Create snapshot in image   snapshot[‘volume_id’] as the volume
       02

       03
   Upload incremental            Create snapshot in image
    snapshot to swift
       04

       05
  Delete old snapshot in    Upload incremental snapshot to swfit
          image


                              Delete the image of the volume
Snapshot – How to Create/Read Snapshot in Image

Create internal snapshot in image
Because there写上你的文字你的文字 command to create internal snapshot
              is not a qemu monitor
        00
in image when the image was attached. So, we add a new qemu monitor
command to create point-in-time snapshot in image.
        01
Accessing QCOW2 image in python
         02                      =
QCOW2 image format use 2-level Lookup Table
http://people.gnome.org/~markmc/qcow-image-format.html
         03
We implement qcow2 python library to read/write qcow2 image data and
metadata.
         04

       05
Snapshot – How to Implement Incremental Snapshot


            00    写上你的文字你的文字                 Compare current snapshot pointer
                                            table and previous snapshot pointer
            01                                         table in image
We need to find out updated data blocks
between two02snapshots.               =
                                                 Find updated data blocks
            03

            04
                                              Upload updated data blocks to
            05                                        objectstorage

Future: modify qcow2 driver in qemu, mark
updated data blocks in pointer table of      Upload snapshot pointer table to
snapshot in image.                                    objectstorage
Discussions


  The 00 写上你的文字你的文字
       storage capacity of a x86 server is limit, but if many
  instances in a server and many volume are attached to
       01
  these instances, it will be a issue.

  The 02
                                   =
      attach volume action is slower than other plugin.
      03
  Optimization?
      04
  Make Island as a independent system?
      05

Weitere ähnliche Inhalte

Was ist angesagt?

Dell openstack boston meetup dell crowbar and open stack
Dell openstack boston meetup   dell crowbar and open stackDell openstack boston meetup   dell crowbar and open stack
Dell openstack boston meetup dell crowbar and open stackDellCloudEdge
 
Ceph, Xen, and CloudStack: Semper Melior-XPUS13 McGarry
Ceph, Xen, and CloudStack: Semper Melior-XPUS13 McGarryCeph, Xen, and CloudStack: Semper Melior-XPUS13 McGarry
Ceph, Xen, and CloudStack: Semper Melior-XPUS13 McGarryThe Linux Foundation
 
Presentation cloud orchestration
Presentation   cloud orchestrationPresentation   cloud orchestration
Presentation cloud orchestrationxKinAnx
 
Automating Your CloudStack Cloud with Puppet
Automating Your CloudStack Cloud with PuppetAutomating Your CloudStack Cloud with Puppet
Automating Your CloudStack Cloud with Puppetbuildacloud
 
Open stack in sina
Open stack in sinaOpen stack in sina
Open stack in sinaHui Cheng
 
Disaster Recovery in the Cloud -- A Failover Testing Case Study
Disaster Recovery in the Cloud -- A Failover Testing Case StudyDisaster Recovery in the Cloud -- A Failover Testing Case Study
Disaster Recovery in the Cloud -- A Failover Testing Case StudyOnline Tech
 
Top 5 Features of Ice Cream Sandwich with Mono for Android 4.0
Top 5 Features of Ice Cream Sandwich with Mono for Android 4.0Top 5 Features of Ice Cream Sandwich with Mono for Android 4.0
Top 5 Features of Ice Cream Sandwich with Mono for Android 4.0Xamarin
 
Servers fail, who cares?
Servers fail, who cares? Servers fail, who cares?
Servers fail, who cares? greggulrich
 
Netflix Velocity Conference 2011
Netflix Velocity Conference 2011Netflix Velocity Conference 2011
Netflix Velocity Conference 2011Adrian Cockcroft
 
Introduction to 360 Video
Introduction to  360 VideoIntroduction to  360 Video
Introduction to 360 VideoSeung-Bum Lee
 
CloudStack for Java User Group
CloudStack for Java User GroupCloudStack for Java User Group
CloudStack for Java User GroupSebastien Goasguen
 
3 Networking CloudStack Developer Day
3  Networking CloudStack Developer Day 3  Networking CloudStack Developer Day
3 Networking CloudStack Developer Day Kimihiko Kitase
 
Hacking apache cloud stack
Hacking apache cloud stackHacking apache cloud stack
Hacking apache cloud stackMurali Reddy
 
Cassandra and Docker Lessons Learned
Cassandra and Docker Lessons LearnedCassandra and Docker Lessons Learned
Cassandra and Docker Lessons LearnedDataStax Academy
 

Was ist angesagt? (20)

Dell openstack boston meetup dell crowbar and open stack
Dell openstack boston meetup   dell crowbar and open stackDell openstack boston meetup   dell crowbar and open stack
Dell openstack boston meetup dell crowbar and open stack
 
Ceph, Xen, and CloudStack: Semper Melior-XPUS13 McGarry
Ceph, Xen, and CloudStack: Semper Melior-XPUS13 McGarryCeph, Xen, and CloudStack: Semper Melior-XPUS13 McGarry
Ceph, Xen, and CloudStack: Semper Melior-XPUS13 McGarry
 
DevCloud and CloudMonkey
DevCloud and CloudMonkeyDevCloud and CloudMonkey
DevCloud and CloudMonkey
 
Apache CloudStack AlpesJUG
Apache CloudStack AlpesJUGApache CloudStack AlpesJUG
Apache CloudStack AlpesJUG
 
Presentation cloud orchestration
Presentation   cloud orchestrationPresentation   cloud orchestration
Presentation cloud orchestration
 
Intro to Cloudstack
Intro to CloudstackIntro to Cloudstack
Intro to Cloudstack
 
Automating Your CloudStack Cloud with Puppet
Automating Your CloudStack Cloud with PuppetAutomating Your CloudStack Cloud with Puppet
Automating Your CloudStack Cloud with Puppet
 
Open stack in sina
Open stack in sinaOpen stack in sina
Open stack in sina
 
Disaster Recovery in the Cloud -- A Failover Testing Case Study
Disaster Recovery in the Cloud -- A Failover Testing Case StudyDisaster Recovery in the Cloud -- A Failover Testing Case Study
Disaster Recovery in the Cloud -- A Failover Testing Case Study
 
Aplura virtualization slides
Aplura virtualization slidesAplura virtualization slides
Aplura virtualization slides
 
Top 5 Features of Ice Cream Sandwich with Mono for Android 4.0
Top 5 Features of Ice Cream Sandwich with Mono for Android 4.0Top 5 Features of Ice Cream Sandwich with Mono for Android 4.0
Top 5 Features of Ice Cream Sandwich with Mono for Android 4.0
 
Servers fail, who cares?
Servers fail, who cares? Servers fail, who cares?
Servers fail, who cares?
 
Netflix Velocity Conference 2011
Netflix Velocity Conference 2011Netflix Velocity Conference 2011
Netflix Velocity Conference 2011
 
Introduction to 360 Video
Introduction to  360 VideoIntroduction to  360 Video
Introduction to 360 Video
 
CloudStack for Java User Group
CloudStack for Java User GroupCloudStack for Java User Group
CloudStack for Java User Group
 
3 Networking CloudStack Developer Day
3  Networking CloudStack Developer Day 3  Networking CloudStack Developer Day
3 Networking CloudStack Developer Day
 
Cassandra via-docker
Cassandra via-dockerCassandra via-docker
Cassandra via-docker
 
Hacking apache cloud stack
Hacking apache cloud stackHacking apache cloud stack
Hacking apache cloud stack
 
CloudStack technical overview
CloudStack technical overviewCloudStack technical overview
CloudStack technical overview
 
Cassandra and Docker Lessons Learned
Cassandra and Docker Lessons LearnedCassandra and Docker Lessons Learned
Cassandra and Docker Lessons Learned
 

Ähnlich wie Island: Local Storage Volume for Cinder

DevOps in a Public OpenStack Cloud - Hui Cheng
DevOps in a Public OpenStack Cloud - Hui ChengDevOps in a Public OpenStack Cloud - Hui Cheng
DevOps in a Public OpenStack Cloud - Hui ChengHui Cheng
 
SDEC2011 Big engineer vs small entreprenuer
SDEC2011 Big engineer vs small entreprenuerSDEC2011 Big engineer vs small entreprenuer
SDEC2011 Big engineer vs small entreprenuerKorea Sdec
 
Mirantis Folsom Meetup Intro
Mirantis Folsom Meetup IntroMirantis Folsom Meetup Intro
Mirantis Folsom Meetup IntroMirantis
 
Delivering Secure OpenStack IaaS for SaaS Products
Delivering Secure OpenStack IaaS for SaaS ProductsDelivering Secure OpenStack IaaS for SaaS Products
Delivering Secure OpenStack IaaS for SaaS ProductsCloudPassage
 
Delivering Secure OpenStack IaaS for SaaS Products - OpenStack 2012.pptx
Delivering Secure OpenStack IaaS for SaaS Products - OpenStack 2012.pptxDelivering Secure OpenStack IaaS for SaaS Products - OpenStack 2012.pptx
Delivering Secure OpenStack IaaS for SaaS Products - OpenStack 2012.pptxOpenStack Foundation
 
Lessons from Building OpenStack Public Cloud
Lessons from Building OpenStack Public CloudLessons from Building OpenStack Public Cloud
Lessons from Building OpenStack Public CloudHui Cheng
 
Swift Install Workshop - OpenStack Conference Spring 2012
Swift Install Workshop - OpenStack Conference Spring 2012Swift Install Workshop - OpenStack Conference Spring 2012
Swift Install Workshop - OpenStack Conference Spring 2012Joe Arnold
 
NTT SIC marketplace slide deck at Tokyo Summit
NTT SIC marketplace slide deck at Tokyo SummitNTT SIC marketplace slide deck at Tokyo Summit
NTT SIC marketplace slide deck at Tokyo SummitToshikazu Ichikawa
 
Optimising Productivity with AWS Developer Tools
Optimising Productivity with AWS Developer ToolsOptimising Productivity with AWS Developer Tools
Optimising Productivity with AWS Developer ToolsAmazon Web Services
 
High Performance Serverless Functions in Scala
High Performance Serverless Functions in ScalaHigh Performance Serverless Functions in Scala
High Performance Serverless Functions in ScalaJason Swartz
 
Scylla on Kubernetes: Introducing the Scylla Operator
Scylla on Kubernetes: Introducing the Scylla OperatorScylla on Kubernetes: Introducing the Scylla Operator
Scylla on Kubernetes: Introducing the Scylla OperatorScyllaDB
 
Zun project update (boston summit)
Zun project update (boston summit)Zun project update (boston summit)
Zun project update (boston summit)hongbin034
 
CloudStack vs OpenStack vs Eucalyptus: IaaS Private Cloud Brief Comparison
CloudStack vs OpenStack vs Eucalyptus: IaaS Private Cloud Brief ComparisonCloudStack vs OpenStack vs Eucalyptus: IaaS Private Cloud Brief Comparison
CloudStack vs OpenStack vs Eucalyptus: IaaS Private Cloud Brief Comparisonbizalgo
 
Running serverless at scale
Running serverless at scaleRunning serverless at scale
Running serverless at scaleYan Cui
 
VSTS Release Pipelines with Kubernetes
VSTS Release Pipelines with KubernetesVSTS Release Pipelines with Kubernetes
VSTS Release Pipelines with KubernetesMarc Müller
 
Snapshot clone-boot-presentation-final
Snapshot clone-boot-presentation-finalSnapshot clone-boot-presentation-final
Snapshot clone-boot-presentation-finalOpen Stack
 
Siebel Server Cloning available in 8.1.1.9 / 8.2.2.2
Siebel Server Cloning available in 8.1.1.9 / 8.2.2.2Siebel Server Cloning available in 8.1.1.9 / 8.2.2.2
Siebel Server Cloning available in 8.1.1.9 / 8.2.2.2Jeroen Burgers
 
Building Fault Tolerant, Highly Available and Secured Architectures - Simon E...
Building Fault Tolerant, Highly Available and Secured Architectures - Simon E...Building Fault Tolerant, Highly Available and Secured Architectures - Simon E...
Building Fault Tolerant, Highly Available and Secured Architectures - Simon E...Amazon Web Services
 
Serverless Multi Region Cache Replication
Serverless Multi Region Cache ReplicationServerless Multi Region Cache Replication
Serverless Multi Region Cache ReplicationSanghyun Lee
 

Ähnlich wie Island: Local Storage Volume for Cinder (20)

DevOps in a Public OpenStack Cloud - Hui Cheng
DevOps in a Public OpenStack Cloud - Hui ChengDevOps in a Public OpenStack Cloud - Hui Cheng
DevOps in a Public OpenStack Cloud - Hui Cheng
 
SDEC2011 Big engineer vs small entreprenuer
SDEC2011 Big engineer vs small entreprenuerSDEC2011 Big engineer vs small entreprenuer
SDEC2011 Big engineer vs small entreprenuer
 
Mirantis Folsom Meetup Intro
Mirantis Folsom Meetup IntroMirantis Folsom Meetup Intro
Mirantis Folsom Meetup Intro
 
Delivering Secure OpenStack IaaS for SaaS Products
Delivering Secure OpenStack IaaS for SaaS ProductsDelivering Secure OpenStack IaaS for SaaS Products
Delivering Secure OpenStack IaaS for SaaS Products
 
Delivering Secure OpenStack IaaS for SaaS Products - OpenStack 2012.pptx
Delivering Secure OpenStack IaaS for SaaS Products - OpenStack 2012.pptxDelivering Secure OpenStack IaaS for SaaS Products - OpenStack 2012.pptx
Delivering Secure OpenStack IaaS for SaaS Products - OpenStack 2012.pptx
 
Lessons from Building OpenStack Public Cloud
Lessons from Building OpenStack Public CloudLessons from Building OpenStack Public Cloud
Lessons from Building OpenStack Public Cloud
 
Swift Install Workshop - OpenStack Conference Spring 2012
Swift Install Workshop - OpenStack Conference Spring 2012Swift Install Workshop - OpenStack Conference Spring 2012
Swift Install Workshop - OpenStack Conference Spring 2012
 
NTT SIC marketplace slide deck at Tokyo Summit
NTT SIC marketplace slide deck at Tokyo SummitNTT SIC marketplace slide deck at Tokyo Summit
NTT SIC marketplace slide deck at Tokyo Summit
 
Optimising Productivity with AWS Developer Tools
Optimising Productivity with AWS Developer ToolsOptimising Productivity with AWS Developer Tools
Optimising Productivity with AWS Developer Tools
 
OpenStack Report
OpenStack ReportOpenStack Report
OpenStack Report
 
High Performance Serverless Functions in Scala
High Performance Serverless Functions in ScalaHigh Performance Serverless Functions in Scala
High Performance Serverless Functions in Scala
 
Scylla on Kubernetes: Introducing the Scylla Operator
Scylla on Kubernetes: Introducing the Scylla OperatorScylla on Kubernetes: Introducing the Scylla Operator
Scylla on Kubernetes: Introducing the Scylla Operator
 
Zun project update (boston summit)
Zun project update (boston summit)Zun project update (boston summit)
Zun project update (boston summit)
 
CloudStack vs OpenStack vs Eucalyptus: IaaS Private Cloud Brief Comparison
CloudStack vs OpenStack vs Eucalyptus: IaaS Private Cloud Brief ComparisonCloudStack vs OpenStack vs Eucalyptus: IaaS Private Cloud Brief Comparison
CloudStack vs OpenStack vs Eucalyptus: IaaS Private Cloud Brief Comparison
 
Running serverless at scale
Running serverless at scaleRunning serverless at scale
Running serverless at scale
 
VSTS Release Pipelines with Kubernetes
VSTS Release Pipelines with KubernetesVSTS Release Pipelines with Kubernetes
VSTS Release Pipelines with Kubernetes
 
Snapshot clone-boot-presentation-final
Snapshot clone-boot-presentation-finalSnapshot clone-boot-presentation-final
Snapshot clone-boot-presentation-final
 
Siebel Server Cloning available in 8.1.1.9 / 8.2.2.2
Siebel Server Cloning available in 8.1.1.9 / 8.2.2.2Siebel Server Cloning available in 8.1.1.9 / 8.2.2.2
Siebel Server Cloning available in 8.1.1.9 / 8.2.2.2
 
Building Fault Tolerant, Highly Available and Secured Architectures - Simon E...
Building Fault Tolerant, Highly Available and Secured Architectures - Simon E...Building Fault Tolerant, Highly Available and Secured Architectures - Simon E...
Building Fault Tolerant, Highly Available and Secured Architectures - Simon E...
 
Serverless Multi Region Cache Replication
Serverless Multi Region Cache ReplicationServerless Multi Region Cache Replication
Serverless Multi Region Cache Replication
 

Mehr von Hui Cheng

OpenStack in China
OpenStack in ChinaOpenStack in China
OpenStack in ChinaHui Cheng
 
OpenStack Foundation Updates
OpenStack Foundation UpdatesOpenStack Foundation Updates
OpenStack Foundation UpdatesHui Cheng
 
COSUG updates
COSUG updatesCOSUG updates
COSUG updatesHui Cheng
 
Cinder intro@open stack china-tour-shenzhen
Cinder intro@open stack china-tour-shenzhenCinder intro@open stack china-tour-shenzhen
Cinder intro@open stack china-tour-shenzhenHui Cheng
 
Open stack with_openflowsdn-torii
Open stack with_openflowsdn-toriiOpen stack with_openflowsdn-torii
Open stack with_openflowsdn-toriiHui Cheng
 
Open Source Software & Open Source Hardware
Open Source Software & Open Source HardwareOpen Source Software & Open Source Hardware
Open Source Software & Open Source HardwareHui Cheng
 
OpenStack Network Planning
OpenStack Network PlanningOpenStack Network Planning
OpenStack Network PlanningHui Cheng
 
Asia pacopenstack joe-draft 2012-08-08
Asia pacopenstack joe-draft 2012-08-08Asia pacopenstack joe-draft 2012-08-08
Asia pacopenstack joe-draft 2012-08-08Hui Cheng
 
The Application of OpenStack in 360buy, by Bin He
The Application of OpenStack in 360buy, by Bin HeThe Application of OpenStack in 360buy, by Bin He
The Application of OpenStack in 360buy, by Bin HeHui Cheng
 
Distributed Block-level Storage Management for OpenStack, by Danile lee
Distributed Block-level Storage Management for OpenStack, by Danile leeDistributed Block-level Storage Management for OpenStack, by Danile lee
Distributed Block-level Storage Management for OpenStack, by Danile leeHui Cheng
 
Swift Architecture and Practice, by Alex Yang
Swift Architecture and Practice, by Alex YangSwift Architecture and Practice, by Alex Yang
Swift Architecture and Practice, by Alex YangHui Cheng
 
Cloud Based VDI with OpenStack, by Shifen Yang
Cloud Based VDI with OpenStack, by Shifen YangCloud Based VDI with OpenStack, by Shifen Yang
Cloud Based VDI with OpenStack, by Shifen YangHui Cheng
 
Openstorage with OpenStack, by Bradley
Openstorage with OpenStack, by BradleyOpenstorage with OpenStack, by Bradley
Openstorage with OpenStack, by BradleyHui Cheng
 
MAKING MONEY from openstack
MAKING MONEY from openstackMAKING MONEY from openstack
MAKING MONEY from openstackHui Cheng
 
KT ucloud storage, by Jaesuk Ahn
KT ucloud storage, by Jaesuk AhnKT ucloud storage, by Jaesuk Ahn
KT ucloud storage, by Jaesuk AhnHui Cheng
 
Dell OpenStack Powered Cloud Solution and Case Sharing
Dell OpenStack Powered Cloud Solution and Case SharingDell OpenStack Powered Cloud Solution and Case Sharing
Dell OpenStack Powered Cloud Solution and Case SharingHui Cheng
 
Network Service in OpenStack Cloud, by Yaohui Jin
Network Service in OpenStack Cloud, by Yaohui JinNetwork Service in OpenStack Cloud, by Yaohui Jin
Network Service in OpenStack Cloud, by Yaohui JinHui Cheng
 
China OpenStack Community Status
China OpenStack Community StatusChina OpenStack Community Status
China OpenStack Community StatusHui Cheng
 
OpenStack Chances and Chanlleges_OSAC_by_ChengHui
OpenStack Chances and Chanlleges_OSAC_by_ChengHuiOpenStack Chances and Chanlleges_OSAC_by_ChengHui
OpenStack Chances and Chanlleges_OSAC_by_ChengHuiHui Cheng
 
Integrating OpenStack To Existing Infrastructure
Integrating OpenStack To Existing InfrastructureIntegrating OpenStack To Existing Infrastructure
Integrating OpenStack To Existing InfrastructureHui Cheng
 

Mehr von Hui Cheng (20)

OpenStack in China
OpenStack in ChinaOpenStack in China
OpenStack in China
 
OpenStack Foundation Updates
OpenStack Foundation UpdatesOpenStack Foundation Updates
OpenStack Foundation Updates
 
COSUG updates
COSUG updatesCOSUG updates
COSUG updates
 
Cinder intro@open stack china-tour-shenzhen
Cinder intro@open stack china-tour-shenzhenCinder intro@open stack china-tour-shenzhen
Cinder intro@open stack china-tour-shenzhen
 
Open stack with_openflowsdn-torii
Open stack with_openflowsdn-toriiOpen stack with_openflowsdn-torii
Open stack with_openflowsdn-torii
 
Open Source Software & Open Source Hardware
Open Source Software & Open Source HardwareOpen Source Software & Open Source Hardware
Open Source Software & Open Source Hardware
 
OpenStack Network Planning
OpenStack Network PlanningOpenStack Network Planning
OpenStack Network Planning
 
Asia pacopenstack joe-draft 2012-08-08
Asia pacopenstack joe-draft 2012-08-08Asia pacopenstack joe-draft 2012-08-08
Asia pacopenstack joe-draft 2012-08-08
 
The Application of OpenStack in 360buy, by Bin He
The Application of OpenStack in 360buy, by Bin HeThe Application of OpenStack in 360buy, by Bin He
The Application of OpenStack in 360buy, by Bin He
 
Distributed Block-level Storage Management for OpenStack, by Danile lee
Distributed Block-level Storage Management for OpenStack, by Danile leeDistributed Block-level Storage Management for OpenStack, by Danile lee
Distributed Block-level Storage Management for OpenStack, by Danile lee
 
Swift Architecture and Practice, by Alex Yang
Swift Architecture and Practice, by Alex YangSwift Architecture and Practice, by Alex Yang
Swift Architecture and Practice, by Alex Yang
 
Cloud Based VDI with OpenStack, by Shifen Yang
Cloud Based VDI with OpenStack, by Shifen YangCloud Based VDI with OpenStack, by Shifen Yang
Cloud Based VDI with OpenStack, by Shifen Yang
 
Openstorage with OpenStack, by Bradley
Openstorage with OpenStack, by BradleyOpenstorage with OpenStack, by Bradley
Openstorage with OpenStack, by Bradley
 
MAKING MONEY from openstack
MAKING MONEY from openstackMAKING MONEY from openstack
MAKING MONEY from openstack
 
KT ucloud storage, by Jaesuk Ahn
KT ucloud storage, by Jaesuk AhnKT ucloud storage, by Jaesuk Ahn
KT ucloud storage, by Jaesuk Ahn
 
Dell OpenStack Powered Cloud Solution and Case Sharing
Dell OpenStack Powered Cloud Solution and Case SharingDell OpenStack Powered Cloud Solution and Case Sharing
Dell OpenStack Powered Cloud Solution and Case Sharing
 
Network Service in OpenStack Cloud, by Yaohui Jin
Network Service in OpenStack Cloud, by Yaohui JinNetwork Service in OpenStack Cloud, by Yaohui Jin
Network Service in OpenStack Cloud, by Yaohui Jin
 
China OpenStack Community Status
China OpenStack Community StatusChina OpenStack Community Status
China OpenStack Community Status
 
OpenStack Chances and Chanlleges_OSAC_by_ChengHui
OpenStack Chances and Chanlleges_OSAC_by_ChengHuiOpenStack Chances and Chanlleges_OSAC_by_ChengHui
OpenStack Chances and Chanlleges_OSAC_by_ChengHui
 
Integrating OpenStack To Existing Infrastructure
Integrating OpenStack To Existing InfrastructureIntegrating OpenStack To Existing Infrastructure
Integrating OpenStack To Existing Infrastructure
 

Kürzlich hochgeladen

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 

Kürzlich hochgeladen (20)

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 

Island: Local Storage Volume for Cinder

  • 1. Island: Local Storage plugin for Cinder 在这里写上你的标题 Presented at OpenStack Summit, San Diego 副标题文字副标题文字 Hui Cheng/RongZhe Zhu freedomhui@gmail.com | zrzhit@gmail.com 作者名字 / 日 2012-09-22 期
  • 2. Content 1. Why 00 写上你的文字你的文字 2. Design 01 3. Snapshot 02 4. Discussions 03 04 05 https://blueprints.launchpad.net/cinder/+spec/local-storage-volume
  • 3. Why AWS EBS 写上你的文字你的文字 Island 00 Network bottleneck High durability 01 Re-mirroring storm = Incremental snapshot 02 The high performance of Low performance Snapshots store in S3 12 disks RAID10 03 Incremental snapshot Our Environment 04 Snapshots store in swift 05 写上你的文字你的文字 Every x86 server have 1000Mb Network 12 disks
  • 4. Design – Island in Cinder KEY REST Users AMQP Nova API Volume API SQL POSIX = Nova cinder Cinder RPC RPC DB Nova DB Swift API Compute Volume Snapshot in Swift Manager Manager Host Island KVM Swift Storage Guest images Island Design Idea= Local Storage + QCOW2 Image + Dependent Snapshot + Swift
  • 5. Design – Incremental Snapshot in Swift 00 VM VM VM Snapshots are saved incrementally. Each subsequent snapshot will only 01 save modified data block to swift. A 02 image image image = Host table of contents points to the latest versions of each data block. Each data block have a reference counter. 03 04 05 snap snap snap 1 2 3 Swift 1 2 3 4 1 2 2 3
  • 6. create_volume delete_volume Design – Island Architecture create_volume_from_snapshot create_snapshot delete_snapshot create_vdisk 写上你的文字你的文字 attach_volume 00 delete_vdisk detach_volume create_snapshot IslandDriver …… 01 delete_snapshot read_vdisk 02 write_vdisk = get_snapshot put_snapshot get_update_blocks Snapshot 03 read_vdisk_meta delete_snapshot get_object 04 put_object delete_object 05 set_object_meta Vdisk ObjectStorage get_object_meta QCOW2 VDMK Swift NFS Glusterfs
  • 7. Design - Process 00 Create volume Do nothing 01 Create volume from snapshot 02 Do nothing = 03 Delete volume Do nothing 04 Delete snapshot 05 Island delete snapshot in swift
  • 8. Design - Process Create the image of the volume 写上你的文字你的文字 Attach volume 00 01 volume old Check the new volume form snapshot 02 = volume 03 Download the last Download snapshot and snapshot and 04wirte it to the new volume wirte it to the image image 05 Update volume[‘host’]
  • 9. Design - Process Create snapshot 00 Detach volume 01 Create a new snapshot DB item, set Create snapshot in image snapshot[‘volume_id’] as the volume 02 03 Upload incremental Create snapshot in image snapshot to swift 04 05 Delete old snapshot in Upload incremental snapshot to swfit image Delete the image of the volume
  • 10. Snapshot – How to Create/Read Snapshot in Image Create internal snapshot in image Because there写上你的文字你的文字 command to create internal snapshot is not a qemu monitor 00 in image when the image was attached. So, we add a new qemu monitor command to create point-in-time snapshot in image. 01 Accessing QCOW2 image in python 02 = QCOW2 image format use 2-level Lookup Table http://people.gnome.org/~markmc/qcow-image-format.html 03 We implement qcow2 python library to read/write qcow2 image data and metadata. 04 05
  • 11. Snapshot – How to Implement Incremental Snapshot 00 写上你的文字你的文字 Compare current snapshot pointer table and previous snapshot pointer 01 table in image We need to find out updated data blocks between two02snapshots. = Find updated data blocks 03 04 Upload updated data blocks to 05 objectstorage Future: modify qcow2 driver in qemu, mark updated data blocks in pointer table of Upload snapshot pointer table to snapshot in image. objectstorage
  • 12. Discussions The 00 写上你的文字你的文字 storage capacity of a x86 server is limit, but if many instances in a server and many volume are attached to 01 these instances, it will be a issue. The 02 = attach volume action is slower than other plugin. 03 Optimization? 04 Make Island as a independent system? 05

Hinweis der Redaktion

  1. Snapshot & backend Clay, John Backup API Plugin(driver) + Backend
  2. Red is Shortcoming Green is Advantage Island Advantages High speed and low latency in local disk I/O Incremental snapshot Snapshots are stored in objectstorage The time to restore the volume is the same for all snapshots Cost reduction from using inexpensive local disks Raid 10 in local storage have a better reliability Its features are like AWS EBS volumes
  3. Design Idea= Local Storage + QCOW2 Image + Dependent Snapshot + Swift
  4. When user create a snapshot of volume, island will (1) Create snapshot in image; (2) Upload snapshot to swift
  5. Island module
  6. IslandDriver Process Attach volume Process (1) Island create the image of the volume. (2) if the volume is new volume, goto (5); (3) if the volume is new volume from snapshot, download the snapshot and write it to the image, goto (5); (4) if the volume is old volume, download the last snapshot and write it to the image; (5) update volume['host'] to new host which the instance besides. the volume is attached to the instance.
  7. Create snapshot (1) Island use qemu monitor command to create snapshot in image; (2) Island upload incremental snapshot to swift; (3) Island delete old snapshot in image. NOTE: snapshot can be created when the volume state is attached. When the volume is detached, there are not changes in volume, so it don't need to create snapshot.
  8. Point-in-time snapshot (instant snapshot) (即时快照)
  9. Except add some code like other plugin do, we did not modify any code in Cinder and Nova.