Grid2012 VC-Migration: Live Migration of Virtual Clusters in the Cloud

1. VC-Migration: Live Migration of Virtual Clusters in the Cloud Kejiang Ye, Xiaohong Jiang, Ran Ma, Fengxi Yan CCNT Lab, College of Computer Science Zhejiang University, China GRID 2012 Sep. 20-23, 2012 Beijing, China

2. Outline  Motivations  VC-Migration Framework  Virtual Cluster Performance  Cross-Domain Virtual Cluster  Virtual Cluster Scalability  Dynamic Migration Strategies of Virtual Cluster  Effects of Memory Configuration  Migration Scalability  Concurrent Migration with Various Granularities  Mutual Migration  Multi-VC Migration Strategies  Live Migration of Homogeneous Virtual Clusters  Live Migration of Heterogeneous Virtual Clusters  Master Node Migration vs. Slave Node Migration  Related Work  Conclusion & Future Work GRID 2012 Sep. 20-23, 2012 Beijing, China

4. Motivations  Currently, live migration of virtual machine s has become a key ingredient behind the management activities of cloud computing system to achieve the goals of:  Load balancing  Energy saving  Failure recovery  System maintenance  … GRID 2012 Sep. 20-23, 2012 Beijing, China

5. Motivations  Virtual Cluster is a group of virtual machines conf igured for a common purpose, such as high perfor mance computing or parallel computing, with asso ciated  Storage resource  Operating system  Software environment  Communication protocol  Network configuration  Two notable features of virtual cluster  Large scale  Intensive communication GRID 2012 Sep. 20-23, 2012 Beijing, China

6. Motivations  Live migration of virtual clusters (VC) faces s everal new challenges:  Huge Amount of Data  Limitation of Network Bandwidth  Intensive Communication between VMs  Synchronous Latency  Complex VC Migration Strategies  It is necessary to study this new migration sc enario to investigate the overheads & bottlen ecks. GRID 2012 Sep. 20-23, 2012 Beijing, China

7. Contributions  Describe a framework VC-Migration to co ntrol the migration of virtual clusters.  Perform a series of experiments to unders tand the performance bottleneck and over heads of virtual cluster migration:  Performance characterization of virtual cluster;  Dynamic migration strategies of virtual cluster s;  Migration of multiple virtual clusters; GRID 2012 Sep. 20-23, 2012 Beijing, China

9. VC-Migration Framework  Framework Design GRID 2012 Sep. 20-23, 2012 Beijing, China

10. VC-Migration Framework  VC-Migration Scenario GRID 2012 Sep. 20-23, 2012 Beijing, China

11. VC-Migration Framework  VC-Migration Strategies a. Concurrent Migration with Various Granularity b. Mutual Migration (a) (b) c. Homogeneous Multi-VC Migration d. Heterogeneous Multi-VC Migration (c) GRID 2012 (d) Sep. 20-23, 2012 Beijing, China

13. Virtual Cluster Performance  Experimental Configuration  Virtual Cluster Configuration  Dell T710 Server, with 2 Quad-core 64bit Xeon processors a nd 32 GB DRAM.  CentOS 5.6 with kernel version 2.6.18-238.12.1.e15xen in D omain 0, and Xen 3.3.1 as the hypervisor.  VM (Guest OS) with Ubuntu 8.10, 1 VCPU & 256 MB DRAM.  MPI version is MPICH 2.1.0.8  All the VM images are stored on a separate NFS storage ser ver  Benchmarks  HPC Challenge Benchmark Suite (HPCC) as the virtual clust er workloads: HPL, DGEMM, STREAM, PTRANS, RandomA ccess, FFT, Communication bandwidth and latency  Virt-LM Benchmark: to measure the migration performance a nd overheads. [Huang et al., ICPE2011] GRID 2012 Sep. 20-23, 2012 Beijing, China

14. Virtual Cluster Performance  Cross-Domain Virtual Cluster GRID 2012 Sep. 20-23, 2012 Beijing, China

15. Virtual Cluster Performance  Virtual Cluster Scalability GRID 2012 Sep. 20-23, 2012 Beijing, China

17. Dynamic Migration Strategies of Virtual Clust er  Effects of Memory Configuration GRID 2012 Sep. 20-23, 2012 Beijing, China

18. Dynamic Migration Strategies of Virtual Clust er  Migration Scalability Overall Time Average Time GRID 2012 Sep. 20-23, 2012 Beijing, China

19. Dynamic Migration Strategies of Virtual Clust er  Concurrent Migration with Various Granularities GRID 2012 Sep. 20-23, 2012 Beijing, China

20. Dynamic Migration Strategies of Virtual Clust er  Mutual Migration GRID 2012 Sep. 20-23, 2012 Beijing, China

22. Multi-VC Migration Strategies  Live Migration of Homogeneous Virtual Clusters GRID 2012 Sep. 20-23, 2012 Beijing, China

23. Multi-VC Migration Strategies  Live Migration of Heterogeneous Virtua l Clusters GRID 2012 Sep. 20-23, 2012 Beijing, China

24. Multi-VC Migration Strategies  Master Node Migration vs. Slave Node Migration GRID 2012 Sep. 20-23, 2012 Beijing, China

25. Experimental Findings  The main contradiction of VC migration is the large amount of image data and the limited network bandwidth.  The virtual machines belonging to the same virtual cluster should be deployed together as far as possible to reduce the communication a nd synchronization latency across different physical machines.  Virtual cluster has good scalability and is suitable for the high perfor mance computing and parallel computing tasks.  When a virtual cluster needs to be migrated, it is important to select a suitable concurrent migration granularity. Large concurrent granul arity will decrease the VC performance dramatically.  Mutual migration should be avoided due to the long overall migratio n time.  The migration of slave node incurs relatively less overhead compar ed to the master node. So it should give a priority to the slave migrat ion.  Migration order is important when multiple virtual clusters need to b e migrated. The long-time cross-domain virtual cluster will decrease the overall performance of applications. GRID 2012  There is a big optimization20-23, 2012 in the live migration of virtual clust Sep. space Beijing, China

27. Related Work  Virtual Cluster Performance Analysis  MPI Cluster [Huang et al., ICS’06; Ye et al., HPCC’10; Merge n et al., SIGOPS Oper. Syst. Rev., 2006]  MapReduce Cluster [Ibrahim et al., CloudCom’09]  Live Migration of Single VM  Pre-copy technique [Clark et al., NSDI’05; Nelson et al., US ENIX’05]  Post-copy technique [Hines et al., VEE’09]  Whole system migration [Luo et al., Cluster’08]  Live Migration of Multiple VM  Resource reservation method [Ye et al., Cloud’10]  Avoid the data de-duplication for concurrent migraiton [Deshpande et al., HPDC’11; Al-Kiswany et al. HPDC’11] However, all the above work didn’t solve the problem of live migration of virtual GRID 2012 cluster in which the intensive Sep. 20-23, 2012 Beijing, China communication overheads can affect the migration performance.

29. Conclusion  We have studied the live migration perfor mance and overheads of virtual clusters fr om the experimental perspective and inve stigated different VC migration strategies.  Experimental results reveal some new dis coveries, based on which we can propose several optimization principles to improve t he migration performance of virtual cluster s. GRID 2012 Sep. 20-23, 2012 Beijing, China

30. Future Work  Optimize the migration mechanism in the hypervisor to improve the migration efficie ncy of virtual clusters  Design efficient migration algorithms for th e virtual clusters. GRID 2012 Sep. 20-23, 2012 Beijing, China

31. Q&A Thank you! GRID 2012 Sep. 20-23, 2012 Beijing, China

Grid2012 VC-Migration: Live Migration of Virtual Clusters in the Cloud

Recommended

Recommended

More Related Content

Similar to Grid2012 VC-Migration: Live Migration of Virtual Clusters in the Cloud

Similar to Grid2012 VC-Migration: Live Migration of Virtual Clusters in the Cloud (20)

Grid2012 VC-Migration: Live Migration of Virtual Clusters in the Cloud