1. Dependable Cloud Computing:
Virtualization-Based Management
for Servers, Clients and Network
Kazuhiko Kato
University of Tsukuba
Japan
NSC-JST Workshop
Nov. 27, 2012

2. 2
Project Members
University of Tsukuba
Kazuhiko Kato, Akiyoshi Sugiki, Koji Hasebe
Yasushi Shinjyo
University of Tokyo
Takahiro Shinaga（Previously, University of Tsukuba)
University of Electro-Communications
Yoshihiro Oyama
Fujisoft Inc.
Yoshiaki Ishii, Kyohei Yano, Seiji Hirooka

3. Overview of Dependable Cloud computing
Developing infrastructural software for cloud computing with
servers, client, and network.
3
Dependability：
Reliability, Availability, Response, Throughput, Security, Privacy
Failure
Guest OS
BitVisor
Hardware
Servers
(in several data centers)NetworkClients
Internet

4. 4
(I) Dependable Server Management
Failure
Guest OS
BitVisor
Hardware
Servers
(in several data centers)NetworkClients
Internet

5. Kumoi（雲居）:
Middleware for Cloud Server Management
• Riding on the Scala programming language
✓ OO & functional
✓ "Scalable" coding (Java-to-Ruby level) with static type
system
• Object/Resource mapping for data centers
✓ Real/virtual machines and network are mapped to HW/SW
objs. (Cf. O/R mapping in db software)
• Incorporated distributed algorithms such as gossip
algorithms and Paxos.
• Available as open source software.
5

6. Kumoi Overview
6
Data center
Manager/operator
Method
call
Result
Interactive/batch
Kumoi shell
Scala
Kumoi kernel
Real
machine
Network
VMM VNet
VM Disk
34K lines of Scala source code

7. Kumoi Scripting
(Cf. Unix scripting)
7
scala> pms.fliter(_.cpuRatio > 0.9).map(_.name)
pms: List of available physical machines
_: Formal arguments for higher-order function

8. Kumoi System Programming:
VM-Compaction
8
def compact(pms: List[VM]) {
def firstFit(v: VM, rest: List[VM]) {
rest match {
case h :: rs if (h.cpuAvailable > v.cpuRatio) => v.migrateTo(h)
case h :: rs => firstFit(v, rs)
case List() =>
}
}
def compacti(pms: List[VM]) {
pms match {
case h :: rest =>
h.vms.foreach(v => firstFit(v, rest.reverse))
compacti(rest)
case List() =>
}
}
compacti(pms.reverse)
}

9. 9
(II) Dependable Client Management
Failure
Guest OS
BitVisor
Hardware
Servers
(in several data centers)NetworkClients
Internet

10. Virtual Machine Monitor
10
仮想マシン
（VM: Virtual Machine）
仮想マシン
（VM: Virtual Machine）
Hardware
Virtual Machine
Virtual Machine Monitor
Guest OS
Hardware
Physical Machine
OS

11. BitVisor: Secure VMM
• Storage management
✓ Encrypting HDD, USB memory
• Network management
✓ VPN (IPsec)
• ID Management
✓ Key management/authentication
with IC card
• VMM Core
✓ Virtualization of CPU and memory
11

12. Utilization of BitVisor
• System file protection of guest OS
• Malware detection
✓ IDS within VMM
• Transparent VPN switching (described in the
next topic)
12

13. System File Protection of Guest OS
•Integrity (code cannot be
modified undetectably)
Kernel image
Device driver
etc.

14. Implementation of System File Protection
• BitVisor monitors every storage
access.
✓Detects system file
modification.
• Mapping between files and
sectors are managed.
Guest&OS
Device
Device&driver
Extended&function
ATA NIC USB
Device&mediator
ATA NIC USB
VM
VMM
Hardware
Protection&policy

15. Malware detection
IDS within VMM
Run$at$the$boot$
+me$of$BitVisor
data$block$
data$block$

16. BitVisor as Research Platform
• HyperSafe [Wang et al., IEEE S&P ‘10]
✓ Integrity of hypervisor itself, i.e., modification
disabled.
• “Return-less” VMM [Li et al., EuroSys ‘10]
✓ Against ROR (Return-Oriented Rootkit)
• TCVisor [Rezaei et al., ICITST ‘10]
✓ Limited storage area can be seen by each user.
16

17. 17

18. 18
(III) Dependable Network
Failure
Guest OS
BitVisor
Hardware
Servers
(in several data centers)NetworkClients
Internet

19. Failure Detection in VMM

20. VPN Switching in VMM

21. Experiments with Real Data Center
21
Fujisoft in Yokohama
Fujisoft in Kyusyu
つくばTsukubaFujisoft in Kyusyu

22. VPN Switching
22
VPN failures
erforms two
ling is used
g IP address
hiding IP ad-
these opera-
a server.
eader like IP
hen a packet
P header has
rver IP as the
s assigned by
s assigned to
hat the guest
nt ID. When
e.g., 128bits)
between the
rds the rela-
0
5
10
15
20
25
30
0 100 200 300 400 500 600 700 800 900
Yokohama
Fukuoka
Tokyo
1000
Latency
Elapsed time [sec]
Figure 7. Transition of Latency to Data Cen-
ters
0
2
4
6
8
10
0 5 10 15 20 25 30
VPNthroughput[Mbit/sec]
Elapsed time [sec]
Failure occurred point Failure recovered point
15.1 19.2
Figure 8. Throughput Transition over Failure
km to Yokohama, and 926 km to Fukuoka. These data cen-
Before:Tsukuba-Tokyo (56Km)
After:Tsukuba-Yokohama (84Km)

23. Newtork Latency and Throughput of
VPN Switching
23
Tsukuba-Tokyo (56Km)
Tsukuba-Yokohama (84Km)
Tsukuba-Fukuoka (926Km)
Tokyo Yokohama Fukuoka
VPN on OS 13.18 12.63 32.04
VPN on VMM 13.46 13.00 32.57
VPN on VMM with relay 13.71 13.23 32.80
0
5
10
15
20
25
30
35
Latency[msec]
Figure 9. Latency
calsuling and NAT in a user-level process. We measured
the overhead in three environments: “VPN on OS” repre-
sents that VPN is implemented in a OS (using YAMAHA
YMS-VPN1), “VPN on VMM” represents that VPN is im-
plemented in BitVisor but not using the packet relay sys-
tem, and “VPN on VMM with relay” represents that VPN
is implemented in BitVisor and using the packet relay sys-
tem. We measured latency and throughput from a client at
Tsukuba to each data center over a VPN.
Figure 9 shows the latency. The overhead of our system
was 0.53–0.76 msec in total for each data center: the VMM
incured 0.28–0.53msec and the packet relay system incured
0.23–0.25 msec. Figure 10 shows the throughput. The over-
head of our system was about 8–30% in total: the VMM in-
cured 4–16% overhead and the packet relay system incured
Tokyo Yokohama Fukuoka
VPN on OS 13.18 12.63 32.04
VPN on VMM 13.46 13.00 32.57
VPN on VMM with relay 13.71 13.23 32.80
0
5
10
15
20
25
30
35
Latency[msec]
Figure 9. Latency
Tokyo Yokohama Fukuoka
VPN on OS 58.88 52.98 26.43
VPN on VMM 49.31 47.45 25.27
VPN on VMM with relay 41.22 41.94 24.45
0
10
20
30
40
50
60
70
Bandwidth[Mbit/sec]
Figure 10. Throughput
calsuling
the overh
sents that
YMS-VP
plemente
tem, and
is implem
tem. We
Tsukuba
Figure
was 0.53–
incured 0
0.23–0.25
head of o
cured 4–1
3–14% o
when con
the packe
the system
6. Relat
Severa
peer-to-p
15, 16, 1
in the bo
tems. Ho
struct ove
system a
prove ava
A rout

24. Summary
Dependable cloud computing environment for servers,
client and network, by using virtualization technologies.
24
Failure
Guest OS
BitVisor
Hardware
Servers
(in several data centers)NetworkClients
Internet

25. Ongoing Work
• Extension and application of Kumoi
✓ Virtual network control with OpenFlow
✓ Failure-oblivious computing
✓ Application: Parallel, distributed parameter tuning
• BitVisor application
✓ Transparent network boot system
✓ Acceleration of guest OS boot
✓ Desktop grid with intra-VMM computation
• Energy-saving distributed storage system
25