Hokkaido university academic cloud which started services in 2011, is the largest academic cloud system in Japan. Its peak performance is 43TFlops and HPC/Hadoop cluster instances can be deployed automatically.

1. Hokkaido University Academic Cloud:
Largest Academic Cloud System in Japan
Masaharu Munetomo
Information Initiative Center,
Hokkaido University, Sapporo, JAPAN.
munetomo@iic.hokudai.ac.jp
Cloud Technical Leadership Forum
@San Francisco Intercontinental Hotel
Apr. 18th, 2012

2. Information Initiative Center, Hokkaido University
• A key institute of “high performance computing infrastructure” (HPCI) in Japan.
• Founded in 1962 as a national supercomputing center.
• University R&D center for Supercomputing, Cloud computing, Networking, IT
systems for education
• Supercomputer (172TFlops, #95 in TOP500) & Cloud System (43TFlops)

3. Hokkaido University Academic Cloud
• Largest Academic Cloud System in Japan: 43TFlops
• More than 2,000 VMs can be deployed
• High-performance cloud system: each physical node has
40-cores, 128GB memory. Network: 10GbE x 2,
Shared Storage: 760TB
Hitach BladeSymphony BS2000
Xeon E7 8870 2.4GHz (10-core) x 4
128GB memory / 10GbE x 2
Hitach NAS Storage
AMS2300: 260TB
AMS2500: 500TB

4. An overview of software architecture
HW Boot/VM Image: SAN Data: NAS
RedHat/CentOS Hypervisor ( XenServer / VMWare)
Cloud&Middleware&(CloudStack)
Portal&system&
(In9house&developed
Univ.&SSO&
CloudStack&
API access

5. Why we employ CloudStack?

6. Why we employ CloudStack?

7. Why we employ CloudStack?
• High quality and usability in user portal and management
console

8. Why we employ CloudStack?
• High quality and usability in user portal and management
console
• Easy to control resources via API

9. Why we employ CloudStack?
• High quality and usability in user portal and management
console
• Easy to control resources via API
• Open source

10. VM Hosting Services: Hosting & Project servers
• Hosting servers: for web-hosting, etc. including CMS
middleware packages (WordPress, MediaWiki, etc.)
• Project servers: for research projects
• Cluster packages are also available: Hadoop/MPI
Class Cores Memory Storage JPY/month
S 1 3GB 100GB 525
M 4 12GB 100GB 2,100
L 10 30GB 100GB 5,250
XL 40 128GB 2TB 21,000

11. In-house developed cloud portal
Selection of service-level
Selection of packages
# of VMs for cluster
Authentication Config.
FW initial Config.

12. Network configuration
• Assign a VLAN to each user (laboratory)
Physical Node #1 Physical Node＃２
Virtual
Router
for user A
eth0%
eth0%
eth0.200 eth0.201
eth0%
eth0.800
eth0%
eth0.200eth0.201
Virtual
Router
for user B
eth1% eth0% eth0%eth0%eth1% eth0%
VLAN+200!VLAN+800! VLAN+201! VLAN+200!VLAN+201!
eth0.800
VLAN+800!
Virtual%Interface
Bridge
Send%with%
VLAN%tag
%Private%Network%for%User%A
%Private%Network%for%User%B
%Global%Network
Virtual%NIC

13. Automated Deployment of VM clusters
• Customizing scheduling policies in CloudStack to balance I/O overheads for
cluster packages (Hadoop / MPI / Torque).
Storage #3
Virtual(
Disk
Storage #4
Virtual(
Disk
Storage #2
Virtual(
Disk
Zone!
POD!
Shared Storage #1
Resource Pool #1
HyperVisor #2
HyperVisor #1
Virtual(
DiskVM(
Balancing!overheads!of!disk!I/O!with!
round8robin!assignment!of!Virtual!disks.!
Storage #1
VM(
VM(
VM(
VM(
Virtual(
DiskHadoop Cluster
Shared Storage #2
Resource Pool #2
HyperVisor #4
HyperVisor #3
Virtual(
Disk
VM(
Shared Storage #3
Resouce Pool #3
HyperVisor #6
HyperVisor #5
Virtual((
Disk
VM(
Shared Storage #4
Resouce Pool #4
HyperVisor #8
HyperVisor #7
Virtual(
Disk
VM(

14. Shared Storage #1
Resource Pool #1
HyperVisor #2
HyperVisor #1
Virtual(
DiskVM(
Shared Storage #2
Resource Pool #2
HyperVisor #4
HyperVisor #3
Virtual(
Disk
VM(
Shared Storage #3
Resouce Pool #3
HyperVisor #6
HyperVisor #5
Virtual((
Disk
VM(
Shared Storage #4
Resouce Pool #4
HyperVisor #8
HyperVisor #7
Virtual(
Disk
VM(
CloudStack+
(ManagementServer)+
CloudStack+
DB+
Planner+
Storage+load+balancing
VM+Alloca<on
Planner+
Request
VM+Deployment+
Job
Deployment(
Informa7on
Host
Storage
Deployment(
request
Deploy(VMs
CloudStack+
Portal

15. Deployment time for clusters
• Fast deployment: 5 min. for 1 VM, around 10 min. for 20 VMs
• A medium size cluster consisting
of 257 VMs can be deployed
in a little more than 1 hour !
• It greatly speedup starting
research projects since
conventional physical
cluster needs more than
a week or even a month
to finish deployment including
purchase process.
y"="0.2267x"+"4.1748"
R²"="0.9989
0"
10"
20"
30"
40"
50"
60"
70"
0" 50" 100" 150" 200" 250" 300"
Deployment*+me*(min)
#*of*VMs

16. Heterogeneous hybrid cloud deployment test
• CloudStack@Hokkaido University - OpenNebula@TokyoTech
We developed a VPN client to
establish connection via API
（Python + OpenSwan + l2tpd）
Gateway
Management Server
Computing Node
Gateway
Virtual Router
VPN Client
VPN Server
CloudStack
Virtual Route VPN

17. SHINCLOM (Simple Heterogeneous INter-CLOud
Manager): Inter-cloud manager for academic cloud systems

18. SHINCLOM (Simple Heterogeneous INter-CLOud
Manager): Inter-cloud manager for academic cloud systems

19. SHINCLOM (Simple Heterogeneous INter-CLOud
Manager): Inter-cloud manager for academic cloud systems