http://www.cs.virginia.edu/~mm5bw/papers/Cloud%20VM%20Startup%20Performance%20Study.pdf http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6253534 www.mingmao.org Amazon Web Service, EC2, Windows Azure, Rackspace, Spot Instances, VM Image, OS,

1. 1
A Performance Study on the VM
Startup Time in the Cloud
Cloud 2012
(June 25, Honolulu, Hawaii)
Ming Mao, Marty Humphrey
CS Department, University of Virginia

2. Introduction
2
 One great advantage of the cloud is dynamic scalability,
BUT…
 Only meaningful when the compute resource is available in time
 The fact is that acquired instances are NOT ready immediately
 Why study the cloud VM startup time
 Important to time sensitive applications
 Important to resource auto-scaling mechanisms
 Inconsistent definitions across cloud providers
 To set up a reference for the community
 To know the difference from two years ago

3. Experiment setup
3
Type OS Image Location  AWS, Windows Azure,
Amazon EC2 Rackspace
m1.small Linux(Fedora) ami-48aa4921 us-east-1a
m1.small Windows (Win Server 2008) us-east-1a  Oct 15th, 2011 – Feb 15th 2012
ami-fbf93092
Windows Azure  A client periodically collects the
Small WebRole South instance startup information
default WebRole app in Azure SDK Central US
Small WorkerRole South  By different factors – time,
default WorkerRole app in Azure SDK Central US
Small VMRole South image size, machine type,
Win Server 2008R2 Central US
Rackspace
location, etc.
Type IV Linux (Fedora) flavor 71 N/A
Type IV Windows (Win Server 2008R2) flavor 28 N/A
VM Definition of VM startup time
Linux VMs The first successful ssh login
Windows VMs The first successful remote desktop connection
WebRoles The first successful http request
WorkerRoles The first successful logging

4. Results (1) – by time
4
Cloud OS Average VM startup time
EC2 Linux 96.9 seconds
EC2 Windows 810.2 seconds
Azure WebRole 374.8 seconds
Azure WorkerRole 406.2 seconds
Azure VMRole 356.6 seconds
Rackspace Linux 44.2 seconds
Rackspace Windows 429.2 seconds

5. Results (2) – by image size
5
Cloud Average Data Transfer Rate
between VM and Image Store
EC2 10.9 MB/s
Azure 1.1MB/s
Rackspace 22.5 MB/s

6. Results (3) – by machine type
6
 Azure, Rackspace – the VM startup time goes longer as the
instance type goes larger
 EC2 – no significant differences among different instance
types, except the micro ones

7. Results (4) – by location
7
 No significant differences among locations
 EC2 - The newly established data center has a slightly slower
startup performance and greater variance

8. Results (5) – multiple instances
8
 EC2 – VM startup time is relatively constant.
 Azure – the last VM instance sometimes took significantly
longer than the first one.
 RS – does not support instance acquisition in batch

9. Results (6 - 1) – spot instances, by time
9
 Spot instances have a longer waiting time but similar VM
startup performance compared to on-demand instances

10. Results (6 - 2) – spot instances, by vm type
10
 Similar to on-demand instances, the VM startup time shows
no significant differences across different machine types

11. Results (6 - 3) – spot instances, by location
11
 The US east region shows longer spot instance acquisition
time

12. Results (6 - 4) – spot instances, multiple instances
12
 Similar to on-demand instances, spot instances show
constant performance across all the 16 machines acquired at
the same time

13. Results (6 - 5) – spot instances, by real-time price
13
 No significant relations between the VM startup time and the
real-time price

14. Result (7) – other facts
14
 Spot instances are not
always cheaper than on-
demand VMs
 Acquisition requests are
not always successfully
served (unusable
machines).
Cloud Success Rate VM Release Time
EC2 99.2% 3-8 seconds
Azure 99.6% 8-21 seconds
Rackspace 92.0% 3-8 seconds

15. Summary
15
 Summary
 VM startup time is independent of time of the day.
 EC2, Rackspace - Windows instances take 9 times longer
than Linux instances. Azure - all three Role instances show
similar performances.
 Both the size of the OS image and the instance type can
largely affect the VM startup time.
 spot instances - longer VM startup time and greater variance
 RS had a higher instance acquisition failure rate (8%) > EC2
(0.8%) & Azure (0.4%).
 Azure – 200-second improvement and smaller variances;
EC2 – performance does not change.

16. Questions
16
Thank you!