Orran Krieger, Sr. Staff Engineer at VMware, giving a presentation at the Stanford Computer Forum Annual Meeting 2009 Plenary on VMware's vCloud initiative.
Comprehensive overview of their direction and the vCloud architecture.
12. Distributed Virtual Switch
• Network-centric view:
core
virtualization blurs the
host/network boundary
distribution
access
hosts
VMs
5
Wednesday, April 15, 2009
13. Distributed Virtual Switch
• Network-centric view:
core
virtualization blurs the
host/network boundary
distribution
• Virtual network switches and
topologies within a host
access • VMs can move anywhere
• The virtual network switch is
a new access layer.
hosts
VMs
5
Wednesday, April 15, 2009
14. Distributed Virtual Switch
• Network-centric view:
core
virtualization blurs the
host/network boundary
distribution
• Virtual network switches and
topologies within a host
access • VMs can move anywhere
• The virtual network switch is
a new access layer.
• Network administrators can
hosts no longer rely on physical
port-based access control
VMs
5
Wednesday, April 15, 2009
15. Distributed Virtual Switch
• Network-centric view:
core
virtualization blurs the
host/network boundary
distribution
• Virtual network switches and
topologies within a host
access • VMs can move anywhere
• The virtual network switch is
a new access layer.
• Network administrators can
hosts no longer rely on physical
port-based access control
Distributed Virtual Switch
VMs
Distributed virtual switch to extend
network access control and
management to virtual environments
5
Wednesday, April 15, 2009
16. Availability
Hot-plug resources
6
Wednesday, April 15, 2009
17. Availability
Hot-plug resources
6
Wednesday, April 15, 2009
18. Availability
Add/remove capacity on demand
Hot-plug resources Improve application availability
6
Wednesday, April 15, 2009
19. Availability
Add/remove capacity on demand
Hot-plug resources Improve application availability
6
Wednesday, April 15, 2009
20. Site Recovery Manager
Production Recovery
VMware Infrastructure VMware Infrastructure
Site Recovery Manager transforms disaster recovery
7
Wednesday, April 15, 2009
21. Site Recovery Manager
Production Recovery
VMware Infrastructure VMware Infrastructure
Site Recovery Manager transforms disaster recovery
7
Wednesday, April 15, 2009
22. Security: VMware VMsafe
API that enables protection of VMs
by inspection of virtual components
in conjunction with hypervisor
Isolation of protection engine from
malware
Application Broad ranging coverage of virtual
machine CPU, memory, storage
Operating System
Protection
Engine
VMware Infrastructure
8
Wednesday, April 15, 2009
23. vApp: A new type of Software
Policies
• Properties
• Policy-based operations 1. Product: eCommerce
2. Topology
• Multi-tier 3. Resources Req: CPU, Mem,
• Distributed as an OVF Disk,Bandwidth
package 4. Only port 80 is used
5. DR RPO: 1 hour
• Built by: 6. VRM: Encrypt w/ SHA-1
7. Decommission in 2 month
• ISVs / Virtual Appliance
Vendors Websphere
Fire
• By internal IT shops Exchange
Wall Tomcat
• By IT administrators
• SI/VARs
SAP
9
Wednesday, April 15, 2009
25. Desktops
Centralized
Virtual Desktop Infrastructure (VDI) Virtual Desktops
• Full desktops run as VMs in the data center
• Connect securely to desktop from anywhere
• Centralized management
• More efficient resource usage VMware
Infrastructure 3
• Higher availability
• Online and offline mode
VMware Virtual
Desktop Manager
Thick or Thin
Clients
10
Wednesday, April 15, 2009
26. Evolution of Virtualization
Separate Consolidate Aggregate Automate Liberate
CapEx OpEx Savings Business No Physical
Savings Agility Boundaries
Automation
Automation
Virtual
Virtual Virtual
Infrastructure
Infrastructure Infrastructure
Management
Management Management Management
Hypervisor
Hypervisor Hypervisor Hypervisor Hypervisor
Cloud Scale
Test and Capacity
Server Self-Managing
Computing
Development On Demand
Consolidation Datacenter
11
Wednesday, April 15, 2009
27. The cloud started with SaaS/Web
Traditional software model
Test
Develop Install
Release Operate
Configure
SaaS model
Test
Develop Operate
12
Wednesday, April 15, 2009
28. The reality: success is very hard
Test Operate
Develop
13
Wednesday, April 15, 2009
29. The reality: success is very hard
Test Operate
Develop
• Most SW companies don’t have the in-house skill to
operate at scale:
• e.g. loadbalancer, network config, security, disaster recovery, …&
70% of investment spent not enhancing application
13
Wednesday, April 15, 2009
30. The reality: success is very hard
Operate
Test
Develop
• Most SW companies don’t have the in-house skill to
operate at scale:
• e.g. loadbalancer, network config, security, disaster recovery, …&
70% of investment spent not enhancing application
13
Wednesday, April 15, 2009
31. The reality: success is very hard
Operate
Test
Develop
• Most SW companies don’t have the in-house skill to
operate at scale:
• e.g. loadbalancer, network config, security, disaster recovery, …&
70% of investment spent not enhancing application
• Enormous investment required in application level to
scale.
13
Wednesday, April 15, 2009
32. The reality: success is very hard
Operate
Develop Test
• Most SW companies don’t have the in-house skill to
operate at scale:
• e.g. loadbalancer, network config, security, disaster recovery, …&
70% of investment spent not enhancing application
• Enormous investment required in application level to
scale.
13
Wednesday, April 15, 2009
33. The reality: success is very hard
Operate
Develop Test
• Most SW companies don’t have the in-house skill to
operate at scale:
• e.g. loadbalancer, network config, security, disaster recovery, …&
70% of investment spent not enhancing application
• Enormous investment required in application level to
scale.
• So, successful SaaS vendors started building re-usable
platforms…
13
Wednesday, April 15, 2009
34. Cloud offerings
• Software as a Service (SaaS):
• Application hosted in the cloud.
• e.g., gmail, google apps, salesforce.com, wikipedia…
• Platform as a Service (PaaS):
• Application development environment and runtime hosted
in cloud.
• e.g., engine yard, Google App Engine, Force.com
• Infrastructure as a Service (IaaS):
• Developers/administrators obtain general computing,
storage, messaging services…
• e.g., terramark, Amazon AWS, Mosso
14
Wednesday, April 15, 2009
36. Example
• Animoto April 2008: Peak EC2 instances:
• Mon 50, Tues 400, Wed 900, Friday 3400
16
Wednesday, April 15, 2009
37. The problems with this evolution
• Vertical offered by a single vendor
• Just a few hundred/thousand developers enhancing offering
• No on-premise offering for enterprise/university/SaaS vendor
• Who wants to trust a single company?
• Limited largely to web applications:
• Limited support legacy, HPC, hosted client, grid,
• Disintermediates OEMs, infrastructure vendors,
traditional management stacks… (VARs for SaaS)
• Ignores the key value in the virtualization needed for
general purpose workloads: over provisioning, SRM,
DRS, DPM, OVF, SVI …
17
Wednesday, April 15, 2009
39. Disclaimer…
• At this point, there is the mandatory… okay
there is a lot of hype here, but…
18
Wednesday, April 15, 2009
40. Disclaimer…
• At this point, there is the mandatory… okay
there is a lot of hype here, but…
18
Wednesday, April 15, 2009
41. Disclaimer…
• At this point, there is the mandatory… okay
there is a lot of hype here, but…
18
Wednesday, April 15, 2009
42. Disclaimer…
• At this point, there is the mandatory… okay
there is a lot of hype here, but…
• The transformation is more profound that is yet
understood, cloud done right will:
• be used for all applications,
• change how we deploy and develop applications,
• enable new OSes, new programming models, new servers,
new storage solutions
• enable new markets for computer services
• …, in other words, its gonna change everything
18
Wednesday, April 15, 2009
43. What do we really want
19
Wednesday, April 15, 2009
44. What do we really want
Original vision of Utility/grid computing:
19
Wednesday, April 15, 2009
45. What do we really want
Original vision of Utility/grid computing:
”If computers of the kind I have advocated become the computers of the
future, then computing may someday be organized as a public utility just
as the telephone system is a public utility... The computer utility could
become the basis of a new and important industry.”
19
Wednesday, April 15, 2009
46. What do we really want
Original vision of Utility/grid computing:
”If computers of the kind I have advocated become the computers of the
future, then computing may someday be organized as a public utility just
as the telephone system is a public utility... The computer utility could
become the basis of a new and important industry.”
John McCarthy, MIT Centennial in 1961
19
Wednesday, April 15, 2009
47. What do we really want
Original vision of Utility/grid computing:
”If computers of the kind I have advocated become the computers of the
future, then computing may someday be organized as a public utility just
as the telephone system is a public utility... The computer utility could
become the basis of a new and important industry.”
John McCarthy, MIT Centennial in 1961
19
Wednesday, April 15, 2009
48. What do we really want
Original vision of Utility/grid computing:
”If computers of the kind I have advocated become the computers of the
future, then computing may someday be organized as a public utility just
as the telephone system is a public utility... The computer utility could
become the basis of a new and important industry.”
John McCarthy, MIT Centennial in 1961
Virtualization converts computation into a fungible
commodity
19
Wednesday, April 15, 2009
49. Why would this be transformative
• Nicholas Carr:
• “As with the factory-owned generators that dominated
electricity production a century ago, today's private IT
plants will be supplanted by large-scale, centralized
utilities.”
• Gets rid of key impediments to innovation:
• Virtual appliance model for distributing installing
applications.
• Avoids need for broad HCL, OS support, …
• Availability of massive capacity on demand.
• Enables long-tail in SW
20
Wednesday, April 15, 2009
50. VMware’s vCloud initiative
• Provide SW so that anyone can easily
create cloud: e.g., SP, University, Enterprise
• Supports all applications
• Controlled programmatically:
• From VMs, from capacity owner, from portal
• OVF based API
• Can scale up to massive data centers.
• Can scale down to small department.
• Can be accessed from VI Client
• New end user interface provide simple self
service experience.
• Enable broad partner and research
collaboration:
• Researchers can replace any part of the service.
• Researchers can replace the entire implementation
and clone the API
21
Wednesday, April 15, 2009
51. VMware’s vCloud initiative
• Provide SW so that anyone can easily
create cloud: e.g., SP, University, Enterprise
• Supports all applications
• Controlled programmatically:
• From VMs, from capacity owner, from portal
• OVF based API
• Can scale up to massive data centers.
vCloud Software
• Can scale down to small department.
• Can be accessed from VI Client
• New end user interface provide simple self
service experience.
• Enable broad partner and research
collaboration:
• Researchers can replace any part of the service.
• Researchers can replace the entire implementation
and clone the API
21
Wednesday, April 15, 2009
52. VMware’s vCloud initiative
• Provide SW so that anyone can easily
create cloud: e.g., SP, University, Enterprise
• Supports all applications
• Controlled programmatically:
• From VMs, from capacity owner, from portal
vCloud API
• OVF based API
• Can scale up to massive data centers.
vCloud Software
• Can scale down to small department.
• Can be accessed from VI Client
• New end user interface provide simple self
service experience.
• Enable broad partner and research
collaboration:
• Researchers can replace any part of the service.
• Researchers can replace the entire implementation
and clone the API
21
Wednesday, April 15, 2009
53. VMware’s vCloud initiative
• Provide SW so that anyone can easily
create cloud: e.g., SP, University, Enterprise
• Supports all applications
• Controlled programmatically:
• From VMs, from capacity owner, from portal
vCloud API
• OVF based API
• Can scale up to massive data centers.
vCloud Software
• Can scale down to small department.
• Can be accessed from VI Client
• New end user interface provide simple self
service experience.
• Enable broad partner and research
collaboration:
• Researchers can replace any part of the service.
• Researchers can replace the entire implementation
and clone the API
21
Wednesday, April 15, 2009
54. VMware’s vCloud initiative
• Provide SW so that anyone can easily
create cloud: e.g., SP, University, Enterprise
• Supports all applications
• Controlled programmatically:
• From VMs, from capacity owner, from portal
vCloud API
• OVF based API
• Can scale up to massive data centers.
vCloud Software
• Can scale down to small department.
• Can be accessed from VI Client
• New end user interface provide simple self
service experience.
• Enable broad partner and research
collaboration:
• Researchers can replace any part of the service.
• Researchers can replace the entire implementation
and clone the API
21
Wednesday, April 15, 2009
55. VMware’s vCloud initiative
• Provide SW so that anyone can easily
create cloud: e.g., SP, University, Enterprise
VI
• Supports all applications
Client
• Controlled programmatically:
• From VMs, from capacity owner, from portal
vCloud API
• OVF based API
• Can scale up to massive data centers.
vCloud Software
• Can scale down to small department.
• Can be accessed from VI Client
• New end user interface provide simple self
service experience.
• Enable broad partner and research
collaboration:
• Researchers can replace any part of the service.
• Researchers can replace the entire implementation
and clone the API
21
Wednesday, April 15, 2009
56. VMware’s vCloud initiative
• Provide SW so that anyone can easily
create cloud: e.g., SP, University, Enterprise
VI
• Supports all applications Self-service UI
Client
• Controlled programmatically:
• From VMs, from capacity owner, from portal
vCloud API
• OVF based API
• Can scale up to massive data centers.
vCloud Software
• Can scale down to small department.
• Can be accessed from VI Client
• New end user interface provide simple self
service experience.
• Enable broad partner and research
collaboration:
• Researchers can replace any part of the service.
• Researchers can replace the entire implementation
and clone the API
21
Wednesday, April 15, 2009
57. VMware’s vCloud initiative
• Provide SW so that anyone can easily
create cloud: e.g., SP, University, Enterprise
VI
• Supports all applications Self-service UI
Client
• Controlled programmatically:
• From VMs, from capacity owner, from portal
vCloud API
• OVF based API
• Can scale up to massive data centers.
vCloud Software
• Can scale down to small department.
• Can be accessed from VI Client
• New end user interface provide simple self
service experience.
• Enable broad partner and research
collaboration:
• Researchers can replace any part of the service.
• Researchers can replace the entire implementation
and clone the API
21
Wednesday, April 15, 2009
58. VMware’s vCloud initiative
• Provide SW so that anyone can easily
create cloud: e.g., SP, University, Enterprise
VI
• Supports all applications Self-service UI
Client
• Controlled programmatically:
• From VMs, from capacity owner, from portal
vCloud API
• OVF based API
• Can scale up to massive data centers. Alternative Implementation
vCloud Software
• Can scale down to small department.
• Can be accessed from VI Client
• New end user interface provide simple self
service experience.
• Enable broad partner and research
collaboration:
• Researchers can replace any part of the service.
• Researchers can replace the entire implementation
and clone the API
21
Wednesday, April 15, 2009
59. Key abstractions vCloud
• Organization with users/
roles
Organization 1
• Cloud to admin on his own
VDC 1 VDC 2
terms:
vApp vApp
• data centers
• L2 networks
“VDCnet”
“foo”
• Persistent VMs
(fenced)
ND
“bar”
• SLA on vApp
“VDCnet”
(isolated)
ND
• overprovisioning
“Private”
• ...
“Public”
• OVF based REST API:
• multi-tiered applications
Mapped as network
• serialized using OVF
adapters into installation
• configuration via OVF
22
Wednesday, April 15, 2009
60. Key abstractions vCloud UI & API
Organization 1 Organization 2
VDC 1 VDC 1
VDC 2 VDC 2
vApp vApp
vApp vApp
“VDCnet” “VDCnet”
“foo” “foo”
(fenced) (fenced)
ND ND
“bar” “bar”
“VDCnet” “VDCnet”
(isolated) (isolated)
ND ND
“Private” “Private”
“Public” “Public”
Same color means same network
23
Wednesday, April 15, 2009
61. It should be easy, VMware already has
• Rich service provider, OEM, ISV partner
ecosystem
• API for controlling virtualization that has
become a defacto standard
• Ability to deal with large numbers of hosts as a
pool of resources
• Support for backup, DR, resource management,
power management, …
• Rich community of users
• Rich user interface loved by administrators
•…
24
Wednesday, April 15, 2009
62. Not so fast…
• Problems with our existing technology:
• Core abstractions exposed physical as well as virtual
• Scaling up to 10000 hosts & 100K VMs fundamentally different
• Security more of a issue in multi-tenancy environment
• Challenges with our approach:
• Need to scale down
• Need to enable partner ecosystem:
• No one circumscribed approach.
• Need to release SW to others to install, configure…
• Need to enable others to innovate.
25
Wednesday, April 15, 2009
71. Scale up versus down
• Large scale service: Service Oriented Architecture
• Each service totally independent with own DB, own set of
machines… all communication through interface:
• Advantages
• Fails independently & can identify failures
• Own reliability based on requirements of data
• Can evolve services independently
• Can scale them independently
• Small scale: shrink wrapped software
• Single general purpose piece of software
• Advantages:
• Minimize overhead
• Simplified installation
27
Wednesday, April 15, 2009
72. Modular design
Large Scale
Modularity also enables extensibility
28
Wednesday, April 15, 2009
73. Modular design
Large Scale Small Scale
Modularity also enables extensibility
28
Wednesday, April 15, 2009
74. Modular design
Small Scale
Modularity also enables extensibility
28
Wednesday, April 15, 2009
75. Key Technologies
REST API : HTTP based resource oriented interface;
All the characteristics of the WWW
• Extensible without breaking client.
• Client only has to know about what it cares about.
• Can route, proxy, cache
Spring: Standard component framework
• Injects dependencies and wires together Spring beans
• Forces programmer into maintainable design pattern; isolates dependencies
OSGI: Standard dynamic module framework
• Global registry of interfaces to instances
• Dynamically load, unload, start, stop bundle
Other technology
• JMS publish/subscribe messaging bus isolates end points
• Hibernate simplifies DB code & DB independence
29
Wednesday, April 15, 2009
76. Architecture/Implementation
vCloud Self
vCloud API client
VMRC
Transfer
Client Service UI
Loadbalancer
Blaze DS
Rest dispatcher
vCloud Task vRP VC IP Pool
VMRC Network
Image Catalg Cloner
OSGI Network Mgr Mgr Control Mgr
Proxy srvc
xfer Netwrk
srvc
Mgmt srvc vCloud
Application
Cell Network Storage
Sched Authnt Security
VC Invent
Directory Handler Mgr
Tasks Mgr Mgr
User/
Message Bus
Model DB Inventory DB Account
Billing
Mgmt
Substrate Inventory
VC Validator Diff
Mgmt Listener
Substrate ESX hosts Networking
Running VM storage Offline
storage
30
Wednesday, April 15, 2009
77. Architecture/Implementation
vCloud Self
vCloud API client
VMRC
Transfer
Client Service UI
Loadbalancer
Blaze DS
Rest dispatcher
vCloud Task vRP VC IP Pool
VMRC Network
Image Catalg Cloner
OSGI Network Mgr Mgr Control Mgr
Proxy srvc
xfer Netwrk
srvc
Mgmt srvc vCloud
Application
Cell Network Storage
Sched Authnt Security
VC Invent
Directory Handler Mgr
Tasks Mgr Mgr
User/
Message Bus
Model DB Inventory DB Account
Billing
Mgmt
Substrate Inventory
VC Inventory Validator Diff
VC Listener Validator Diff
Mgmt Listener
ESX hosts Networking
Running VM storage Offline
Substrate ESX hosts Networking
Running VM storage Offline
storage
storage
30
Wednesday, April 15, 2009
78. Architecture/Implementation
vCloud Self
vCloud API client
VMRC
Transfer
Client Service UI
Loadbalancer
Blaze DS DS
Rest dispatcher Blaze
Rest dispatcher
vCloud Task vRPvRP VC VC IP Pool
VMRC
VMRC Network
Image Task IP Pool
Catalg Cloner
Network
Image Network MgrMgr MgrMgr Control
Catalg Cloner MgrMgr
OSGI Proxy Network
srvcsrvc
xfer xfer Netwrk Control
Proxy srvcNetwrk
srvc
srvcsrvc vCloud
Mgmt vCloud
Application
Application
Cell Network Storage
Sched Authnt Security
Network Storage
Sched Authnt Security
MgrMgr VC Invent
Directory Handler VC Invent Mgr
Tasks MgrMgr
Directory Handler Tasks Mgr
User/
Message Bus
Model DB Inventory DB Account
Billing
Mgmt
Substrate Inventory
VC Inventory Validator Diff
VC Listener Validator Diff
Mgmt Listener
ESX hosts Networking
Running VM storage Offline
Substrate ESX hosts Networking
Running VM storage Offline
storage
storage
30
Wednesday, April 15, 2009
79. Architecture/Implementation
vCloud Self
vCloud API client
VMRC
Transfer
Client Service UI
Loadbalancer
Blaze DS DS
Rest dispatcher Blaze
Rest dispatcher
vCloud Task vRPvRP VC VC IP Pool
VMRC
VMRC Network
Image Task IP Pool
Catalg Cloner
Network
Image Network MgrMgr MgrMgr Control
Catalg Cloner MgrMgr
OSGI Proxy Network
srvcsrvc
xfer xfer Netwrk Control
Proxy srvcNetwrk
srvc
srvcsrvc vCloud
Mgmt vCloud
Application
Application
Cell Network Storage
Sched Authnt Security
Network Storage
Sched Authnt Security
MgrMgr VC Invent
Directory Handler VC Invent Mgr
Tasks MgrMgr
Directory Handler Tasks Mgr
User/
Message Bus
Model DB Inventory DB Account
Billing
Mgmt
Substrate Inventory
VC Validator Diff
Mgmt Listener
Substrate ESX hosts Networking
Running VM storage Offline
storage
30
Wednesday, April 15, 2009
80. Architecture/Implementation
vCloud Self
vCloud API client
VMRC
Transfer
Client Service UI
Loadbalancer
Blaze DS
Rest dispatcher
vCloud Task vRP VC IP Pool
VMRC Network
Image Catalg Cloner
OSGI Network Mgr Mgr Control Mgr
Proxy srvc
xfer Netwrk
srvc
Mgmt srvc vCloud
Application
Cell Network Storage
Sched Authnt Security
VC Invent
Directory Handler Mgr
Tasks Mgr Mgr
User/
Message Bus
Model DB Inventory DB Account
Billing
Mgmt
Substrate Inventory
VC Validator Diff
Mgmt Listener
Substrate ESX hosts Networking
Running VM storage Offline
storage
30
Wednesday, April 15, 2009
81. Architecture/Implementation
vCloud Self
vCloud API client
VMRC
Transfer
Client Service UI
Loadbalancer
Blaze DS
Rest dispatcher
vCloud Task vRP VC IP Pool
VMRC Network
Image Catalg Cloner
OSGI Network Mgr Mgr Control Mgr
Proxy srvc
xfer Netwrk
srvc
Mgmt srvc vCloud
Application
Cell Network Storage
Sched Authnt
VC Invent
Directory Handler Mgr
Tasks Mgr
User/
Message Bus
Model DB Inventory DB Account
Billing
Mgmt
Substrate Inventory
VC Validator Diff
Mgmt Listener
Substrate ESX hosts Networking
Running VM storage Offline
storage
30
Wednesday, April 15, 2009
82. Architecture/Implementation
vCloud Self
vCloud API client
VMRC
Transfer
Client Service UI
Loadbalancer
Blaze DS
Rest dispatcher
vCloud Task vRP VC IP Pool
VMRC Network
Image Catalg Cloner
OSGI Network Mgr Mgr Control Mgr
Proxy srvc
xfer Netwrk
srvc
Mgmt srvc vCloud
Application
Cell Network Storage
Sched Authnt Security
VC Invent
Directory Handler Mgr
Tasks Mgr Mgr
User/
Message Bus
Model DB Inventory DB Account
Billing
Mgmt
Substrate Inventory
VC Validator Diff
Mgmt Listener
Substrate ESX hosts Networking
Running VM storage Offline
storage
30
Wednesday, April 15, 2009
83. Research directions
• New operating systems:
• e.g., for Java, security services, clustering services, HPC/
multi-core, ...
• library OS, multi-core, accelerator to OS, sharing
• Fungible computing:
• performance, cost to user, cost to SP
• Grid/HPC/batch scheduling, e.g., snowflock
• Federation:
• long tail for data de-duplication, encryption/security for
data, trading floor/futures market, vmotion to user
• What changes when your desktop is in the
cloud?
• Mom’s sysadmin, Fusion on steroids, disaster recovery...
31
Wednesday, April 15, 2009
84. Concluding reports
• Cloud computing is going to be transformative
to our industry
• VMware building a platform so that anyone
can play
• There are a wealth of research opportunities
• We will be providing the SW to universities
this year
• This is just the start...
32
Wednesday, April 15, 2009