CloudStack Design Camp in Taiwan Overview Slide by TCloud Computing

1. CloudStack
Overview
(Cont’d)
Roxanne_Chang
@
Tcloud
compu4ng

2. Outline
• Management
Server
Internals
• Workflow
of
Management
Server
• Extending
Plugins
to
CloudStack
• High
Availability
• Scalability
• CloudStack
v.s.
OpenStack
v.s.
Eucalyptus
• CloudStack
Par4cipa4on

3. Management
Server
Internals
• Architecture
• Workflow
• High
Availability
• Scalability

4. Old
Architecture
EC2
CloudStack
API
Layer
Access
Control
Virtual
Machine
Manager
Console
Proxy
Manager
Async
Job
Manager
Template
Manager
Snapshot
Manager
Network
Manager
Storage
Manager
…
Agent Manager
XenServer
KVM
Resource
SRX
Resource
NetScaler
Other
Resources
F5
Resource
Resource
Resource
4

5. New
Deployment
Architecture

6. New
Architecture
–
API
Server
Other
UI
Cloud
Portal
CLI
Clients
REST
API Server
Pluggable
API
Engine
OAM&P
API
End
User
API
EC2
API
Other
APIs
Management
Services
&
Authen4ca4on
ACL
Integra4on
-­‐ Resource
management
-­‐ Accounts,
Domains,
and
-­‐ Configura4on
Projects
-­‐ Addi4onal
opera4ons
added
by
-­‐ ACL,
limits
checking
third
party
Framework
-­‐ Job
Queue
-­‐ Database
Access
Layer
-­‐ OSGi

7. New
Architecture
–
Execu4on
Server
Execution Server
Services
API
Kernel
Plugins
• Drives
long
running
VM
opera4ons
• Storage
Handling
• Syncs
between
resources
managed
and
• Network
Handling
DB
• Deployment
• Generates
events
planning
• Hypervisor
Handling
Framework
• Cluster
Management
• Job
Management
• Alert
&
Event
Management
• Database
Access
Layer
• Messaging
Layer

8. New
Architecture
–
Resources
Agent
• Resources
are
carried
in
Hypervisor
Resources
service
VMs
to
be
in
close
network
proximity
to
the
Network
Resources
physical
resources
it
Storage
Resources
manages
Image
&
Template
Resources
• Easily
scales
to
u4lize
the
most
abundant
resource
in
Snapshot
Resources
data
center
(CPU
&
RAM)
• Communicates
with
Execu4on
Server
over
message
bus
(JSON)
• Can
be
replicated
for
fault
tolerance

9. Comparison
of
two
Approaches
• Stats
Collector
–
collects
capacity
sta4s4cs
– Fires
every
five
minutes
to
collect
stats
about
host
CPU
and
memory
capacity
– Smart
server
and
dumb
client
model:
• Resource
only
collects
info
and
management
server
processes
• VM
Sync
– Fires
every
minute
– Peer
to
peer
model:
• Resource
does
a
full
sync
on
connec4on
and
delta
syncs
thereacer.
Management
server
trusts
on
resource
for
correct
informa4on.

10. Cloud
Other
UI
CLI
Clients
Portal
Management
Server
REST
API
OAM&P
API
End
User
API
EC2
API
Other
APIs
Pluggable
Service
API
Engine
Console
Proxy
ACL
&
Authen4ca4on
Security
Adapters
Management
-­‐ Accounts,
Domains,
and
Projects
-­‐ ACL,
limits
checking
Account
Management
Connectors
Template
Access
Services
API
Deployment
Planning
Plugin
API
Kernel
HA
-­‐ Drives
long
running
VM
opera4ons
Network
Configura4ons
Services
API
-­‐ Syncs
between
resources
managed
Usage
and
DB
Calcula4ons
-­‐ Generates
events
Network
Elements
Addi4onal
Services
Hypervisor
Gurus
Cluster
Resource
Job
Alert
&
Event
Database
Management
Management
Management
Management
Access
Event
Bus
Message
Bus
Hypervisor
Network
Storage
Image
Snapshot
Resources
Resources
Resources
Resources
Resources

11. Workflow
of
Management
Server

12. Workflow
of
Management
Server
Plugins
cmd.execute()
Plugins
Cmds
Plugins
Async
CS
API
API
Job
Services
Servlet
Queue
API
Mgr
Kernel
Responses
Agent
API
(Commands)
Agent
Resources
Manager
Local
Or
Remote
Hypervisor
Network
Na4ve
Device
APIs
API
MySQL

13. Balancing
Incoming
Requests
• Each
management
server
has
two
worker
thread
pools
– Executor
threads
provided
by
tomcat
– Job
threads
wai4ng
on
job
queue
• DB
opera4ons
– short
in
dura4on
– executed
by
executor
threads
• Requests
needing
resources
– ocen
have
long
running
dura4ons,
– checked
against
ACL
by
the
executor
threads
– queued
and
executed
by
job
threads.

14. Interac4ons
OVM
Cluster
Primary
Storage
vcenter
Monitoring
Primary
CS
API
vSphere
Cluster
Storage
End
User
UI
Primary
XS
Cluster
Storage
Admin
UI
Clustered
CloudStack
XAPI
Domain
CS
Admin
&
CloudStack
CloudStack
End-­‐user
API
Primary
Admin
UI
Management
JSON
KVM
Cluster
Storage
Server
NetConf
Juniper
SRX
Cloud
user
Nitro
API
{API
client
(Fog/etc)}
VNC
JSON
ec2
API
JSON
Netscaler
Cloud
user
Console
Console
{ec2
API
client
}
Proxy
VM
Proxy
VM
NFS
MySQL
Server
{Proxied}
SSH
Sec.
Storage
NFS
NFS
Sec.
Storage
VM
Ajax
HTTPS
VM
Console
Router
VM
HTTP
(Template
Download)
Router
VM
HTTP
(Template
Copy)
Router
VM
Cloud
user
HTTP
(Swic)

15. Extends
Plugins
to
CS

16. Plugins
• Various
ways
to
add
more
capability
to
CloudStack
• Implements
clearly
defined
interfaces
• All
opera4ons
must
be
idempotent
• Compiles
only
against
the
Plugin
API
module

17. Anatomy
of
a
Plugin
Rest
API
-­‐ Op4onal.
Required
only
if
needs
to
expose
configura4on
API
to
admin.
ServerResource
-­‐ Op4onal.
Required
if
Plugin
needs
to
be
co-­‐located
with
the
resource
-­‐ Implements
Plugin
API
transla4on
layer
to
Implementa4on
talk
to
resource
-­‐ Communicates
with
server
component
via
JSON
Data
Access
Layer

18. Anatomy
of
a
Plugin
• Can
be
two
jars:
– server
component
to
be
deployed
on
management
server
– an
op4onal
ServerResource
component
to
be
deployed
co-­‐located
with
the
resource
• Server
component
can
implement
mul4ple
Plugin
APIs
to
affect
its
feature
– As
an
example,
OVS
plugin
actually
implements
both
NetworkGuru
and
NetworkElement
• Can
expose
its
own
API
through
Pluggable
Service
so
administrators
can
configure
the
plugin

19. Plugin
Interfaces
Available
• NetworkGuru
–
Implements
various
network
isola4on
technologies
and
ip
address
technologies
• NetworkElement
–
Facilitate
network
services
on
network
elements
to
support
a
VM
(i.e.
DNS,
DHCP,
LB,
VPN,
Port
Forwarding,
etc)
• DeploymentPlanner
–
Different
algorithms
to
place
a
VM
and
volumes.
• Inves4gator
–
Ways
to
find
out
if
a
host
is
down
or
VM
is
down.
• Fencer
–
Ways
to
fence
off
a
VM
if
the
state
is
unknown
• UserAuthen4cator
–
Methods
of
authen4ca4ng
a
user
• SecurityChecker
–
ACL
access
• HostAllocator
–
Provides
different
ways
to
allocate
host
• StoragePoolAllocator
–
Provides
different
ways
to
allocate
volumes

20. High
Availability

21. High
Availability
• Service
Offering
contains
a
flag
for
whether
HA
should
be
supported
for
the
VM
• Does
not
use
the
na4ve
HA
capability
of
hypervisors
• Uses
adapters
to
fine
tune
HA
process

22. Triggering
High
Availability
VM
HA
are
triggered
via
the
following
methods:
• VM
Sync
detects
out
of
band
VM
changes
• Resource
Management
detects
that
a
resource
is
unreachable
and
its
state
can
not
be
determined.
• VM
start/stop
has
been
sent
to
the
resource
but
resource
does
not
return
• Details
of
how
high
availability
is
done
is
at
hip://docs.cloudstack.org/CloudStack_Documenta4on/Design_Documents/CloudStack_High_Availability_-­‐
_Developer's_Guide

23. Scalability

24. Current
Status
• 10k
resources
managed
per
management
server
node
• Real
produc4on
deployment
of
tens
of
thousands
of
resources
• Internal
tes4ng
with
socware
simulators
up
to
30k
physical
resources
with
300k
VMs
managed
by
4
management
server
nodes

25. CloudStack
vs.
OpenStack
vs.
Eucalyptus

26. CloudStack
• Mainly
wriien
in
Java
•
ASL2.0
license
• Has
more
than
100
produc4on
clouds
(Around
May,
2012)
• Support
private/hybrid/public
cloud
• Scale
to
30K
physical
host
in
commercial
environment
• Support
XenServer/Vsphere/KVM/OVM/Baremetal
as
hypervisor
• Mul4ple
geographically
distributed
datacenters
management
• Flexible
and
rich
network
func4onality
• Easy
installa4on
and
management
• Amazon
EC2
API
compa4ble
• Well
documented
• Ac4ve
community

27. OpenStack
• Mainly
wriien
in
Python
•
ASL2.0
license
• Support
private/hybrid/public
cloud
•
Immature
for
commercial
usage
•
Support
XenServer/Vsphere/KVM/Xen/Hyper-­‐V
as
hypervisor
• Network
is
single
point
of
failure
• Weak
VPN
support
for
enterprise
hybrid
cloud
• All
inter-­‐module
communica4on
are
based
on
Message
Queue
• Not
well
documented
• A
bit
hard
to
install
• Amazon
EC2
API
par4ally
compa4ble

28. Eucalyptus
(Open
Source
edi4on)
• Mainly
wriien
in
Java
• GPLv3
license
• Focus
on
private
cloud
• Support
KVM/Xen
as
hypervisor
• Fully
compa4ble
with
Amazon
EC2
• Fully
compa4ble
with
Amazon
S3
via
Walrus
• Both
web
UI
and
command
line
tools
for
cloud
administra4on
• Well
documented
• Difficult
to
gepng
started

29. CloudStack
Par4cipa4on

30. Par4cipate
Incubator
Projects
• As
a
user
• Who
uses
the
socware
• Ac4ve
user
communi4es
• Get/Provide
help
• As
a
developer
• Help
to
develop
the
socware
• Known
as
contributor,
Be
a
coder,
documenter
or
answerer
• The
first
step
to
be
a
commiier
• As
a
commi8er
• Write
access
to
source
repo
• Vote/nominate
developers
(PMC)
• As
a
Mentor
• Informal:
Familiar
with
open
source/apache
knowledge
• Formal:
Self-­‐introduc4on,
volunteer
their
services
to
general@incubator.apache.org
during
development

31. Apache
CloudStack
–
Gepng
Started
• Homepage
hip://incubator.apache.org/cloudstack/
Apache
Incubator
Site
hip://incubator.apache.org/projects/cloudstack.html
• Tracker
hips://issues.apache.org/jira/browse/CLOUDSTACK
• Jenkins
hip://jenkins.cloudstack.org/
• Git
hips://git-­‐wip-­‐us.apache.org/repos/asf/incubator-­‐cloudstack.git
• wiki:
hips://cwiki.apache.org/CLOUDSTACK/
• Freenode
IRC:
#cloudstack;
#cloudstack-­‐dev;
#cloudstack-­‐mee4ng

32. CloudStack
Mailing
Lists
• cloudstack-­‐users@incubator.apache.org
• For
Users
&
Administrators
• Subscrip4on:
cloudstack-­‐users-­‐subscribe@incubator.apache.org
• cloudstack-­‐users-­‐cn@incubator.apache.org
• For
Users
&
Administrators
• Subscrip4on:
cloudstack-­‐users-­‐cn-­‐subscribe@incubator.apache.org
• cloudstack-­‐dev@incubator.apache.org
• CloudStack
developments
discussion
center
• Subscrip4on:
cloudstack-­‐dev-­‐subscribe@incubator.apache.org
• cloudstack-­‐commits@incubator.apache.org
• Automa4c
commit
message
from
CloudStack’s
SCM
• Subscrip4on:
cloudstack-­‐commits-­‐subscribe@incubator.apache.org

33. Thank
you
!