Slides from my talk at the South East Linux Fest about building an IaaS dev/test cloud with Apache CloudStack

1. Building a Dev/Test cloud with
Apache CloudStack
David Nalley
PMC Member Apache CloudStack
Member, Apache Software Foundation
ke4qqq@apache.org
Twitter: @ke4qqq

2. #whoami
• Apache Software Foundation Member
• Apache CloudStack PMC Member
• Recovering Sysadmin
• Fedora Project Contributor
• Zenoss contributor
• Employed by Citrix in the Open Source Business Office

3. Why use cloud?
From a dev point of view the process looks like:
• Start new project
• File ticket for resources....wait....wait....wait
• Get resources, that aren't configured....wait...
• Get network access.....get permission....wait
• Get things done.

4. Why use cloud?
• What IT Ops provides is not what developers want.

5. Get rid of the waiting!
●
Remove constraints - developers empowered to get
things done.
●
Agility
●
Enforce automated process instead of manual ones

6. What does a dev/test cloud look like?
●
Self-service - developers can provision their own
environments
●
Usage measurement - we worry about VM sprawl
●
Isolated networks - must not let dev/test interfere with
the real world.
●
Commodity - as cheap as practical
●
May also house production workloads

7. Self service
●
Provisioning manually doesn't add value
●
Can be completely automated
●
Do they need full control or just pushing pre-configured
environments?

8. Self service
●
UI
●
API
●
Some external tool

10. API or Command-line
� cloudmonkey> deploy virtualmachine
serviceofferingid=d8611d07-acf5-4cd4-a630-5c4d937ef043
templateid=081358ff-2427-44f8-adcc-1bb002fab361
zoneid=d06193b2-7980-4ad1-bd8-7b2f2eda63c3
curl 'http://localhost:8096/client/api?
command=listUsers'

11. Config Management deployment

12. {
"name": "hadoop_cluster_a",
"description": "A small hadoop cluster with hbase",
"version": "1.0",
"environment": "production",
"servers": [
{
"name": "zookeeper-a, zookeeper-b, zookeeper-c",
"description": "Zookeeper nodes",
"template": "rhel-5.6-base",
"service": "small",
"port_rules": "2181",
"run_list": "role[cluster_a], role[zookeeper_server]",
"actions": [
{ "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
]
},
{
"name": "hadoop-master",
"description": "Hadoop master node",
"template": "rhel-5.6-base",
"service": "large",
"networks": "app-net, storage-net",
"port_rules": "50070, 50030, 60010",
"run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
},
{
"name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
"description": "Hadoop worker nodes",
"template": "rhel-5.6-base",
"service": "medium",
"port_rules": "50075, 50060, 60030",
"run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
"actions": [
{ "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
{ "http_request": "http://${hadoop-master}:50070/index.jsp" }
]
}
}

13. Use a tool

14. Usage
● Jevons Paradox
● Plenty of waste possible as well - will developers always
destroy a machine when they are done with it?
● Important to show what projects and groups are
consuming resources as well as how they are using
those resources

15. Commodity Storage
● Commodity storage - this is dev/test environment - high
performance, resilient storage isn't needed.
● Local storage tends to be the best mix of cheap and
performant
● No failover, but it's dev/test - do you need it?

16. Commodity Networking
● Layer 3 isolation - (aka Security Groups)
● VLANs - (not as commodity, but still relatively cheap on
a small scale, but not at a large scale)
● Virtual routers (provide DHCP, DNS, LB, Firewall, PF,
NAT, etc)

17. Commodity Hypervisor
● KVM is my personal choice in this space.
● Easiest to consume - completely open source

18. Limiting Resources
● Limit the number of VMs, snapshots, IP addresses, etc.
● Use 'projects' to share resources
● This means most folks will never have problems, but
heaviest users will not be able to interrupt service for
others.

19. Questions

20. Resources
●
http://cloudstack.apache.org
●
#cloudstack on irc.freenode.net
●
http://cloudstack.apache.org/docs