Introduction on open source technologies that can be used to deploy and manage cloud computing environments. Especially geared toward Infrastructure-as-a-service environments. Updated for presentation at Indiana Linuxfest (3/26/2011). Updates: - Open source cloud storage (CEPH, Swift, Gluster) - Orchestration - MCollective - Cloud Infrastructure Diagrams

1. Crash Course in
Open Source
Cloud Computing
Mark R. Hinkle
VP of Community
Cloud.com
Twitter: @mrhinkle
Email: mrhinkle@gmail.com

2. 2
%whoami
• Responsible for Driving Adoption of
CloudStack Open Source Cloud Computing
Software
• Former manager of Zenoss Open Source
project 100,000 users, 1.5 million downloads
• Former Linux Desktop Advocate (Zealot?)
• Former LinuxWorld Magazine Editor-in-Chief
• Open Management Consortium Conspirator
• Open Desktop Consortium Instigator
• Author - “Windows to Linux Business Desktop
Migration” - Thomson
• NetDirector Project - Open Source
Con guration Management Project
• Sometimes Author and Blogger at
SocializedSoftware.com/NetworkWorld
• Start-up junkie, Glutton for punishment

3. 3
Agenda
• Cutting Through the Cloud Computing
Hype
• Open Source Building Blocks for Cloud
Computing
• Open Source Tools for Cloud
Management
• Questions (and Hopefully Answers)

4. Cloud Computing:
Cutting through
the Hype

5. 5
Cloud Computing Adoption
• 365,000 web sites are running on Amazon
EC2 - Netcraft May 2010
• £76 billion spent on Cloud Computing in
the U.K. in 2010 - IT Candor
• Cloud Computing will be a $126 billion
market by 2012 - IBM
• 20% of Businesses won’t have IT Assets
by 2012 - Gartner
• December 2010,Amazon Web Services
housed 262 billion objects, more than
doubling in size from 102 billion objects
at the close of 2009 - Amazon

6. 6
Open Source Virtual Adoption
• 41% of users prefer to deploy servers virtually, 29.3%
use virtualization whenever possible
• Xen is 2nd most popular virtualization technology after
VMware followed by Linux Kernel-Based Virtual
Machines (KVM) with 21.3% of the market
• 49.2% of users planed to deploy hosted Linux servers
in 2010 while 32.6% indicated that they would be
deploying hosted Microsoft Windows.
• 50.8% indicated they used no specific management
tools for cloud computing, 33.3% indicated using tools
provided by their hosting provider
Source: Zenoss Cloud Computing State of the Union - http://www.zenoss.com/in/virtualization_survey.html

7. 7
Five Characteristics of Clouds
1. On-Demand Self-Service
2. Broad Network Access
3. Resource Pooling
4. Rapid Elasticity
5. Measured Service

8. 8
Cloud Computing Service Models
USER CLOUD a.k.a. SOFTWARE AS A SERVICE
Single application, multi-tenancy, network-based, one-to-many delivery of
applications, all users have same access to features.
Examples: Salesforce.com, Google Docs, Red Hat Network/RHEL
DEVELOPMENT CLOUD a.k.a. PLATFORM-AS-A-SERVICE
Application developer model, Application deployed to an elastic service that
autoscales, low administrative overhead. No concept of virtual machines or
operating system. Code it and deploy it.
Examples: Google AppEngine, Windows Azure, Rackspace Site, Red Hat
Makara
SYSTEMS CLOUD a.k.a INFRASTRUCTURE-AS-A-SERVICE
Servers and storage are made available in a scalable way over a network.
Examples: EC2,Rackspace CloudFiles, OpenStack, CloudStack,
Eucalyptus, Ubuntu Enterprise Cloud, OpenNebula

9. Deployment Models 9
Public,Private & Hybrid Clouds

10. 10
Benefits of Cloud Computing
Public Cloud Private Cloud
• Pay for What You Need • Full control of
• Numerous Geographical infrastructure
Options • Security (if you are good)
• Virtually unlimited • Automation Capabilities
capacity • Flexibility, Flexibility,
• Op-Ex versus Cap-Ex Flexibility
• Automation Capabilities

11. Building Compute Clouds
with Open Source Software

12. 12
Open Source Hypervisors
Open Source
• Xen Cloud Platform (XCP)
• KVM – Kernel-based Virtualization
• VirtualBox* - Oracle supported Virtualization Solutions
• OpenVZ* - Container-based, Similar to Solaris Containers or BSD Zones
• LXC – User Space chrooted installs
Proprietary
• VMware
• Citrix Xenserver
• Microsoft Hyper-V

13. 13
Open Source Compute Clouds
Year Started License Hypervisors
Supported
2010 (Development GPL Xenserver, VMware,
Since 2008) KVM, Hyper-V
2008 GPL Xen, KVM, VMware
2010 (Developed by Apache Xen, KVM, Hyper-V
NASA by Anso Labs
peviously)
Ubuntu 2009 (Karmic Koala) GPL Xen,KVM
Enterprise
Cloud
2009 (Development LGPL v3 VMware ESX and ESXi,
2006) Microsoft Hyper-V, Xen,
KVM and Virtual Box

14. Open Source 14
Cloud Computing Storage
• GlusterFS – Scale Out NAS system agregating storage
over Ethernet or Infiniband
• CEPH – Distributed file storage system
• OpenStack Storage (Swift) – Long-term storage object
storage system
• Sheepdog – Distributed storage for KVM hypervisors
• NFS – Old standby, tried and true, not designed for
cloud scale or performance

15. Cloud APIs Aren’t Created Equal
15
Open Source Abstractions
• jclouds
• libcloud
• deltacloud

16. 16
Private Cloud Architecture

17. Managing Clouds with
Open Source Tools

18. 18
Cloudy Tools
• Network Capable
• Cloud “Aware”
• Easy-to-Integrate
• Adhere to Open Standards
• Lend Themselves to Automation

19. 19
The Myth of the Nines
Availability % Downtime per Downtime per Downtime per
Year Month Week
99.9% (three nines) 8.76 hours 43.2 minutes 10.1 minutes
99.95% 4.38 hours 21.56 minutes 5.04 minutes
99.99% (four nines) 52.6 minutes 4.32 minutes 1.01 minutes
99.999% ( ve nines) 5.26 minutes 25.9 minutes 6.05 minutes
99.9999% (six nines) 31.5 seconds 2.59 seconds .0605 seconds
Average polling interval for monitoring? 5 minutes?
Even superhuman operations people can’t be alerted and take action in under 5 minutes.
One outage per year could drop service level to three nines or worse.

20. Automation Unlocks 20
the Potential of the Cloud
• MeatCloud, Can’t
Keep up with Cloud
Computing
• Devops & Agile IT
Philosophy
• Script Repetitive Tasks
• Automate, Automate,
Automate

21. 21
4 Types of Management Tools
Provisioning
Installation of operating systems and other software
Configuration Management
Sets the parameters for servers, can specify
installation parameters
Orchestration/Automation
Automate tasks across systems
Monitoring
Records errors and health of IT infrastructure

22. 22
Management Toolchains
Toolchain (n):
Monitoring
A set of tools where
the output of one Patching and
Provisioning
tool becomes the
input of another tool Con guration

23. Open Source 23
Provisioning Tools
Year Started Language License Installation
Targets
Cobbler 2007 Python GPL Red Hat,
OpenSUSE
Fedora, Debian,
Ubuntu
Fully Automatic 2000 Perl GPL Debian
Installation (FAI)
Kickstart ? Python GPL Most .dep and
RPM based Linux
distros
Spacewalk 2008 Perl, Python, Java GPL Fedora, Centos
Viper 2008 Perl GPL Debian

24. Open Source 24
Configuration Management Tools
Year Started Language License Client/Server
Bcfg2 2003 Python BSD Yes
Cfengine 1993 C Apache Yes
Chef 2009 Ruby Apache Chef Solo – No
Chef Server - Yes
Puppet 2004 Ruby GPL yes

25. Open Source 25
Monitoring Tools
Year Started License Language Type of Collection
Monitoring Methods
Cacti 2001 GPL PHP Performance SNMP, syslog
Nagios 1999 GPL C/PHP Availability SNMP,TCP, ICMP,
IPMI, syslog
OpenNMS 2000 GPL Java Availability/ SNMP,
Performance
Zabbix 2001 GPL C/PHP Availability/ SNMP, TCP/ICMP,
Performance and IPMI, Synthetic
more Transactions
Zenoss 2005 GPL Python Availability, SNMP, ICMP, SSH,
Performance, Event syslog, WMI
Management

26. Open Source 26
Automation/Orchestration Tools
Year Language License Client/ Support
Started Server Organization
AutomateIT 2009 Ruby GPL No None
Capistrano 2006 Ruby MIT Yes None
RunDeck 2010 Java Apache Yes DTO Solutions
Func 2007 Python GPL Yes Fedora Project
MCollective 2009 Ruby Apache Yes PuppetLabs

27. 27
Automated Toolchain
Capistrano
Command and Application Service RunDeck
Fabric
Control Orchestration Func
Provisioning Activity
BCFG2
Con guration Cfengine
System Con guration Chef
Puppet
Cloud: OS Install:
Eucalyptus Kickstart
Cloud Image OS OpenStack Cobbler
Bootstrapping
Launch Install CloudStack Spacewalk
Abiquo

28. Questions?

29. 29
Contact Me
Professional: mrhinkle@cloud.com
Personal: mrhinkle@gmail.com
Professional: mrhinkle@cloud.com
Personal: mrhinkle@gmail.com
Professional: http://open.cloud.com Mark R. Hinkle
Personal: http://socializedsoftware.com VP of Community
Open Source Fanboy
Twitter: @mrhinkle

30. Crash Course in Cloud Computing
by Mark R. Hinkle is licensed under a
Creative Commons Attribution-
ShareAlike 3.0 United States License.