2. Deploying & Managing a Cloud is not simple.
• Deploying to physical gear on layered networks
• Multiple interlocking projects
• Hundreds of configuration options
• An aggressive 6 month delivery cycle
• Constant innovation on core technologies
It does not have to be hard!
3. Problem: Clouds do not operate themselves!
Operations
Servers
Networks
Storage
Etc.
Expertise
4. Ops automation (“devops”) jumpstarts learning,
reduces complexity and manages scale.
Operations
Automation
ic
(DevOps)
ur Top
O
Servers
Networks
Storage
Etc.
Operations
Expertise
6. Foundation for Success: Open Operations
‣ Start with “Operations as Code”
‣ Leverage the community
‣ Write Operations code so it can be openly shared
‣ Learn from other people’s experience
‣ Make operations a priority
‣ Engage developers in operations challenges
‣ Make deployability a dev & test concern
‣ These are central issues to Opscode & Dell
7. What is Chef?
‣ An Open Source automation platform to continuously
define, build, and manage infrastructure.
‣ Chef enables people to easily build & manage complex &
dynamic applications at massive scale
‣ Built on the idea of Infrastructure as Code
‣ Programmatically provision and configure nodes
‣ Reconstruct business from code repository, data backup, and
bare metal resources.
8. How Can Chef Help?
Blueprint Your Build Anything… And Manage It Simply
Infrastructure
Provision compute • Introduce continuous
• Compute resources in the Data incremental change or total
change.
• Application Center and the Cloud
• Infrastructure • Automatically reconfigure
• Storage everything
• Security • Application Stacks • Re-provision for disaster
• Network • Big Data recovery
• Configuration Standards • HPC • Fail-over to bare metal
• IaaS • Monitor for compliance
• PaaS • Cloud migrations become
Using 1,000’s of man- trivial
• SaaS
days of prior art!
10. The Path to Full Automation
Full
Continuous Infrastructure
Deployment
Automation
Application
Configuration Management
Common
Discovery and Management
Automation
Visibility
Tasks:
Scripts, OS
Compliance,
Updates &
Patches
13. Clouds are a complex: not like an app
‣ Building up OpenStack requires a system view
‣ Breaking projects into discrete cookbooks enables maintainability
‣ Conventions are needed so that projects and cross reference each other
‣ Networking is critical
‣ Each component requires correct VLAN, teaming and virtual bridge configuration.
‣ Each site has unique networking needs (IP, subnet, VLAN)
‣ NIC enumeration can break even the simplest scipts
‣ Hardware configuration is boring but very important
‣ Application choices force configuration changes (late binding)
‣ Small mistakes can be very expensive to find and fix
‣ We needed a way to make deploy simpler, faster & repeatable
14. OpenStack Installer: Dell Crowbar Project
‣ Field Proven Technologies
‣ Build on locally deployed Chef Server, Raw servers to full cloud in <2
hours, Multi-Node from the ground up, Hardened with over a year of
real deployments
‣ Apache 2 Open Source (on github/dellcloudedge)
‣ Multi-Apps (OpenStack & Hadoop), Multi-OS (Ubuntu, RHEL, CentOS,
SUSE), Development “in the open” w/ community engagement
‣ OpenStack Releases (dell.com/OpenStack)
‣ Cactus, Diablo and Essex!
‣ Not Dell Hardware Limited
15. Crowbar Builds Up Clouds in layers
APIs, User Access, &
Ecosystem Partners
Ops Management
Dell “Crowbar”
Cloud Infrastructure
Core Components &
Operating Systems
Physical Resources
17. 2) OS and Ops Infrastructure
APIs, User Access, &
Ecosystem Partners
Ops Management
Dell “Crowbar”
Cloud Infrastructure
Core Components &
Operating Systems *
Physical Resources
* Suse pending, based on open source activity
18. 3) Cloud Platform Integration
APIs, User Access, &
Ecosystem Partners
Ops Management
Dell “Crowbar”
Cloud Infrastructure
Core Components &
Operating Systems *
Physical Resources
* Suse pending, based on open source activity
20. OpenStack Orchestration: Chef & Crowbar
Status (po
st)
Orchestration
State Machine
OpenStack Components
Ops Management
Crowbar Magic Happens!
nt
Clie
Dell “Crowbar”
f-
Che
Run List
Chef-Client
Network Configuration
Chef
Server Chef-C
lient
Operating System
Config.
Che
f-C
lien
t
Hardware Config
(uses “Sledgehammer”)
DHCP
Server(s)
t
PXE Boo
21. Crowbar & OpenStack Cookbooks
‣ Having cookbooks is not enough, Crowbar also
‣ Understands how the cookbooks connect together
‣ Helps you plan out the networking & server allocations
‣ Crowbar has a consistent Chef design approach
‣ “Proposal” databags to setup attributes and run-lists
‣ Consistent naming conventions
‣ Roles to link things together – A LOT OF ROLES
‣ Crowbar adds many cookbooks for network,
monitoring, provisioning & hardware config.
22. Chef for OpenStack
‣ Community for the automated deployment and
management of OpenStack
‣ opscode.com/openstack/
‣ Cookbooks
‣ Documentation
‣ Mailing list
‣ groups.google.com/group/opscode-chef-openstack
‣ IRC
‣ #openstack-chef on irc.freenode.net
23. Chef for OpenStack Goals
Already a lot of OpenStack deployments managed with Chef
‣ Provide a stable set of core OpenStack cookbooks
‣ Document how the pieces work together
‣ Evolve to keep up with latest changes
‣ Expand support into new technologies
‣ Operating systems, databases, hypervisors, etc.
‣ Alternate storage types
‣ Alternate networking implementations for Quantum
24. Chef for OpenStack Roadmap
‣ Get stable Essex cookbooks out
‣ Establish patterns of collaboration
‣ Continuous Integration and gated testing
‣ Folsom milestones
‣ Documentation, documentation, documentation
25. Upstreaming accelerates strong deploys
‣ Feedback loops enable cookbooks to mature quickly
Community Cookbooks
Dell Crowbar
Chef
Cloud A
Host B
Crowbar Dell Cloud A
Chef Community
Community
Customers
Customers
26. Infrastructure as Code Repositories
‣ Deploy your data center with Crowbar
‣ Store this in version control
‣ Test your changes
‣ Promote to production
‣ Rinse, repeat
‣ OpenStack deployment is managed with Chef
‣ See above
‣ Deploy your applications on OpenStack with Chef
‣ See above
27. Deploy Your Applications on OpenStack
‣ Nova Dashboard
‣ Knife OpenStack
‣ enStratus
‣ Hybrid Cloud!?
29. knife-openstack
$ knife openstack
Available openstack subcommands: (for details, knife SUB-
COMMAND --help)
** OPENSTACK COMMANDS **
knife openstack flavor list (options)
knife openstack image list (options)
knife openstack server create (options)
knife openstack server delete SERVER [SERVER] (options)
knife openstack server list (options)
30. knife-openstack
knife openstack server create --flavor 1 --image 7
-S keypair -r 'role[hadoop-worker]’
‣ Deploy applications on top of OpenStack
‣ Manage them with Chef
‣ Configuration backed by version control
‣ Transfer your infrastructure anywhere
31. Challenges
‣ Making Community work sharable
‣ Testing / Validation
‣ Community
‣ Keeping on Upstreams
‣ Automation (Tempest & CI)
‣ Linking Dev and Ops
‣ Upgrade Process
‣ # of configurations
32. Join the Community & Get Started
‣ Invest in reusing community code
‣ Respect abstractions & patterns from community code
‣ Participate in communities
‣ Submit bugs (and patches!)
‣ Use Upstreams!
‣ Community Cookbooks… Opscode
‣ community.opscode.com
‣ Deploy Infrastructure… Dell Crowbar
‣ github.com/dellcloudedge/crowbar
33. Thanks!
Rob Hirschfeld
‣ rob_hirschfeld@dell.com
‣ Twitter: @zehicle
‣ Blog: robhirschfeld.com
Matt Ray
‣ matt@opscode.com
‣ Twitter/GitHub/IRC: @mattray
‣ Blog: leastresistance.net