Call Girls In Pratap Nagar Delhi šÆCall Us š8264348440š
Ā
Rackspace Private Cloud presentation for ChefConf 2014
1. 1
April 16, 2014
Ops and Chefs: Lessons learned while working on
OpenStack deployment cookbooks
#chefconf 2014
2. RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
Who Are We?
Justin Shepherd
Principal Architect
Rackspace Private Cloud
github.com/galstrom21
Joseph Breu
Software Dev Team Lead US
Rackspace Private Cloud
github.com/rackerjoe
@rackerjoe
3. 3RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
Support Team
ā¢Manages and Operates OpenStack clouds for our customers
ā¢Troubleshoots any issues a customer might encounter with their cloud
Dev Team
ā¢Creates tools for deploying and configuring OpenStack clouds
ā¢Makes initial decision on when a new project is āstableā enough to
enable in the customer clouds
A little context
4. 4RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
In the beginningā¦.
The Dev Team were experts at deployments
Best Practices from the Ops Team
5. 5RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
Over timeā¦.
The Ops team took the reigns and became the experts.
New expertise breeds new Best Practices!
Best Practices from the Ops Team
6. 6RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
In the endā¦.
How do you codify this new knowledge?
Best Practices from the Ops Team
7. 7RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
Nova
ā¢ Has 1000+ configuration options
ā¢ has support for 4 hypervisor platforms
Neutron
ā¢ has support for ~15 network platforms
Cinder
ā¢ has support for ~20 storage platform
Flexibility vs. Standardization
8. 8RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
Limit platform options based on current expertise
ā¢ We chose KVM because it had the best support in the
community, and we have lots of Linux expertise in house
ā¢ We chose NetApp initially and leveraged our existing storage
team
Add more options as needed based on demand
ā¢ We need compelling reasons to add features
Each addition increases QE test matrices exponentialy!
Flexibility vs. Standardization
9. 9RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
Even limiting scope it takes 61,480 lines of cookbook to implement a
working OpenStack cluster
Flexibility vs. Standardization
10. 10RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
Issues with cookbooks
ā¢ Setting an incorrect value in novaās config file
ā¢ Absence of a useful attribute
ā¢ Cookbook dependencies and upstream breakage (COOK-2676)
Issues in the upstream OpenStack projects
ā¢ Hopefully these are known issues, and the community is addressing them
Issues in the upstream OS packages
ā¢ Vendor X breaks a package (i.e. kernel and openvswitch)-where X is all of them!
Issues in foundation technologies
ā¢ MySQL
ā¢ RabbitMQ
Issue Tracking
11. 11RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
The usual suspects
ā¢ Ubuntu LTS
ā¢ RHEL
ā¢ CentOS
The usual problems
ā¢ Independent packaging process
ā¢ Separate bug tracking process
ā¢ Packages are cut at different points in time
Multiple Base Operating Systems
12. 12RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
OpenStack releases a new āstableā version every 6 months.
ā¢ There are also 3 scheduled minor version releases for the stable
version during the next 6 month cycle.
ā¢ Although none of the upstream packages match these releases
Deciding on your Release Schedule
13. 13RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
We support the current āstableā version and one prior
ā¢ So our customers do not have to update every 6 months
ā¢ Upgrades are currently disruptive
We currently release minor version updates every other month
ā¢ Includes cookbook bug fixes
ā¢ Includes latest available packages
Deciding on your Release Schedule
14. 14RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
This is probably one of the easiest decision to make and the
hardest to get right
ā¢ We have changed our model multiple times in the last 3 years
ā¢ We will probably change it again ļ
Go ask @claco Thursday 2:50pm Regency A/B about this
Deciding on your Release Schedule
15. 15RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
Major changes can occur every 6 months
ā¢ Package renames
ā¢ Service renames
ā¢ New/Deprecated config options
Services get split into their own projects
ā¢ Nova-Volumes -> Cinder
ā¢ Nova-Network -> Neutron (kind of)
ā¢ Nova-Scheduler -> Gantt
Managing Chef deployed OpenStack
16. 16RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
Inter-release upgrades
ā¢ Sweeping attribute key renames
ā¢ node[āquantumā] -> node[āneutronā]
ā¢ Have to remap customers environments
ā¢ https://github.com/rcbops/mungerator
ā¢ New BUGS!
ā¢ That never happens.. it is a new version.. it must be a feature
OS Upgrades
ā¢ Terrible, terrible, and more terrible
Managing Chef deployed OpenStack
17. 17RACKSPACEĀ® HOSTING | WWW.RACKSPACE.COM
Jenkins is used for testing and gating of our cookbooks
ā¢ Syntax Verification (pep8, foodcritic)
ā¢ Full deployment of OpenStack utilizing the cookbooks with the
proposed changeset applied
ā¢ Functional Tests of OpenStack
ā¢ OpenStack API Testing
ā¢ OpenStack CLI Testing
Upstream opscode cookbooks are tested before inclusion
Testing