Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
From chef09 to chef11 
my approach to a real devops task 
Antonio Peña @apenav for @MadridDevops 
September 2014 http://ab...
http://madrid.devops.es
Motivation 
● feedback 
● sharing experience 
● learning 
● talking about devops
Impostor Syndrome
Three Parts 
● Devops 
● Chef 
● Migration
About 
DevOps
Software is broken or will be
Devops (I)
Devops (II)
Devops (III)
So little time, so much to talk about
Big Business. DevOps Myths (I)
Big Business. DevOps Myths (II)
My approach to a 
configuration management project 
Antonio Peña @apenav for @MadridDevops 
September 2014 http://about.me...
Best tool to make config mgmt? 
● cfengine 
● puppet 
● chef 
● salt 
● ansible 
-- Business decision-making. Make your ch...
About 
Chef
What is Chef? 
● a company 
○ Opscode Inc. 
○ getchef.com 
○ @chef 
● a product (with paid support) 
○ packages, not recom...
Beginners, experts and middle class
Berkshelf.I
Berkshelf.II
“Spanish” Way 
Master Chef RTVE
Howto devops in Spain? (I) 
● e.g. Demo in a Big Company
Howto devops in Spain? (II) 
● startup 
● why not? 
● sandbox
Howto devops in Spain? (III) 
● consulting
Working in a company 
as a Developer. 
Daily tasks
Initial problem 
Problem to be solved (in aprox. 2 weeks): 
1. install chef-server 11 in a centos6 behind a 
VPN 
2. insta...
Real problem 
1. refactor/migrate/write from scratch about 20 
old-fashioned (2009) but very reliable (daily 
used, very d...
From scratch 
http://devopsreactions.tumblr.com/post/97637875636/what-happens-when-i-am-allowed-to-rewrite-code-from
Methodology or the like 
“Big bang” approach, but iterative: 
● use chef11 community cookbooks 
● almost no direct migrati...
Criteria to use community cookbooks? 
https://github.com/opscode-cookbooks/mysql 
When in doubt use popular 
opscode 
comm...
About 
Migration
To the kitchen
Let’s cook! 
1. Hosting Environment 
2. Cookbook Development 
3. Monolithic cookbook approach 
4. Everything is a cookbook...
Chef Server 
Internals
1.- Hosting Environment (I) 
Open Source chef-server in a centos6. 
Modify and test undocumented chef-server 
config, with...
Asking for opening ports in firewall 
http://devopsreactions.tumblr.com/post/41094252078/asking-the-security-team-for-a-fi...
2.- Cookbook Development (I) 
Opscode free(*) hosting 
● Great help because Multiproject! 
● http://learn.getchef.com 
● B...
2.- Cookbook Development (II) 
Chef-DK: 
● ruby binaries in specific /opt folders 
● not needed rvm/rbenv/ruby source code...
2.- Cookbook Development (III) 
● http://jtimberman.housepub.org/blog/2014/04/30/chefdk-and- 
ruby/ 
● Berkshelf 3.0. 
● T...
2.- Cookbook Development (IV) 
Better practices? Any? 
● No time for tests nor TDD (to learn about) 
● Not enough RAM in l...
2.- Cookbook Development (V) 
First stages of development 
● make risks, quick tests, quickwin, try and try 
● think in ad...
3.- Monolithic cookbook approach 
Thanks a lot to Mathias Lafeldt @mlafeldt 
“monolithic cookbook” idea, easy to refactor ...
4.- Everything is a cookbook 
● https://tomduffield.com/everything-as-a-cookbook- 
chefconf2014/ 
● https://speakerdeck.co...
Cool guys don’t look back to explosions?
5.-ProductionCookbookDevelopment 
● A new beginning. Almost from scratch. 
Develop new recipes and cookbooks against a 
ne...
6.-ProductionCookbookDeployment 
● chef site cookbook install “cookbook” (new 
git branch in ~/chef-repo/cookbook/) 
● che...
7.- Opscode packages (I) 
● WARNING: chefdk.rpm installs binaries 
under /usr/bin and gems in its own gemsdir 
● REMOVE an...
7.- Opscode packages (II) 
NO RVM nor system ruby allowed 
● chef-server.rpm 
● chef.rpm 
● chefdk.rpm 
every opscode rpm ...
8.- Developers isolated ruby environment (I) 
● Developers need multiple rubies installed 
(1.9.3 & 2.0.X) 
● unprivileged...
8.- Developers isolated ruby environment (II) 
Opscode chef.rpm package provides 2.1 ruby 
● install passenger-apache libr...
8.- Developers isolated ruby environment(III) 
Explicit ruby version (1.9.3 or 2.0.X) in 
appropriate virtualhost apache f...
9. Final Steps (I) 
After just 2 weeks, it almost worked ok. 
Traditional approach to development: logrotate 
and monit fo...
9. Final Steps (II) 
Pareto principle! 80/20 Rule. 
● Two more weeks with “fringes” 
● create new organization in opscode ...
Angry chef?
9. Final Steps (III) 
Create git repo (first commit in 2 weeks) 
● install explicit dependencies 
● “chef cookbook site in...
9. Final Steps (IV) 
“In extremis”: Refactor the “monolithic” 
cookbook 
Monolithic cookbook refactored as 3 cookbooks 
Th...
9. Final Steps (V) 
● only two “old fashioned” forked recipes: 
monit and passenger 
● new recipes forces a different way ...
10. Conclusions (I) 
To FIX: 
● move logrotate recipes in railssites2 to 
another new bootstrapping-node-sysadmin 
cookboo...
10. Conclusions (II) 
● You need to have “Luck” 
○ no way to configure staging chef-server (only 
working through 127.0.0....
10. Conclusions (III) 
To be improved: 
● ubuntu/RHEL cookbooks 
● tests 
● mysql m/s cookbook 
● monit recipe 
● opensour...
THANKS FOR YOUR 
PATIENCE
MadridDevops September 2014: "From chef09 to chef11, one approach to devops"
MadridDevops September 2014: "From chef09 to chef11, one approach to devops"
MadridDevops September 2014: "From chef09 to chef11, one approach to devops"
MadridDevops September 2014: "From chef09 to chef11, one approach to devops"
MadridDevops September 2014: "From chef09 to chef11, one approach to devops"
Nächste SlideShare
Wird geladen in …5
×

MadridDevops September 2014: "From chef09 to chef11, one approach to devops"

701 Aufrufe

Veröffentlicht am

MadridDevops September 2014 Meeting: "From chef09 to chef11, one approach to devops"

Madrid, September 25, 2014

http://madrid.devops.es

Just an excuse to talk about devops, chef (and puppet as comparison), its culture , its community and a migration project I have been involved with.

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

MadridDevops September 2014: "From chef09 to chef11, one approach to devops"

  1. 1. From chef09 to chef11 my approach to a real devops task Antonio Peña @apenav for @MadridDevops September 2014 http://about.me/apenav
  2. 2. http://madrid.devops.es
  3. 3. Motivation ● feedback ● sharing experience ● learning ● talking about devops
  4. 4. Impostor Syndrome
  5. 5. Three Parts ● Devops ● Chef ● Migration
  6. 6. About DevOps
  7. 7. Software is broken or will be
  8. 8. Devops (I)
  9. 9. Devops (II)
  10. 10. Devops (III)
  11. 11. So little time, so much to talk about
  12. 12. Big Business. DevOps Myths (I)
  13. 13. Big Business. DevOps Myths (II)
  14. 14. My approach to a configuration management project Antonio Peña @apenav for @MadridDevops September 2014 http://about.me/apenav
  15. 15. Best tool to make config mgmt? ● cfengine ● puppet ● chef ● salt ● ansible -- Business decision-making. Make your choice -- Strengths/weakness. Tend to converge.
  16. 16. About Chef
  17. 17. What is Chef? ● a company ○ Opscode Inc. ○ getchef.com ○ @chef ● a product (with paid support) ○ packages, not recommended install from gemfiles ● an open source project ● a community (open source but not only)
  18. 18. Beginners, experts and middle class
  19. 19. Berkshelf.I
  20. 20. Berkshelf.II
  21. 21. “Spanish” Way Master Chef RTVE
  22. 22. Howto devops in Spain? (I) ● e.g. Demo in a Big Company
  23. 23. Howto devops in Spain? (II) ● startup ● why not? ● sandbox
  24. 24. Howto devops in Spain? (III) ● consulting
  25. 25. Working in a company as a Developer. Daily tasks
  26. 26. Initial problem Problem to be solved (in aprox. 2 weeks): 1. install chef-server 11 in a centos6 behind a VPN 2. install 2 web frontends + 2 backends a. (redis s/m + mysql m/s)
  27. 27. Real problem 1. refactor/migrate/write from scratch about 20 old-fashioned (2009) but very reliable (daily used, very difficult to extend for an outsider) ubuntu ruby18-webapp-related recipes to newer ones 2. proof of concept to migrate chef09 company’ s repository to anything better
  28. 28. From scratch http://devopsreactions.tumblr.com/post/97637875636/what-happens-when-i-am-allowed-to-rewrite-code-from
  29. 29. Methodology or the like “Big bang” approach, but iterative: ● use chef11 community cookbooks ● almost no direct migration from chef09 ones
  30. 30. Criteria to use community cookbooks? https://github.com/opscode-cookbooks/mysql When in doubt use popular opscode community cookbooks. So many content to follow!
  31. 31. About Migration
  32. 32. To the kitchen
  33. 33. Let’s cook! 1. Hosting Environment 2. Cookbook Development 3. Monolithic cookbook approach 4. Everything is a cookbook 5. Production Cookbook Deployment 6. Production Cookbook Development 7. Opscode packages 8. Developers’ isolated ruby environment 9. Final Steps 10. Conclusions
  34. 34. Chef Server Internals
  35. 35. 1.- Hosting Environment (I) Open Source chef-server in a centos6. Modify and test undocumented chef-server config, with some “little” external problems: - shared server behind a VPN - unavailable ports: reassign 80/443 to 81/8443 - 8000 not opened in firewall (so no reports will be available)
  36. 36. Asking for opening ports in firewall http://devopsreactions.tumblr.com/post/41094252078/asking-the-security-team-for-a-firewall-exception
  37. 37. 2.- Cookbook Development (I) Opscode free(*) hosting ● Great help because Multiproject! ● http://learn.getchef.com ● Berkshelf helped a lot with dependencies (**) ● Good practices freezing versions (*) not open-source (limited to 10 nodes) (**) not so easy when not opscode server
  38. 38. 2.- Cookbook Development (II) Chef-DK: ● ruby binaries in specific /opt folders ● not needed rvm/rbenv/ruby source code/... ● own path & pre-installed useful gems and tools
  39. 39. 2.- Cookbook Development (III) ● http://jtimberman.housepub.org/blog/2014/04/30/chefdk-and- ruby/ ● Berkshelf 3.0. ● The Test Kitchen integration testing framework. ● ChefSpec, for unit testing cookbooks. ● Foodcritic, static code analysis on cookbooks. ● All of the Chef tools you're already familiar with: Chef Client, Knife, Ohai and Chef Zero.
  40. 40. 2.- Cookbook Development (IV) Better practices? Any? ● No time for tests nor TDD (to learn about) ● Not enough RAM in laptop nor in AWS micro instances for making testing/CI with Vagrant ● Foodcritic and more, but later ● I hope to use them ASAP: just because I like it. The same with Puppet.
  41. 41. 2.- Cookbook Development (V) First stages of development ● make risks, quick tests, quickwin, try and try ● think in advance, virtualhosts library ● ugly code you know it will be easy to change ● the community code is better than yours ● extend, not create from the ground ● parametrize cookbooks: redis
  42. 42. 3.- Monolithic cookbook approach Thanks a lot to Mathias Lafeldt @mlafeldt “monolithic cookbook” idea, easy to refactor later (see Puppet Module Structure Redux). ● http://mlafeldt.github.io/practicing-ruby-cookbook ● https://github.com/elm-city-craftworks/ practicing-ruby-cookbook
  43. 43. 4.- Everything is a cookbook ● https://tomduffield.com/everything-as-a-cookbook- chefconf2014/ ● https://speakerdeck.com/tduffield/everything-as- a-cookbook-1
  44. 44. Cool guys don’t look back to explosions?
  45. 45. 5.-ProductionCookbookDevelopment ● A new beginning. Almost from scratch. Develop new recipes and cookbooks against a new chef organization in opscode free hosting. ● install, fix, check-apply and repeat in frontend staging instances (Poor’s man plan–do–check–adjust) https://en.wikipedia.org/wiki/PDCA
  46. 46. 6.-ProductionCookbookDeployment ● chef site cookbook install “cookbook” (new git branch in ~/chef-repo/cookbook/) ● chef site cookbook install chef-client
  47. 47. 7.- Opscode packages (I) ● WARNING: chefdk.rpm installs binaries under /usr/bin and gems in its own gemsdir ● REMOVE any puppet or ruby preexistent package or binary, no interferences please! ● BEWARE: never use binaries without explicit PATH e.g. /opt/chef/bin/chef-client
  48. 48. 7.- Opscode packages (II) NO RVM nor system ruby allowed ● chef-server.rpm ● chef.rpm ● chefdk.rpm every opscode rpm package installs one or more ruby binaries in its own path
  49. 49. 8.- Developers isolated ruby environment (I) ● Developers need multiple rubies installed (1.9.3 & 2.0.X) ● unprivileged users will install its own gems, without interfering with chef ruby-binaries SOLUTION: compile explicit versions from source code, rewrite PATH in user environment and install “bundler gem” as root.
  50. 50. 8.- Developers isolated ruby environment (II) Opscode chef.rpm package provides 2.1 ruby ● install passenger-apache library+binary compiled and installed as a gem under chef. rpm gemlib path (a community recipe is in charge of it) ● compatible to both 2.0.x and 1.9.x user rubies
  51. 51. 8.- Developers isolated ruby environment(III) Explicit ruby version (1.9.3 or 2.0.X) in appropriate virtualhost apache files (from templates) NOTE: Foodcritic tool helped a lot when looking for errors (e.g. specially with chef templates)
  52. 52. 9. Final Steps (I) After just 2 weeks, it almost worked ok. Traditional approach to development: logrotate and monit forked recipes with “999” suffix added to “semver” in metadata.rb ● not perfect ● Explicit installation of dependencies when in Opensource Opscode server
  53. 53. 9. Final Steps (II) Pareto principle! 80/20 Rule. ● Two more weeks with “fringes” ● create new organization in opscode free account ● chef-solo-search for “local” databags ● BIG CRISIS: no chef-server available -> chef-solo deployment
  54. 54. Angry chef?
  55. 55. 9. Final Steps (III) Create git repo (first commit in 2 weeks) ● install explicit dependencies ● “chef cookbook site install” ● only needed “ancient” (fork and modify) approach in two cookbooks: monit and passenger
  56. 56. 9. Final Steps (IV) “In extremis”: Refactor the “monolithic” cookbook Monolithic cookbook refactored as 3 cookbooks Three layers of attribute+template files: ● general cookbook (i.e. language, servers,...) ○ company cookbook (i.e. final customer specific) ■ project cookbook (i.e. virtualhost config)
  57. 57. 9. Final Steps (V) ● only two “old fashioned” forked recipes: monit and passenger ● new recipes forces a different way to use databags
  58. 58. 10. Conclusions (I) To FIX: ● move logrotate recipes in railssites2 to another new bootstrapping-node-sysadmin cookbook ● not exactly applied but inspired in "everything is a cookbook"
  59. 59. 10. Conclusions (II) ● You need to have “Luck” ○ no way to configure staging chef-server (only working through 127.0.0.1 and VPN) ○ no way to configure ubuntu from opscode deb packages ● You need big motivation to success
  60. 60. 10. Conclusions (III) To be improved: ● ubuntu/RHEL cookbooks ● tests ● mysql m/s cookbook ● monit recipe ● opensource chef-server hosting
  61. 61. THANKS FOR YOUR PATIENCE

×