Jon Cowie, Etsy.com @jonlives

1. Michelin Starred
Cooking with Chef
Jon Cowie, Etsy.com
jcowie@etsy.com
@jonlives
Wednesday, June 27, 12

2. What?
Wednesday, June 27, 12

3. What?
• Chef at Etsy
Wednesday, June 27, 12

4. What?
• Chef at Etsy
• Familiarity and Understanding
Wednesday, June 27, 12

5. What?
• Chef at Etsy
• Familiarity and Understanding
• Critical Approach and Experimentation
Wednesday, June 27, 12

6. What?
• Chef at Etsy
• Familiarity and Understanding
• Critical Approach and Experimentation
• Use The Source
Wednesday, June 27, 12

7. What?
• Chef at Etsy
• Familiarity and Understanding
• Critical Approach and Experimentation
• Use The Source
• A liberal sprinkling of screwups
Wednesday, June 27, 12

8. What?
• Chef at Etsy
• Familiarity and Understanding
• Critical Approach and Experimentation
• Use The Source
• A liberal sprinkling of screwups
• Open Sourced Goodness - We’re all here!
Wednesday, June 27, 12

9. What?
• Chef at Etsy
• Familiarity and Understanding
• Critical Approach and Experimentation
• Use The Source
• A liberal sprinkling of screwups
• Open Sourced Goodness - We’re all here!
• [x] = http://tiny.cc/velocity2012
Wednesday, June 27, 12

10. Opscode is Orange,
Velocity is Blue.
In Soviet Russia,
Cookbook writes you.
Wednesday, June 27, 12

11. Chef at Etsy
Wednesday, June 27, 12

12. Our Setup
Wednesday, June 27, 12

13. Our Setup
• Open Source chef server 0.10.4
Wednesday, June 27, 12

14. Our Setup
• Open Source chef server 0.10.4
• Backup to Opscode Platform
Wednesday, June 27, 12

15. Our Setup
• Open Source chef server 0.10.4
• Backup to Opscode Platform
• ~800 self-hosted nodes (Mainly CentOS,
some RHEL & mac)
Wednesday, June 27, 12

16. Our Setup
• Open Source chef server 0.10.4
• Backup to Opscode Platform
• ~800 self-hosted nodes (Mainly CentOS,
some RHEL & mac)
• KVM & lxc virts, self hosted too.
Wednesday, June 27, 12

17. Our Setup
• Open Source chef server 0.10.4
• Backup to Opscode Platform
• ~800 self-hosted nodes (Mainly CentOS,
some RHEL & mac)
• KVM & lxc virts, self hosted too.
• Never test in production!
Wednesday, June 27, 12

18. Our Setup
• Open Source chef server 0.10.4
• Backup to Opscode Platform
• ~800 self-hosted nodes (Mainly CentOS,
some RHEL & mac)
• KVM & lxc virts, self hosted too.
• Never test in production!
• Many chefs don’t spoil our soup
Wednesday, June 27, 12

19. Our Setup
• Open Source chef server 0.10.4
• Backup to Opscode Platform
• ~800 self-hosted nodes (Mainly CentOS,
some RHEL & mac)
• KVM & lxc virts, self hosted too.
• Never test in production!
• Many chefs don’t spoil our soup
Wednesday, June 27, 12

20. Familiarity and
Understanding
Wednesday, June 27, 12

21. not
http://www.flickr.com/photos/photo_secessionist/5555167113/ (C) Alexander McQueen
Wednesday, June 27, 12

22. Wednesday, June 27, 12

23. • Insight
Wednesday, June 27, 12

24. • Insight
• Simplicity
Wednesday, June 27, 12

25. • Insight
• Simplicity
• Standards
Wednesday, June 27, 12

26. Insight
Wednesday, June 27, 12

27. Insight
• You should never have to say “I don’t
know.”
Wednesday, June 27, 12

28. Insight
• You should never have to say “I don’t
know.”
• What, where, when, why, how long?
Wednesday, June 27, 12

29. Insight
• You should never have to say “I don’t
know.”
• What, where, when, why, how long?
• Easy, and I’ll show you how!
Wednesday, June 27, 12

30. Wednesday, June 27, 12

31. Chef Dashboard
Wednesday, June 27, 12

32. Chef Dashboard
• Chef handler sends metrics to graphite [4]
Wednesday, June 27, 12

33. Chef Dashboard
• Chef handler sends metrics to graphite [4]
• git clone git://github.com/etsy/chef-handlers.git
Wednesday, June 27, 12

34. Chef Dashboard
• Chef handler sends metrics to graphite [4]
• git clone git://github.com/etsy/chef-handlers.git
• Set your graphite server’s URL in
graphite.rb
Wednesday, June 27, 12

35. Chef Dashboard
• Chef handler sends metrics to graphite [4]
• git clone git://github.com/etsy/chef-handlers.git
• Set your graphite server’s URL in
graphite.rb
• Add the following to client.rb
Wednesday, June 27, 12

36. Chef Dashboard
• Chef handler sends metrics to graphite [4]
• git clone git://github.com/etsy/chef-handlers.git
• Set your graphite server’s URL in
graphite.rb
• Add the following to client.rb
• require "<clonedir>/graphite.rb"
graphite_handler = GraphiteReporting.new
report_handlers << graphite_handler
exception_handlers << graphite_handler
Wednesday, June 27, 12

37. Chef Dashboard
• Chef handler sends metrics to graphite [4]
• git clone git://github.com/etsy/chef-handlers.git
• Set your graphite server’s URL in
graphite.rb
• Add the following to client.rb
• require "<clonedir>/graphite.rb"
graphite_handler = GraphiteReporting.new
report_handlers << graphite_handler
exception_handlers << graphite_handler
• Etsy dashboards framework [5]
Wednesday, June 27, 12

38. [12:54:01] <irccat> Chef run failed on gfernandez.vm.ny4dev.etsy.com
[12:54:02] <irccat> https://github.etsycorp.com/gist/384228
[12:54:02] <irccat>
[12:54:07] <irccat> Chef run failed on buildtest11.ny4dev.etsy.com
[12:54:07] <irccat> https://github.etsycorp.com/gist/384227
[12:54:07] <irccat>
Wednesday, June 27, 12

39. Chef irccat Alerts
Wednesday, June 27, 12

40. Chef irccat Alerts
• Chef handler send fails to irc[4] via irccat [6]
Wednesday, June 27, 12

41. Chef irccat Alerts
• Chef handler send fails to irc[4] via irccat [6]
• git clone git://github.com/etsy/chef-handlers.git
Wednesday, June 27, 12

42. Chef irccat Alerts
• Chef handler send fails to irc[4] via irccat [6]
• git clone git://github.com/etsy/chef-handlers.git
• Set your irccat[6] server’s URL in
logtoirc.rb
Wednesday, June 27, 12

43. Chef irccat Alerts
• Chef handler send fails to irc[4] via irccat [6]
• git clone git://github.com/etsy/chef-handlers.git
• Set your irccat[6] server’s URL in
logtoirc.rb
• Add the following to client.rb
Wednesday, June 27, 12

44. Chef irccat Alerts
• Chef handler send fails to irc[4] via irccat [6]
• git clone git://github.com/etsy/chef-handlers.git
• Set your irccat[6] server’s URL in
logtoirc.rb
• Add the following to client.rb
• require "<clonedir>/logtoirc.rb"
exception_handlers << Etsy::LogToIRC.new
Wednesday, June 27, 12

45. ~ > knife node lastrun buildtest11.ny4dev.etsy.com
Status failed
Elapsed Time 4.628171438
Start Time 2012-06-18 10:06:28 +0000
End Time 2012-06-18 10:06:32 +0000
Recipe Action Resource Type Resource
Backtrace
<snip>
Exception
Chef::Exceptions::Package: package[php] (php::buildtest line 20) had
an error: Version 5.3.10-1.el5 of php not found. Did you specify both
version and release? (version-release, e.g. 1.84-10.fc6)
Wednesday, June 27, 12

46. ~ > knife search node 'lastrun_debug_formatted_exception:Chef:
:Exceptions::Package*' -a lastrun.debug.formatted_exception
5 items found
id: masterrestore.ny4.etsy.com
lastrun.debug.formatted_exception: Chef::Exceptions::Package:
package[postgresql-server] (postgresql::server-8.3 line 1) had an
error: Installed package postgresql-server-8.3.16-1PGDG_id is newer
than candidate package postgresql-server-8.3.11-1PGDG_id.rhel5
id: buildtest11.ny4dev.etsy.com
lastrun.debug.formatted_exception: Chef::Exceptions::Package:
package[php] (php::buildtest line 20) had an error: Version
5.3.10-1.el5 of php not found. Did you specify both version and
release? (version-release, e.g. 1.84-10.fc6)
<snip>
Wednesday, June 27, 12

47. Chef lastrun Info
Wednesday, June 27, 12

48. Chef lastrun Info
• Chef handler and knife plugin [7]
Wednesday, June 27, 12

49. Chef lastrun Info
• Chef handler and knife plugin [7]
• gem install knife-lastrun
Wednesday, June 27, 12

50. Chef lastrun Info
• Chef handler and knife plugin [7]
• gem install knife-lastrun
• Add the following to client.rb
Wednesday, June 27, 12

51. Chef lastrun Info
• Chef handler and knife plugin [7]
• gem install knife-lastrun
• Add the following to client.rb
• require "lastrun_update"
handler = LastRunUpdateHandler.new
report_handlers << handler
exception_handlers << handler
Wednesday, June 27, 12

52. Chef lastrun Info
• Chef handler and knife plugin [7]
• gem install knife-lastrun
• Add the following to client.rb
• require "lastrun_update"
handler = LastRunUpdateHandler.new
report_handlers << handler
exception_handlers << handler
• knife node lastrun <nodename>
Wednesday, June 27, 12

53. Simplicity
Wednesday, June 27, 12

54. Simplicity
• Think of yourself at 3AM!
Wednesday, June 27, 12

55. Simplicity
• Think of yourself at 3AM!
• Please, won’t you think of the new guy?
Wednesday, June 27, 12

56. Simplicity
• Think of yourself at 3AM!
• Please, won’t you think of the new guy?
• Minimize the logics!
Wednesday, June 27, 12

57. Simplicity
• Think of yourself at 3AM!
• Please, won’t you think of the new guy?
• Minimize the logics!
• As few logical steps from start to finish
as possible.
Wednesday, June 27, 12

58. Simplicity - Not!
Date: Mon Dec 05 2011 23:07:18 GMT+0000 (GMT)
Subject: so close to death
# Don't install v2 on search or Cent 5.6 nodes
-if node[:fqdn] !~ /b(^(preprod-)?search[0-9]{2}|ny4dev.etsy.com|^(preprod-)?
giftsweb[0-9]{2}|^db(shard|spare|data)[0-9]{2}|^qa-web01|^devsearch[0-9]{2}|^nagios01|
^webnest[0-9]{2}|^prodking[0-9]{2}|^sandboxweb[0-9]{2}|^virt((0[5-9])|(1[0-9]))|
^msysmgr[0-9]{2}|^msysmta[0-9]{2}|^dbconvo[0-9]{2}|^dbshowcase01|atlasweb[0-9]{2}|
devnagios[0-9]{2}|cimaster02|worker[0-9]{2}|^ganglia[0-9]{2}|^imgcache[0-9]{2}|
imgconvert[0-9]{2}|^imgwriter[0-9]{2}|dev-img02|^datacache04|^graphite01|^graphite03|
^webutil03|^webutil04|^statsd01|^maintweb[0-9]{2}|^(dev-|preprod-)?convosearch[0-9]{2}|
deployinator[0-9]{2}|^wpadmin01|^(preprod-)?dbtasks[0-9]{2})b/ and node.role?("Web56") ==
false and node.role?("Preprodweb56") == false and node.role?("Princess53") == false
+if node[:fqdn] !~ /b(^(preprod-)?search[0-9]{2}|ny4dev.etsy.com|^(preprod-)?
giftsweb[0-9]{2}|^db(shard|spare|data)[0-9]{2}|^qa-web01|^devsearch[0-9]{2}|^nagios01|
^webnest[0-9]{2}|^prodking[0-9]{2}|^sandboxweb[0-9]{2}|^virt((0[5-9])|(1[0-9]))|
^msysmgr[0-9]{2}|^msysmta[0-9]{2}|^dbconvo[0-9]{2}|^dbshowcase01|atlasweb[0-9]{2}|
devnagios[0-9]{2}|cimaster02|worker[0-9]{2}|^ganglia[0-9]{2}|^imgcache[0-9]{2}|
imgconvert[0-9]{2}|^imgwriter[0-9]{2}|dev-img02|^datacache04|^graphite01|^graphite03|
^webutil03|^webutil04|^statsd01|^maintweb[0-9]{2}|^(dev-|preprod-)?convosearch[0-9]{2}|
deployinator[0-9]{2}|^wpadmin01|^(preprod-)?dbtasks[0-9]{2})b/ and node.role?("Web56") ==
false and node.role?("Preprodweb56") == false and node.role?("Princess53") == false and
node.role?("Auth") == false
Wednesday, June 27, 12

59. Simplicity - Better!
if node.chef_environment == "libmemcached_upgrade"
package "libmemcached" do
version "1.0.4-1"
action :install
end
<snip>
else
package "libmemcached" do
version "0.53-1.1"
action :install
end
<snip>
end
Wednesday, June 27, 12

60. Simplicity - Complexity
Wednesday, June 27, 12

61. Simplicity - Complexity
• Sometimes you need complex behaviour
Wednesday, June 27, 12

62. Simplicity - Complexity
• Sometimes you need complex behaviour
• Don’t fight it, try to abstract it.
Wednesday, June 27, 12

63. Simplicity - Complexity
• Sometimes you need complex behaviour
• Don’t fight it, try to abstract it.
• Case in point: Syslog-ng refactor
Wednesday, June 27, 12

64. Case Study: Syslog-ng
Wednesday, June 27, 12

65. Case Study: Syslog-ng
• 36 recipes
Wednesday, June 27, 12

66. Case Study: Syslog-ng
• 36 recipes
• 30 versions of syslog-ng.conf
Wednesday, June 27, 12

67. Case Study: Syslog-ng
• 36 recipes
• 30 versions of syslog-ng.conf
• 27 manually configured files in /etc/syslog-
ng.d on central server
Wednesday, June 27, 12

68. Case Study: Syslog-ng
• 36 recipes
• 30 versions of syslog-ng.conf
• 27 manually configured files in /etc/syslog-
ng.d on central server
• Edge cases and exceptions galore
Wednesday, June 27, 12

69. Case Study: Syslog-ng
Wednesday, June 27, 12

70. Case Study: Syslog-ng
• Down to:
Wednesday, June 27, 12

71. Case Study: Syslog-ng
• Down to:
• 2 recipes (one client, one server)
Wednesday, June 27, 12

72. Case Study: Syslog-ng
• Down to:
• 2 recipes (one client, one server)
• 2 templates (one for syslog-ng.conf, one
for stuff in /etc/syslog-ng.d)
Wednesday, June 27, 12

73. Case Study: Syslog-ng
• Down to:
• 2 recipes (one client, one server)
• 2 templates (one for syslog-ng.conf, one
for stuff in /etc/syslog-ng.d)
• Attributes in roles
Wednesday, June 27, 12

74. Case Study: Syslog-ng
• Down to:
• 2 recipes (one client, one server)
• 2 templates (one for syslog-ng.conf, one
for stuff in /etc/syslog-ng.d)
• Attributes in roles
• Not open sourced yet, sorry :(
Wednesday, June 27, 12

75. Case Study: Syslog-ng
"syslog": {
"group": "preprod_web",
"items": {
"web_apache_access_log": {
"source": "/var/log/httpd/access_log",
"source_program_override": "APACHEACCESS: ",
"destination": "<snip>/access.log",
"destination_filters": [
"host('^preprod-web')",
"match('APACHEACCESS')"
],
"destination_options": [
"template_escape(no)"
]
},
}
}
Wednesday, June 27, 12

76. Remember, No
Panacea!
Wednesday, June 27, 12

77. Remember, No
Panacea!
• A new package hits the repo.
Wednesday, June 27, 12

78. Remember, No
Panacea!
• A new package hits the repo.
• Are you in control of when it goes out?
Wednesday, June 27, 12

79. Remember, No
Panacea!
• A new package hits the repo.
• Are you in control of when it goes out?
• Memcached Outage
Wednesday, June 27, 12

80. Remember, No
Panacea!
• A new package hits the repo.
• Are you in control of when it goes out?
• Memcached Outage
• Do you know what services are going to
restart and when?
Wednesday, June 27, 12

81. Remember, No
Panacea!
• A new package hits the repo.
• Are you in control of when it goes out?
• Memcached Outage
• Do you know what services are going to
restart and when?
• Image Service Outage
Wednesday, June 27, 12

82. Standards
Wednesday, June 27, 12

83. Standards
• Not going to talk about testing [8]!
Wednesday, June 27, 12

84. Standards
• Not going to talk about testing [8]!
• But I don’t have time for standards!
Wednesday, June 27, 12

85. Standards - No Time!
Wednesday, June 27, 12

86. Standards - No Time!
• I won’t say “Make Time”, but you should...
Wednesday, June 27, 12

87. Standards - No Time!
• I won’t say “Make Time”, but you should...
• For a quick win, try Foodcritic
Wednesday, June 27, 12

88. Standards - No Time!
• I won’t say “Make Time”, but you should...
• For a quick win, try Foodcritic
• Good out of the box rules
Wednesday, June 27, 12

89. Standards - No Time!
• I won’t say “Make Time”, but you should...
• For a quick win, try Foodcritic
• Good out of the box rules
• Jenkins integration in seconds
Wednesday, June 27, 12

90. Standards - No Time!
• I won’t say “Make Time”, but you should...
• For a quick win, try Foodcritic
• Good out of the box rules
• Jenkins integration in seconds
• Supports custom rules
Wednesday, June 27, 12

91. Standards - No Time!
• I won’t say “Make Time”, but you should...
• For a quick win, try Foodcritic
• Good out of the box rules
• Jenkins integration in seconds
• Supports custom rules
• Plays well with others
Wednesday, June 27, 12

92. Foodcritic
Wednesday, June 27, 12

93. Foodcritic
•gem install foodcritic
Wednesday, June 27, 12

94. Foodcritic
•gem install foodcritic
•foodcritic <cookbook_repo>
Wednesday, June 27, 12

95. Foodcritic
•gem install foodcritic
•foodcritic <cookbook_repo>
•Simple Jenkins job:
Wednesday, June 27, 12

96. Foodcritic
•gem install foodcritic
•foodcritic <cookbook_repo>
•Simple Jenkins job:
#!/usr/bin/env rvm-shell 1.9.3
foodcritic -f correctness .
Wednesday, June 27, 12

97. Standards at Etsy
Wednesday, June 27, 12

98. Standards at Etsy
• “style” not “correctness”[9]
Wednesday, June 27, 12

99. Standards at Etsy
• “style” not “correctness”[9]
• ETSY001 - Package or yum_package resource used with :upgrade action
Wednesday, June 27, 12

100. Standards at Etsy
• “style” not “correctness”[9]
• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
Wednesday, June 27, 12

101. Standards at Etsy
• “style” not “correctness”[9]
• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
Wednesday, June 27, 12

102. Standards at Etsy
• “style” not “correctness”[9]
• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or
action :nothing
Wednesday, June 27, 12

103. Standards at Etsy
• “style” not “correctness”[9]
• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or
action :nothing
• ETSY005 - Action :restart sent to a core service
Wednesday, June 27, 12

104. Standards at Etsy
• “style” not “correctness”[9]
• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or
action :nothing
• ETSY005 - Action :restart sent to a core service
• ETSY006 - Execute resource used to run chef-provided command
Wednesday, June 27, 12

105. Standards at Etsy
• “style” not “correctness”[9]
• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or
action :nothing
• ETSY005 - Action :restart sent to a core service
• ETSY006 - Execute resource used to run chef-provided command
• ETSY007 - Package or yum_package resource used to install core
package without specific version number
Wednesday, June 27, 12

106. Standards at Etsy
• “style” not “correctness”[9]
• ETSY001 - Package or yum_package resource used with :upgrade action
• ETSY002 - Execute resource used to run git commands
• ETSY003 - Execute resource used to run curl or wget commands
• ETSY004 - Execute resource defined without conditional or
action :nothing
• ETSY005 - Action :restart sent to a core service
• ETSY006 - Execute resource used to run chef-provided command
• ETSY007 - Package or yum_package resource used to install core
package without specific version number
Wednesday, June 27, 12

107. Standards at Etsy
Wednesday, June 27, 12

108. Standards at Etsy
• ETSY001 - Written after Memcached
Outage
Wednesday, June 27, 12

109. Standards at Etsy
• ETSY001 - Written after Memcached
Outage
• Package or yum_package resource used
with :upgrade action
Wednesday, June 27, 12

110. Standards at Etsy
• ETSY001 - Written after Memcached
Outage
• Package or yum_package resource used
with :upgrade action
• package "memcached" do
action :upgrade
end
Wednesday, June 27, 12

111. Standards at Etsy
Wednesday, June 27, 12

112. Standards at Etsy
• ETSY005 - Written after a total Image
service outage
Wednesday, June 27, 12

113. Standards at Etsy
• ETSY005 - Written after a total Image
service outage
• Action :restart sent to a core service
Wednesday, June 27, 12

114. Standards at Etsy
• ETSY005 - Written after a total Image
service outage
• Action :restart sent to a core service
• cookbook_file "/etc/httpd/conf.d/myvhost.conf" do
source "myvhost.conf"
notifies :restart, resources(:service => "httpd")
end
Wednesday, June 27, 12

115. Critical Approach and
Experimentation
Wednesday, June 27, 12

116. CA&E
Wednesday, June 27, 12

117. CA&E
• Chef is by necessity generic
Wednesday, June 27, 12

118. CA&E
• Chef is by necessity generic
• ...so don’t take Opscode’s word for it.
Wednesday, June 27, 12

119. CA&E
• Chef is by necessity generic
• ...so don’t take Opscode’s word for it.
• If it doesn’t work well for you, change it!
Wednesday, June 27, 12

120. CA&E
• Chef is by necessity generic
• ...so don’t take Opscode’s word for it.
• If it doesn’t work well for you, change it!
• Case Study - Etsy Environments rollout
Wednesday, June 27, 12

121. Environments Rollout
Wednesday, June 27, 12

122. Environments Rollout
• We knew we needed them
Wednesday, June 27, 12

123. Environments Rollout
• We knew we needed them
• Simple enough, right?
Wednesday, June 27, 12

124. Environments Rollout
• We knew we needed them
• Simple enough, right?
• Let’s look at the workflow...
Wednesday, June 27, 12

125. Env: Standard Workflow
Wednesday, June 27, 12

126. Env: Standard Workflow
• knife cookbook show php
Wednesday, June 27, 12

127. Env: Standard Workflow
• knife cookbook show php
• Change version number in metadata.rb
Wednesday, June 27, 12

128. Env: Standard Workflow
• knife cookbook show php
• Change version number in metadata.rb
• Change version constraint in foo.json
Wednesday, June 27, 12

129. Env: Standard Workflow
• knife cookbook show php
• Change version number in metadata.rb
• Change version constraint in foo.json
• Commit and push changes to git
Wednesday, June 27, 12

130. Env: Standard Workflow
• knife cookbook show php
• Change version number in metadata.rb
• Change version constraint in foo.json
• Commit and push changes to git
• knife cookbook upload php --freeze
Wednesday, June 27, 12

131. Env: Standard Workflow
• knife cookbook show php
• Change version number in metadata.rb
• Change version constraint in foo.json
• Commit and push changes to git
• knife cookbook upload php --freeze
• knife environment from file foo.json
Wednesday, June 27, 12

132. Env: Our Issues
Wednesday, June 27, 12

133. Env: Our Issues
• 41 people with Chef repo access
Wednesday, June 27, 12

134. Env: Our Issues
• 41 people with Chef repo access
• Most with knife keys
Wednesday, June 27, 12

135. Env: Our Issues
• 41 people with Chef repo access
• Most with knife keys
• All editing the same 2 files with every
change...
Wednesday, June 27, 12

136. Env: Solutions?
Wednesday, June 27, 12

137. Env: Solutions?
• Go with it and hope for the best?
Wednesday, June 27, 12

138. Env: Solutions?
• Go with it and hope for the best?
• Don’t use environments?
Wednesday, June 27, 12

139. Env: Solutions?
• Go with it and hope for the best?
• Don’t use environments?
• Write a totally new workflow?
Wednesday, June 27, 12

140. Env: Solutions?
• Go with it and hope for the best?
• Don’t use environments?
• Write a totally new workflow?
• Tweak the existing one with some
tooling?
Wednesday, June 27, 12

141. Env: Solution!
Wednesday, June 27, 12

142. Env: Solution!
• Tweak with some tooling!
Wednesday, June 27, 12

143. Env: Solution!
• Tweak with some tooling!
• Presenting knife-spork [9]...
Wednesday, June 27, 12

144. Spork: Workflow
Wednesday, June 27, 12

145. Spork: Workflow
• Wrapper around standard environments
workflow
Wednesday, June 27, 12

146. Spork: Workflow
• Wrapper around standard environments
workflow
• check - cookbook versioning
Wednesday, June 27, 12

147. Spork: Workflow
• Wrapper around standard environments
workflow
• check - cookbook versioning
• bump - increment version component
Wednesday, June 27, 12

148. Spork: Workflow
• Wrapper around standard environments
workflow
• check - cookbook versioning
• bump - increment version component
• upload - upload and freeze
Wednesday, June 27, 12

149. Spork: Workflow
• Wrapper around standard environments
workflow
• check - cookbook versioning
• bump - increment version component
• upload - upload and freeze
• promote - set env constraints
Wednesday, June 27, 12

150. Spork: Check
$> knife spork check apache2
Checking versions for cookbook apache2...
Current local version: 1.0.6
Remote versions (Max. 5 most recent only):
*1.0.6, frozen
1.0.5, frozen
<snip>
DANGER: Your local cookbook has same version number as
the starred version above!
Please bump your local version or you won't be able to
upload.
Wednesday, June 27, 12

151. Spork: Bump
$> knife spork bump apache2 <major|minor|patch|manual>
Bumping patch level of the apache2 cookbook from 1.0.6
to 1.0.7
Wednesday, June 27, 12

152. Spork: Upload
$> knife spork upload apache2
Uploading and freezing apache2 [1.0.7]
upload complete
Wednesday, June 27, 12

153. Spork: Promote
$> knife spork promote foo php
Adding version constraint php = 1.0.6
Saving changes into foo.json
Promotion complete! Please remember to upload your
changed Environment file to the Chef Server.
---
$> knife spork promote foo php --remote
Adding version constraint php = 0.1.0
Saving changes into foo.json
Uploading foo to server
Wednesday, June 27, 12

154. Spork
Wednesday, June 27, 12

155. Spork
• Worked well, avoided the issues it was
designed for
Wednesday, June 27, 12

156. Spork
• Worked well, avoided the issues it was
designed for
• Subsequently evolved
Wednesday, June 27, 12

157. Spork
• Worked well, avoided the issues it was
designed for
• Subsequently evolved
• A lot of input & work came from Devs
Wednesday, June 27, 12

158. Spork
• Worked well, avoided the issues it was
designed for
• Subsequently evolved
• A lot of input & work came from Devs
• Organic evolution
Wednesday, June 27, 12

159. Spork
• Worked well, avoided the issues it was
designed for
• Subsequently evolved
• A lot of input & work came from Devs
• Organic evolution
• Open sourced, of course
Wednesday, June 27, 12

160. Spork: Evolution
Wednesday, June 27, 12

161. Spork: Evolution
• Safety Checks
Wednesday, June 27, 12

162. Spork: Evolution
• Safety Checks
• Extra Features
Wednesday, June 27, 12

163. Spork: Safety Checks
Wednesday, June 27, 12

164. Spork: Safety Checks
• Before promoting, check version is
uploaded...
Wednesday, June 27, 12

165. Spork: Safety Checks
$> knife spork promote php --remote
<snip>
WARNING: It looks like you have multiple cookbook paths
defined so I can't tell if you're running inside a git
repo.
Checking that php version 0.1.93 exists on the server
before promoting (any error means it hasn't been
uploaded yet)...
ERROR: The object you are looking for could not be found
Response: Cannot find a cookbook named php with version
0.1.93
Wednesday, June 27, 12

166. Spork: Safety Checks
• Before promoting, check version is
uploaded...
• Check if you’re promoting changes to more
than you thought....
Wednesday, June 27, 12

167. Spork: Safety Checks
WARNING: You're about to promote changes to several
cookbooks:
WARNING:
ganglia: = 0.1.26 changed to = 0.1.25
installerz: = 0.1.66 changed to = 0.1.65
php: = 0.1.92 changed to = 0.1.93
Are you sure you want to continue? (Y/N) N
You said no, so I'm done here.
Would you like to reset your local development.json to
match the server?? (Y/N) Y
<snip>
development.json reset.
Wednesday, June 27, 12

168. Spork: Features
Wednesday, June 27, 12

169. Spork: Features
• Default Environments
Wednesday, June 27, 12

170. Spork: Features
• Default Environments
• Git support
Wednesday, June 27, 12

171. Spork: Features
• Default Environments
• Git support
• Chat notifications + Gist (upload and
promote)
Wednesday, June 27, 12

172. Spork: Features
• Default Environments
• Git support
• Chat notifications + Gist (upload and
promote)
• IRCCat
Wednesday, June 27, 12

173. Spork: Features
• Default Environments
• Git support
• Chat notifications + Gist (upload and
promote)
• IRCCat
• Hipchat (courtesy of Secondmarket)
Wednesday, June 27, 12

174. Features: Chat
Notifications
[19:43:46] <irccat> CHEF: pmcdonnell uploaded and froze
cookbook immount version 0.0.24
[19:44:00] <irccat> CHEF: pmcdonnell uploaded environment
production https://github.etsycorp.com/gist/385043
[19:44:01] <irccat> CHEF: pmcdonnell uploaded environment
development https://github.etsycorp.com/gist/385044
Wednesday, June 27, 12

175. Spork: Features
• Default Environments
• Git support
• Chat notifications + Gist (upload and
promote)
• IRCCat
• Hipchat (courtesy of Secondmarket)
• Graphite (on promote)
Wednesday, June 27, 12

176. Features: Graphite
Wednesday, June 27, 12

177. Spork: Features
• Default Environments
• Git support
• Chat notifications + Gist (upload and
promote)
• IRCCat
• Hipchat (courtesy of Secondmarket)
• Graphite (on promote)
• Foodcritic (on upload)
Wednesday, June 27, 12

178. Features: Foodcritic
$> knife spork upload system
<snip>
Lint checking system...
ERROR: Lint check failed. Halting upload.
ERROR: Lint check output:
ERROR: ETSY003: Execute resource used to run curl or
wget commands: /Users/jcowie/dev/etsy/chef/cookbooks/
system/recipes/dev-ssl.rb:41
Wednesday, June 27, 12

179. Use the Source!
tiny.cc/velocity2012
Wednesday, June 27, 12

180. We’re hiring!
BoF on Tuesday
or just come and say Hi :)
Wednesday, June 27, 12