1. A PRACTICAL
INTRODUCTION TO DEVOPS
WITH CHEF
Nick Barendt - LeanDog
CWRU ACM Link-State 2012

2. WHO AM I
• CWRU Alumnus - Wayward Electrical Engineer
• LeanDog - Director of Labs
• CWRU EECS Adjunct Faculty
• Embedded Linux (Bootloaders, Kernel Drivers, Daemons)
• Web/Cloud (AWS, Python Django)

3. LEANDOG
http://leandog.com
http://leandog.com/dogtreats/
Copyright 2012, Nick Barendt

4. DEVOPS

5. DEVELOPMENT
+
OPERATIONS

6. DEVELOPMENT
+
OPERATIONS

7. DEVOPS

8. • Agile/Lean Development
• Frequent Releases (Minimize WIP)
• Continuous Integration → Continuous Delivery
• Reduced Stress: Release is Non-Event; TDD/BDD
• Holistic - code/infrastructure are inextricably linked

9. TOPICS FOR TODAY
• Configuration Management
• Deployment Automation

10. SAMPLE APPLICATION
• Ubuntu 12.04 LTS
• Apache2
• Python Django
• PostgreSQL

11. Apache2
static content mod_wsgi Postgresql

12. STATIC CONTENT
• aka “files”
• HTML, CSS, JavaScript, images, etc.

13. MOD_WSGI
• WSGI - Web Server Gateway Interface
• Python Universal HTTP interface
• mod_wsgi Apache plugin
• executes Python/Django code in response to HTTP requests

14. POSTGRESQL
• Relational Database Management System
• Django’s Object Relational Mapper (ORM) supports:
• Postgresql
• MySQL
• SQLite
• Oracle

15. HOW DO WE DEPLOY?

16. PART 1: THE HARD WAY

17. INSTALL DISTRO PACKAGES

19. CREATE THE VIRTUALENV

20. Virtualenv
• Python “environment” isolated from rest of system
• Analogous to Ruby’s rvm
• Python Interpreter + 3rd party libraries
• pip - Python Package Manager
• requirements.txt - 3rd party library manifest, with versions

21. requirements.txt
django==1.4.2
psycopg2==2.4.5

23. CONFIGURE POSTGRESQL

24. • Configure Postgresql to listen on TCP Port 5432
• Configure Postgresql Authentication
• Create User “roster”
• Create Database “roster”

26. SOURCE!

28. THAT WASN’T PRETTY

29. OR EASILY REPEATABLE

30. PAUSE

31. WHAT DOES THE APP DO?

33. BACK TO THE STORY

34. CONFIGURATION
• Repeatable and works on a minimal, standard install image
• TIP: Use virtual machines for development/testing:
• Snapshot Virtual Machine
• Test Configuration Process
• Verify and/or Fix Configuration Process
• Rollback

35. SNAPSHOT TREE
VMware Fusion
Copyright 2012, Nick Barendt

36. HOW TO IMPROVE?

37. TAKE A FEW
NOTES?

38. WE CAN DO SLIGHTLY BETTER

39. SCRIPTS!

40. “prepare_ubuntu”
#!/bin/bash
# must be run as root!
apt-get install -y apache2 libapache2-mod-wsgi
apt-get install -y python-dev python-virtualenv
apt-get install -y git
apt-get install -y postgresql-9.1 libpq-dev
apt-get install -y vim

41. “create_virtualenv”
#!/bin/bash
ENV_DIR=env
rm -rf $ENV_DIR
virtualenv $ENV_DIR
$ENV_DIR/bin/pip install -r requirements.txt

42. HOW ABOUT CONFIG FILES...

43. YEAH....ABOUT THAT

44. FINALLY, WE CAN TALK
ABOUT CHEF

45. PART 2: CHEF!

46. LET’S WAIT JUST A LITTLE
BIT LONGER

47. IN AN IDEAL WORLD, WHAT
WOULD WE WANT?

48. Configuration IS Source
• Configuration in Version Control
• Text Files
• Diff ’able
• Tie Infrastructure Configuration and Production Releases

49. A DECLARATIVE DSL
database_user 'roster' do
password 'spartans'
database_name 'roster'
end

50. CHEF
• Opscode - Private company
• 2009
• Open Source - Apache License 2.0
• Ruby DSL for configuration “recipes” or “cookbooks”

51. CHEF ARCHITECTURE
Chef Server chef client
knife

52. CHEF ARCHITECTURE

53. BUT I JUST WANT TO
MANAGE A FEW MACHINES...

54. CHEF-SOLO

55. chef-solo
• Essentially chef-client, but operates without chef server
• Only uses local resources
• Simpler to start learning
• “Recipes” are inherently the same as “full blown” chef

56. CHEF CONCEPTS
• NODES - “servers”, “computers”, “machines”
• RESOURCES - abstraction of the “thing” you’re configuring
• RECIPES - collection of Resource descriptions
• COOKBOOKS - collections of Recipes and Attributes

57. NODES
• Ubuntu • Scientific Linux
• Debian • OS X
• Red Hat • Windows
• Fedora
• CentOS
• Oracle
Copyright 2012, Nick Barendt

58. RESOURCES
• File • User and Group
• Template • Git
• Execute • Subversion
• Cron
• Link
• Directory
Copyright 2012, Nick Barendt

59. RECIPES
package "libpq-dev"
python_virtualenv node[:roster]["VIRTUALENV_DIR"] do
interpreter "python2.7"
action :create
end
pip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")
requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")
execute "#{pip_cmd} install -r #{requirements_txt}" do
end

60. RECIPES
package "libpq-dev"
python_virtualenv node[:roster]["VIRTUALENV_DIR"] do
interpreter "python2.7"
action :create
end
pip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")
requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")
execute "#{pip_cmd} install -r #{requirements_txt}" do
end

61. RECIPES
package "libpq-dev"
python_virtualenv node[:roster]["VIRTUALENV_DIR"] do
interpreter "python2.7"
action :create
end
pip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")
requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")
execute "#{pip_cmd} install -r #{requirements_txt}" do
end

62. RECIPES
Install Distro Package
package "libpq-dev"
python_virtualenv node[:roster]["VIRTUALENV_DIR"] do
interpreter "python2.7"
action :create
end
pip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")
requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")
execute "#{pip_cmd} install -r #{requirements_txt}" do
end

63. RECIPES
create virtualenv @
package "libpq-dev"
specified path
python_virtualenv node[:roster]["VIRTUALENV_DIR"] do
interpreter "python2.7"
action :create
end
pip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")
requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")
execute "#{pip_cmd} install -r #{requirements_txt}" do
end

64. RECIPES
some variables
package "libpq-dev"
and Ruby code
python_virtualenv node[:roster]["VIRTUALENV_DIR"] do
interpreter "python2.7"
action :create
end
pip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")
requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")
execute "#{pip_cmd} install -r #{requirements_txt}" do
end

65. RECIPES
package "libpq-dev"
python_virtualenv node[:roster]["VIRTUALENV_DIR"] do
interpreter "python2.7"
action :create
end
pip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")
requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")
execute "#{pip_cmd} install -r #{requirements_txt}" do
end
execute command-line

66. RECIPES
package "libpq-dev"
python_virtualenv node[:roster]["VIRTUALENV_DIR"] do
interpreter "python2.7"
action :create
end
pip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin", "pip")
requirements_txt = File.join(node[:roster]["DEPLOY_DIR"], "requirements.txt")
execute "#{pip_cmd} install -r #{requirements_txt}" do
end

67. COOKBOOKS
• Collections of
• Attributes
• Recipes
• Templates
Copyright 2012, Nick Barendt

68. TONS OF COOKBOOKS
HTTPS://GITHUB.COM/OPSCODE-COOKBOOKS
• Java • Erlang
• Python • Django
• Nagios • Rsyslog
• openssh • Nginx
• Apache2 • Perl
• Postfix • Many, Many More
Copyright 2012, Nick Barendt

69. CONVERGENCE
• Chef attempts to converge a machine’s configuration to the
desired state
• Withtemporary errors, repeated runs will drive towards
convergence
• Resources are (generally) idempotent
• Robust
• Try, try again

70. HOW DOES THIS ACTUALLY
WORK?

71. • Merge Defaults and Node Attributes
• Recipe Compilation
• Recipe Execution

73. sudo chef-solo -c solo.rb -j solo-configs/standalone.json
solo.rb
# minimal configuration file for chef-solo
cookbook_path File.expand_path File.dirname(__FILE__) + "/cookbooks"
standalone.json
{
"roster": {
"app_server": {
"load_default_data": true
}
},
"postgresql": {
"password":{
"postgres": "roster_123"
}
},
"run_list": [
"recipe[roster::database]",
"recipe[roster::app_server]"
]
}

74. app_server.rb
include_recipe "apache2"
include_recipe "apache2::mod_wsgi"
include_recipe "python"
node[:roster]["DEPLOY_DIR"] = File.expand_path(
File.join( File.expand_path( File.dirname(__FILE__) ),
"..","..","..",".."))
node[:roster]["VIRTUALENV_DIR"] = File.expand_path(
File.join(node[:roster]["DEPLOY_DIR"], "env"))
# disable default, regardless of how it is symlinked
apache_site "default" do
enable false
end
apache_site "000-default" do
enable false
end
# we need a few packages
package "libpq-dev"
python_virtualenv node[:roster]["VIRTUALENV_DIR"] do
interpreter "python2.7"
action :create
end

75. continued
pip_cmd = File.join(node[:roster]["VIRTUALENV_DIR"], "bin",
"pip")
requirements_txt = File.join(node[:roster]["DEPLOY_DIR"],
"requirements.txt")
execute "#{pip_cmd} install -r #{requirements_txt}" do
end
web_app "roster" do
template "roster_apache.conf.erb"
end
django_admin_cmd = File.join(node[:roster]["DEPLOY_DIR"],
"cwru_acm", "manage.py")
python_exec = File.join(node[:roster]["VIRTUALENV_DIR"], "bin",
"python")
execute "#{python_exec} #{django_admin_cmd} syncdb --noinput"
do
end
execute "#{python_exec} #{django_admin_cmd} collectstatic --
noinput" do
end
execute "#{python_exec} #{django_admin_cmd} loaddata roster
some_initial_data" do
only_if {node[:roster][:app_server][:load_default_data]}
end

76. database.rb
include_recipe "postgresql::client"
include_recipe "postgresql::server"
r = package "ruby-pg" do
package_name "libpgsql-ruby"
action :nothing
end
r.run_action(:upgrade)
# Snippet from opscode to reload gems
require 'rubygems'
Gem.clear_paths
require "pg"
include_recipe "database"
postgresql_connection_info = {:host => "127.0.0.1", :port => 5432, :username =>
'postgres', :password => node['postgresql']['password']['postgres']}
postgresql_database "roster" do
connection postgresql_connection_info
action :create
end
database_user 'roster' do
connection postgresql_connection_info
password 'spartans'
database_name 'roster'
provider Chef::Provider::Database::PostgresqlUser
action :create
end
database_user 'roster' do
connection postgresql_connection_info
password 'spartans'
provider Chef::Provider::Database::PostgresqlUser
action :grant
end

77. But Wait, There’s More
• Parametric Configs. (environments): Dev, Staging, Production
• Roles
- collect recipes based on how server is used (e.g.,
Apache2 + Memcache for “front-end”)
• Test Driven Infrastructure - Configuration is just code!
• Bootstrapping in AWS EC2 and other Clouds
• Chef Server, Knife

78. Questions?
nick@barendt.com Twitter: @nickbarendt
Code From Today:
https://github.com/nbarendt/cwru_acm_2012

79. FOR MORE INFO
• http://wiki.opscode.com
• Fast
Start Linux Guide: http://wiki.opscode.com/display/chef/
Fast+Start+Guide
• AWSEC2 Bootstrapping: http://wiki.opscode.com/display/chef/
EC2+Bootstrap+Fast+Start+Guide
• O’Reilly: Test-Driven Infrastructure with Chef