This document discusses testing principles and tools for infrastructure as code using Chef. It covers test-driven development (TDD) principles like writing tests first to define desired behavior. Chef testing tools covered include ChefSpec for unit testing cookbooks, ServerSpec for integration/black-box testing, and Foodcritic for linting cookbooks. It also discusses Test Kitchen for testing cookbooks across platforms and continuous integration (CI) using these tools.

1. Test-Driven
Infrastructure with Chef
Principles & Tools

2. @kaktusmimi

3. Today
① Testing Principles
② Chef Testing Tools
③ Continuous Integration

4. Write
Test
Run Test
(it fails)
Write
Code
Test
Passes
Refactor
TDD

5. BDD
TDDverification of a unit of code
specification of how code should
„behave“

7. T
DD
pitfall

8. Acceptance
Tests
Integration
Tests
Unit
Tests

9. To
d
ay

10. Today
① Testing Principles
② Chef Testing Tools
③ Continuous Integration
✔

11. ChefSpec
• Unit Testing Framework for Chef Cookbooks
• Runs locally without converging a (virtual) machine

12. $ gem install chefspec

13. !"" site-cookbooks
!"" pt_jenkins
#"" recipes
$ !"" default.rb
!"" spec
!"" default_spec.rb

14. execute "Install Jenkins from Ports" do
command "cd #{node['pt_jenkins']['jenkins_port_dir']} && make install clean BATCH="YES""
not_if {File.exist?(node['pt_jenkins']['jenkins_war_file'])}
end
directory node['pt_jenkins']['jenkins_home'] do
owner node['pt_jenkins']['jenkins_user']
group node['pt_jenkins']['jenkins_group']
mode '0766'
action :create
end
node.default['user']['git_ssh_wrapper'] = "/tmp/git_ssh_wrapper"
file node['user']['git_ssh_wrapper'] do
owner node['pt_jenkins']['jenkins_user']
group node['pt_jenkins']['jenkins_group']
mode 0777
content "/usr/bin/env ssh -A -o 'StrictHostKeyChecking=no' $1 $2"
action :create
end
cookbook_file 'Copy private vagrant key' do
path "/home/vagrant/.ssh/id_rsa"
source 'vagrant_private_key'
owner 'vagrant'
group 'vagrant'
backup false
mode 0600
action :create
end
cookbook_file 'Copy SSH config' do
path "/home/vagrant/.ssh/config"
source 'ssh_config'
owner 'vagrant'
group 'vagrant'
backup false
mode 0600
action :create
end
service node['pt_jenkins']['jenkins_service'] do
supports :status => true, :restart => true, :reload => true
action [ :enable, :start ]
end

15. require 'chefspec'
RSpec.configure do |config|
config.cookbook_path = [‚cookbooks','site-cookbooks']
config.role_path = 'roles'
end
describe 'pt_jenkins::default' do
let(:chef_run) do
ChefSpec::SoloRunner.new do |node|
node.set['jenkins']['plugins'] = {}
node.set['user']['git_ssh_wrapper'] = '/tmp/git_ssh_wrapper'
end.converge(described_recipe)
end
it 'installs Jenkins from Ports' do
expect(chef_run).to run_execute('Install Jenkins from Ports')
end
it 'creates a ssh wrapper file' do
expect(chef_run).to create_file('/tmp/git_ssh_wrapper')
end
it 'copies the ssh config' do
expect(chef_run).to create_cookbook_file('Copy SSH config')
end
it 'copies the private Vagrant key' do
expect(chef_run).to create_cookbook_file('Copy private vagrant key')
end
it 'starts the Jenkins service' do
expect(chef_run).to start_service('jenkins')
end
end

16. $ time bundle exec rspec site-cookbooks/pt_jenkins
pt_jenkins::default
installs Jenkins from Ports
creates a ssh wrapper file
copies the ssh config
copies the private Vagrant key
starts the Jenkins service
Finished in 0.34419 seconds
4 examples, 0 failures
1.43s user 0.24s system 76% cpu 2.186 total

17. Pros & Cons ChefSpec
Pro Contra
Fast White Box Testing
Can be run without
convergence
Harder to implement
Easy Setup Some tricky configuration
Don’t know what system
looks like at the end

18. ServerSpec
• Describe desired state
• Check whether convergence result matches
expectations
• Platform independent
• Run it „from inside“ or „from outside“

19. $ gem install serverspec
$ serverspec-init
Select OS type:
1) UN*X
2) Windows
Select number: 1
Select a backend type:
1) SSH
2) Exec (local)
Select number: 1
Vagrant instance y/n: y
Auto-configure Vagrant from Vagrantfile? y/n: y

20. $ rake -T
rake spec:pttech # Run serverspec tests to pttech

21. #"" site-cookbooks
$ !"" spec
$ #"" pt
$ $ !"" jenkins_spec.rb
$ !"" spec_helper.rb
!"" Rakefile

22. require 'serverspec'
require 'net/ssh'
require 'tempfile'
set :backend, :ssh
if ENV['ASK_SUDO_PASSWORD']
begin
require 'highline/import'
rescue LoadError
fail "highline is not available. Try installing it."
end
set :sudo_password, ask("Enter sudo password: ") { |q| q.echo = false }
else
set :sudo_password, ENV['SUDO_PASSWORD']
end
host = ENV['TARGET_HOST']
`vagrant up #{host}`
config = Tempfile.new('', Dir.tmpdir)
`vagrant ssh-config #{host} > #{config.path}`
options = Net::SSH::Config.for(host, [config.path])
options[:user] ||= Etc.getlogin
set :host, options[:host_name] || host
set :ssh_options, options

23. require 'rake'
require 'rspec/core/rake_task'
task :spec => 'spec:all'
task :default => :spec
namespace :spec do
targets = []
Dir.glob('./spec/*').each do |dir|
next unless File.directory?(dir)
targets << File.basename(dir)
end
task :all => targets
task :default => :all
targets.each do |target|
desc "Run serverspec tests on #{target}"
RSpec::Core::RakeTask.new(target.to_sym) do |t|
ENV['TARGET_HOST'] = target
t.pattern = "spec/#{target}/*_spec.rb"
end
end
end

24. $ rake
Server contains
- jenkins.war at the expected location
- a folder /usr/local/jenkins, owned by user jenkins
- a folder /usr/local/jenkins, with mode 766

25. jenkins_config = JSON.parse(File.open("#{File.dirname(__FILE__)}/../../roles/jenkins.json").read)
jenkins_config["override_attributes"]["jenkins"]["plugins"].each do | plugin_name, plugin_config |
describe "Plugin: #{plugin_name}" do
if plugin_config['enabled']
it "has expected .hpi file in plugins directory" do
expect(file("/usr/local/jenkins/plugins/#{plugin_name}.hpi")).to be_file
end
end
if plugin_config['pinned']
it "is marked as pinned and has the .hpi.pinned file in the plugins directory" do
expect(file("/usr/local/jenkins/plugins/#{plugin_name}.hpi.pinned")).to be_file
end
it "is listed as pinned in the Jenkins plugin status" do
expect(check_jenkins_plugin_pinned(plugin_name)).to be_truthy
end
else
it "isn't marked as pinned and has no .hpi.pinned file in the plugins directory" do
expect(file("/usr/local/jenkins/plugins/#{plugin_name}.hpi.pinned")).not_to be_file
end
end
if plugin_config['version']
it "is version #{plugin_config['version']}" do
expect(check_jenkins_plugin_version(plugin_name, plugin_config['version'])).to be_truthy
end
end
end
end

26. Pros & Cons ServerSpec
Pro Contra
Easy Setup
& well documented
No specific Feedback
Black Box Tests Requires running Machine
Small but mighty
command set
Slow
(Easily?) extendable
Can only be run once per
convergence Run

27. Test Kitchen
• Test and Infrastructure Management
• Interesting for Cookbook Development
• Quite some (configuration) overhead
• Good Tutorial
• Bad Documentation

28. ---
driver:
name: vagrant
network:
- ["private_network", {ip: "10.20.30.41", netmask: "255.255.255.0"}]
provisioner:
name: chef_zero
require_chef_omnibus: false
client_rb:
cookbook_path: ["/var/chef/cookbooks", "/var/chef/site-cookbooks"]
platforms:
- name: FreeBSD-10-PtTech-TestKitchen
driver:
name: vagrant
box: nanobsd-hosting-amd64.box-20140917
box_url: http:/vagrantbox.es/some-box-to-use
synced_folders:
- [".", "/var/chef", "create: true, type: :nfs"]
suites:
- name: server
run_list:
- recipe[testkitchen]
- role[jenkins]
attributes:

29. Pros & Cons TestKitchen
Pro Contra
Handles complex Setups
„Touches“ your Machine
after Convergence
Multi-platform Testing for
Cookbook Development
No ServerSpec via ssh
implemented
„Standard“ in many
Tutorials
Requires machine startup
from Scratch
Many Plugins

30. Foodcritic
• Linting for Chef Cookbooks
• Complex Rule Set far beyond Code Indentation
• Goals (from Foodcritic Website)
• To make it easier to flag problems in your Cookbooks
• Faster feedback.
• Automate checks for common problems

32. $ gem install foodcritic
$ foodcritic site-cookbooks/pt_jenkins -t FC001
# FC001 Use strings in preference to symbols to access node attributes
FC001: Use strings in preference to symbols to access node attributes:
site-cookbooks/pt_jenkins/attributes/default.rb:55
FC001: Use strings in preference to symbols to access node attributes:
site-cookbooks/pt_jenkins/attributes/default.rb:56
FC001: Use strings in preference to symbols to access node attributes:
site-cookbooks/pt_jenkins/recipes/default.rb:21
FC001: Use strings in preference to symbols to access node attributes:
site-cookbooks/pt_jenkins/recipes/default.rb:97
FC001: Use strings in preference to symbols to access node attributes:
site-cookbooks/pt_jenkins/recipes/default.rb:98
FC001: Use strings in preference to symbols to access node attributes:
site-cookbooks/pt_jenkins/recipes/default.rb:109
FC001: Use strings in preference to symbols to access node attributes:
site-cookbooks/pt_jenkins/recipes/default.rb:141

33. Further Testing Tools
• BATS: Bash Automated Testing System
• Shell Scripts 😟
• Cucumber Chef
• Great idea, but prototype state only
• Last commit > 1 year ago 😟

34. Today
① Testing Principles
② Chef Testing Tools
③ Continuous Integration
✔
✔

35. Deployment Stage
Knife Upload to
Chef Server
Acceptance Stage
Serverspec Test
Commit Stage
Build Project
ChefSpec Tests
Foodcritic
triggers uploads

36. Today
① Testing Principles
② Chef Testing Tools
③ Continuous Integration
✔
✔
✔

38. http://serverspec.org/
https://github.com/sethvargo/chefspec
http://kitchen.ci/
http://acrmp.github.io/foodcritic/
https://sethvargo.com/unit-testing-chef-cookbooks/
http://leopard.in.ua/2013/12/01/chef-and-tdd/