Harmonious Development: Standardizing The Deployment Process via Vagrant and Puppet
1. Prepared For: Acquia
October 4, 2012
Title: Harmonious Development:
Standardizing The Deployment Process via Vagrant and Puppet
Achieve – Quality on Time
1
2. Achieve Internet Overview
7+ years developing media and entertainment web
properties
Over 60,000 development hours developing Drupal web &
mobile-optimized solutions
Deep expertise in UX
(theming), Globalization, Integration, and Performance
Company commitment to
quality, transparency, continuity, reliability and client
experience
5. Pop Quiz
<?php
/**
* Transform upper camel case to lower camel case.
*
* @param string $word
* Upper camel case word.
*
* @return string
* $word with first letter in lower case.
*
* @see http://www.php.net/manual/en/function.lcfirst.php
*/
function my_module_camel_case($word) {
return lcfirst($word);
}
echo my_module_camel_case('HelloWorld');
7. Agenda (Cont.)
What’s all the fuss?
What is Virtualization?
Vagrant – Creating (and destroying) environments on the fly
Provisioning (via Puppet)
Examples
Future Directions
Q&A
8. What We Are Not Covering
Performance Tuning
These are examples only
Version Control
All provisioning can be placed into VCS
Puppet vs. Chef
9. What’s all the fuss?
“Write once, run anywhere” -- Sun Microsystems
“Write Once, Debug Everywhere” -- Anonymous / Reality
“In many distributed computing environments, failures are
reported in a way that violates even the simplest notions of
consistency.” -- IEEE Software, Bradford Glade. K. Birman
18. What is Virtualization?
Technical Definition….
A logical representation of a computer in software. By decoupling the
physical hardware from the operating system, virtualization provides
more operational flexibility and increases the utilization rate of the
underlying physical hardware.1
In a Nutshell….
Putting a small computer [guest/virtual machine] (operating system,
memory, video display, peripherals, etc.) into the confines of another
bigger physical system [host].
19. VirtualBox
Open Source virtualization software for
Windows, Mac, and Linux
Can host multiple operating systems at
once (within memory and CPU
resources)
Free!
21. Vagrant
Manages your virtual machines
Works with VirtualBox
Extremely customizable
Over 50+ pre-built base boxes
Free!
22. Why Vagrant?
Creates consistent, reproducible environments
Rapid setup – Start development in a few quick steps
Reduce duplicated effort
Increased reliability
Allows you to tinker around, more on this later…
Cost savings!
23. Agenda - Revisited
What’s all the fuss?
What is Virtualization?
Vagrant – Creating (and destroying)
environments on the fly.
Provisioning (via Puppet)
Examples
Future Directions
Q&A
24. Using Vagrant
Objective:
Part 1: Create a Ubuntu 8.04 LTS Virtual Machine (VM)
One Time Steps:
List available boxes:
$ vagrant box list
Add a new Vagrant base box to inventory if not available:
(@see http://www.vagrantbox.es/)
Usage: vagrant box add <alias> <location>
$ vagrant box add hardy /path/to/hardy.box
Create a directory for your Vagrant VMs
$ mkdir <vm_dir>/myproject
Create a Vagrant file (do this in <vm_dir>/myproject)
Usage: vagrant init <alias>
$ vagrant init hardy
25. Using Vagrant
Let’s Rock!!
Start up a virtual machine:
$ vagrant up
Delete a virtual machine:
$ vagrant remove
Connect to virtual machine:
$ vagrant ssh
Success! Instant VM. Do the happy dance!
26. Using Vagrant
Other Useful Commands
Go to lunch:
$ vagrant suspend # Save state of virtual machine
$ vagrant resume # Start virtual machine up again
Reboot:
$ vagrant halt # Shutdown the VM
$ vagrant up –no-provision # Restart VM, skip provisioning*
* = Online documentation says no reprovisioning will occur…
Reload configuration (automatically does a provisioning)
$ vagrant reload
Reprovision a system:
$ vagrant reprovision
Repackage running system:
$ vagrant package
27. Example 1
$ mkdir <vm_dir>/myproject
$ vagrant init hardy
Customize our VM Instance (Excerpts of Vagrantfile)
config.vm.network :hostonly, "192.168.33.10”
# Host: <vm_dir>/<project_name> == Guest: /vagrant
config.vm.share_folder "v-root", "/vagrant", ".", :owner => 'www-data', :group => 'www-
data’
# Configure the VM with puppet.
config.vm.provision :puppet, :module_path => "modules", :options => ["--
environment", "local"] do |puppet|
puppet.manifests_path = "manifests”
puppet.manifest_file = ”myproject.pp”
End
$ vagrant up
28. Example 1
Scenario: Admin for a Day
Wonder what this does… (WARNING)*
$ sudo chmod 444 /etc/sudoers
* Warning, doing this on a Production system may have an adverse effect on
your career. This will remove any ability to sudo.
Solution (Assuming all configuration is provisioned):
$ vagrant destroy
$ vagrant up
29. Vagrant++
Instant development environments!
No physical hardware required.
Creates a safe environment for configuration optimization.
31. Puppet
Provisioning = preparing a device/server for usage, aka configuration
Allows enforceable system configuration
Similar to Microsoft’s System Center Configuration Manager/SMS or Apple’s Profile
Manager
Uses a Resource Abstraction Layer (RAL) to interact with the host.
Core Resource Types include: notify, file, package, service, exec, cron, user, group
RAL can be used to read and modify resources
Configurations can be store in manifests. See resources for prebuilt modules.
33. Puppet: Miscellaneous Info (Cont.)
Describes a resource, use -s to summarize
$ puppet describe <resource>
Inspect individual resource in “Puppet-speak”
$ puppet resource service apache2
service { 'apache2':
ensure => 'stopped',
enable => 'true',
}
Test a manifest on the local system
$ puppet apply --modulepath=<module_path> <manifest>.pp
34. Agenda - Revisited
What’s all the fuss?
What is Virtualization?
Vagrant – Creating (and destroying)
environments on the fly.
Provisioning (via Puppet)
Examples
Future Directions
Q&A
35. Puppet: Important Notes
Common Pattern:
Package, File, Service
Resources are processed asynchronously! Use dependency
relationships (require/before) if any ordering is necessary.
@see http://forge.puppetlabs.com/ for prebuilt Puppet
modules.
36. Puppet Main Manifest
myproject.pp
class myproject {
Site specific ensure => 'absent';
$site = ’myproject' $site:
ensure => 'present'; Site/DB
# Vagrant specific setup, set up hostname, hosts file }
class { 'vagrant': apache::mod {
site => $site, 'rewrite': ensure => 'present'
} }
# Cool stuff mysql::schema { $site:
class { 'vim': } user => $site,
password => $site,
# LAMP stack }
class { 'linux': }
class { 'apache': } Base LAMP class { 'drupal::prepare':
class { 'mysql': } site => $site,
class { 'php': admin_pw => 'admin',
error_reporting => 'E_ALL & ~(E_WARNING|E_NOTICE)', }
} }
class { 'phpmyadmin': }
class { 'myproject': }
# Drupal
class { 'drupal':
server_type => 'dev',
}
# Solr
class { 'tomcat': }
class { 'solr': } Search
# Site specific stuff
apache::site {
'default':
vhost => 'default',
37. Example 2: Puppet Module
/modules/apache/init.pp (Header)
# Class: apache # vhost => 'default',
# # ensure => 'absent';
# This module installs apache and sets up the virtual.
#
# '<site_name_1>':
# ensure => 'present'; Advanced usage
# Mike Lee <michael.lee@achieveinternet.com> # '<site_name_2>':
# 2012-09-07
#
# docroot => '/custom/path',
# ensure => 'present';
Enable host/mod
# Tested platforms: #}
# - Ubuntu 8.04 Hardy Heron #
# # Enable/disable modules.
# Parameters: # apache::mod {
# N/A # 'rewrite': ensure => 'present’
# #}
# Actions:
# Installs and configures apache. Will automatically create the site file.
#
# Requires:
# N/A
#
# Sample Usage:
#
# Make sure apache is present.
# class { 'apache': }
# Base usage
# Enable/disable sites.
# <site_name> = Simple site name. The vagrant domain will be added
automatically.
# i.e. <site_name> = "example" creates a site named "example.vagrant".
# <docroot> = Docroot. Defaults to /vagrant/<site_name>/docroot.
# If this is overridden, this must be the ABSOLUTE path to the docroot.
#
# apache::site {
# 'default':
38. Puppet Sample Module
/modules/apache/init.pp (Code)
class apache { case $ensure {
$apache2_sites = '/etc/apache2/sites' present: {
$apache2_mods = '/etc/apache2/mods' file { "/etc/apache2/sites-available/$site_name":
ensure => file,
# OS Specific settings
case $::operatingsystem { OS Specific require => Package[$apache::package_name],
content => template("apache/${apache::conf_template}"),
ubuntu: { }
$package_name = 'apache2'
$conf_template = 'site.conf.debian.erb' exec { "/usr/sbin/a2ensite $site_name":
} unless => "/bin/readlink -e ${apache2_sites}-enabled/$site_name",
default: { notify => Exec['force-reload-apache'],
notify { "${module_name}_unsupported": require => Package[$apache::package_name],
message => "The ${module_name} module is not supported on }
${::operatingsystem}.", }
} absent: {
error("OS support for ${::operatingsystem} needs to be configured") exec { "/usr/sbin/a2dissite $site_name":
} #onlyif => "/bin/readlink -e ${apache2_sites}-enabled/$site_name", #
} readlink not returning 0
# Define new type: site
Define type: site notify => Exec['force-reload-apache'],
require => Package[$apache::package_name],
define site ($ensure = 'present', $vhost = undef, $docroot = undef) { }
if $vhost { }
$site_name = $vhost default: {
} err("Unknown ensure value: '$ensure'")
else { }
$site_name = "${title}.vagrant" }
} }
if $docroot {
$path = $docroot
}
else {
$path = "/vagrant/${title}/docroot"
}
39. Puppet Sample Module
/modules/apache/init.pp (Code)
# Define new type: mod
define mod ($ensure = 'present') { service { $package_name:
case $ensure {
present: {
ensure => running,
hasstatus => true,
Package/Service
exec { "/usr/sbin/a2enmod $title": hasrestart => true,
unless => "/bin/readlink -e ${apache2_mods}-enabled/${title}.load", require => Package[$package_name],
notify => Exec['force-reload-apache'], }
require => Package[$apache::package_name], }
}
}
absent: {
Define type: mod
exec { "/usr/sbin/a2dismod $title":
#onlyif => "/bin/readlink -e ${apache2_mods}-enabled/${title}.load", #
readlink not returning 0
notify => Exec['force-reload-apache'],
require => Package[$apache::package_name],
}
}
default: {
err("Unknown ensure value: '$ensure'")
}
}
}
# Force reload all the time, doesn't take that much more in resources.
exec { 'force-reload-apache':
command => '/etc/init.d/apache2 force-reload',
refreshonly => true,
}
package { $package_name:
ensure => installed,
}
40. Puppet Sample Module
ERB Templates: Create files on the fly.
/modules/apache/templates/site.conf.debian.erb
# Puppet generated file. DO NOT EDIT!
# Managed by Class['apache']
<VirtualHost *:80>
ServerAdmin webmaster@achieveinternet.com
DocumentRoot <%= path %>
ServerName <%= site_name %>
<Directory <%= path %>>
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
allow from all
</Directory>
</VirtualHost>
41. Puppet++
Construct fully functioning development environments on
the fly!
Bring new team members up to speed in a few simple
commands.
CONSISTENCY! CONSISTENCY! CONSISTENCY!
Everyone is working with the same virtual machine.
42. Agenda - Revisited
What’s all the fuss?
What is Virtualization?
Vagrant – Creating (and destroying)
environments on the fly.
Provisioning (via Puppet)
Examples
Future Directions
Q&A
44. Next Steps: Veewee
Want your own custom Vagrant Box or one doesn’t exist?
Veewee has predefined definitions for creating a variety of Vagrant base boxes.
Installation Requirements (Mac OS X)
- Install RVM (Ruby Version Manager)
$ curl -L https://get.rvm.io | bash -s stable
- Install Xcode, Command Line Tools for Xcode (required by Homebrew)
- Install Homebrew
$ ruby <(curl -fsSkL raw.github.com/mxcl/homebrew/go)
- Install apple-gcc42 GCC Required!
$ brew update
$ brew tap homebrew/dupes
$ brew install autoconf automake apple-gcc42
$ rvm pkg install openssl
45. Next Steps: Veewee
Veewee Installation (Mac OS X)
$ rvm install 1.9.2
$ git clone https://github.com/jedi4ever/veewee.git
$ cd veewee
(official – Never worked for me…)
$ gem install bundler
$ bundle install
(unoffical)
$ gem build veewee.gemspec
$ gem install veewee-*.gem (run as sudo)
$ unalias veewee
46. Next Steps: Veewee
Veewee Base Box Creation (Mac OS X)
In veewee directory:
- Copy new iso into [currentDir]/iso (optional, saves veewee from finding)
Create VM definition off of template:
Usage: veewee vbox define <box_name> <template>
$ veewee vbox define hardy ubuntu-8.04.4-server-i386
Customize VM Definition:
- Edit <veewee_dir>/definitions/<box_name>/definition.rb
Build virtual box (This should start VirtualBox)
Usage: veewee build <box_name>
$ veewee vbox build hardy # go get a coffee
Usage: veewee vbox validate <box_name>
$ veewee vbox validate hardy
Build Vagrant base box:
Usage: vagrant basebox export <box_name>
$ vagrant basebox export hardy
47. Next Steps: Puppet Enterprise
Allow for remote management of puppet nodes.
“One server to rule them all.”
Clone your configurations to EC2.
Audit your configurations.
48. Gotchas
Be mindful of Puppets asyncronous execution order:
Make sure dependecies are properly set.
Starbucks syndrome:
Switching networks can sometimes cause your VM’s networking to
get confused. Halt and restart to correct.
Provision everything… or as much as you can:
This will make resetting your environment go much more smoothly.
50. Further Resources
Prebulit Vagrant Base Boxes (http://www.vagrantbox.es/)
Puppet Forge (http://forge.puppetlabs.com/)
Learn Puppet (http://docs.puppetlabs.com/learning/ral.html)
51. Conclusion
…makes the "works on my machine" excuse a relic of the past. -- Vagrant
Created a disposable test bed to play with configuration changes
Created reusable modules to easily build customized setups
Provided canned environment for those who simply want to jump into a project. Allows
team members to focus on core competencies.
Allows you to quickly and efficiently archive projects and come back to them later
CONSISTENCY - REDUCED ERRORS
EASY & INEXPENSIVE
INSTANT DEVELOPMENT - SAVE $$$
52. Achieve Internet Pillars of Expertise
Technical Foundation Built On the
Following Core Areas of Expertise
and Knowledge:
1. User Experience (UX) / Theming
2. Globalization/Localization
3. Performance & Deployment
4. Integration
54. Thank You
Michael Lee
Senior Architect
michael.lee@achieveinternet.com
www.achieveinternet.com
mlee11111
@ach_mikelee
/pub/michael-lee/4/297/a6b
Direct: 858-453-5760