This is the presentation given at the Symfony Live 2011 conference. It is an introduction to the new agile movement spreading in the technical operations community called DevOps and how to adopt it on web development projects, in particular Symfony projects. Plan of the slides : - Configuration Management - Development VM - Scripted deployment - Continuous deployment Tools presented in the slides: - Puppet - Vagrant - Fabric - Jenkins / Hudson

1. Adopt Devops philosophy on your
Symfony projects
An introduction to Devops
by Fabrice Bernhard

2. Me
Fabrice Bernhard

3. @theodo
fabriceb@theodo.fr

4. Co-founder / CTO of Theodo and Allomatch.com

5. Theodo creates web-based applications
with open-source web technologies, agile methodologies and the highest
standards of quality
to guarantee rapid application development, risk-free deployment and easy
maintenance for every client.

6. @skoop: Now, let's look for a bar in Paris where I could watch the FC Utrecht match
:)

7. Allomatch.com is the website for watching sports in bars in France and Spain
(sportandbar.es)
Allomatch is used by 500 barmen (clients) and visited by more than 200,000
unique visitors per month
Peaks on the biggest days can go up to 20,000 people in the 2 hours preceding
the game
Allomatch.com is hosted on a cluster of 6 servers

8. What is DevOps?
Introduction

9. How many here consider themselves SysAdmins?

10. How many here have never deployed an application on a
server?

11. Wikipedia
Definition of DevOps
DevOps is a set of processes, methods and systems for communication,
collaboration and integration between departments for Development
(Applications/Software Engineering), Technology Operations and Quality Assurance
(QA).
It relates to the emerging understanding of the interdependence of development
and operations in meeting a business' goal to producing timely software products
and services

12. The fundamental DevOps contradiction
Devs VS Ops
Developers are asked to deliver new value, often and fast
Operations people are asked to protect the current value
Pro-Change VS Pro-Stability

13. Silos

14. Break the silos

15. DevOps do RADD

16. DevOps create the infrastructure that empower devs from
the first line of code to the delivery
How to be DevOps?
Configuration management for rapid, repeatable server setup
Deployment scripts to abstract sysadmin tasks and empower developers
Development VMs with prod configuration to ensure consistency and avoid
unexpected system-related bugs
Continuous deployment to make it fast!

17. DevOps is spreading agility to the whole IT project lifecycle

18. Rapid and repeatable server setup
Configuration management with Puppet

19. What is configuration management?
Writing the system configuration of your servers in files
Applying these files automatically
That's it!

20. Why do configuration management?
To do fast cluster deployment: who wants to manually setup 50 EC2 servers???
To do fast crash-recovery: configuration management is the best documentation
for a server's setup
To have consistent environments for development and production

21. Puppet or Chef
Configuration management tools
Two popular recent tools for configuration management: Puppet and Chef
A master server contains different "recipes" describing system configurations
Client servers connect to the master server, read their recipe, and apply the
configuration

22. Puppet

23. Puppet references

24. Let us create a Symfony-ready server with Puppet
Introduction to Puppet manifests

25. class lighttpd
{
package { "apache2.2-bin":
ensure => absent,
}
package { "lighttpd":
ensure => present,
}
service { "lighttpd":
ensure => running,
require => Package["lighttpd", "apache2.2-bin"],
}
}

26. class lighttpd-phpmysql-fastcgi inherits lighttpd
{
package { "php5-cgi":
ensure => present,
}
package { "mysql-server":
ensure => present,
}
exec { "lighttpd-enable-mod fastcgi":
path => "/usr/bin:/usr/sbin:/bin",
creates => "/etc/lighttpd/conf-enabled/10-fastcgi.conf",
require => Package["php5-cgi", "lighttpd"],
}
}

27. class symfony-server inherits lighttpd-phpmysql-fastcgi
{
package { ["php5-cli", "php5-sqlite"]:
ensure => present,
notify => Service["lighttpd"],
}
package { "git-core":
ensure => present,
}
exec { "git clone git://github.com/symfony/symfony1.git":
path => "/usr/bin:/usr/sbin:/bin",
cwd => "/var/www",
creates => "/var/www/symfony1",
require => Package["lighttpd", "git-core"],
}
}

28. class symfony-live-server inherits symfony-server
{
file { "/etc/lighttpd/conf-available/99-hosts.conf":
source => "/vagrant/files/conf/hosts.conf",
notify => Service["lighttpd"],
}
exec { "lighttpd-enable-mod hosts":
path => "/usr/bin:/usr/sbin:/bin",
creates => "/etc/lighttpd/conf-enabled/99-hosts.conf",
require => File["/etc/lighttpd/conf-available/99-hosts.conf"],
notify => Service["lighttpd"],
}
}
include symfony-live-server
notice("Symfony server is going live!")

29. Why not use shell scripts?
Shell scripts are for administrators. Is all your team composed of admin experts?
Even for admin experts, Puppet and Chef recipes are more readable
Puppet and Chef make inheritance and modules easy
Puppet and Chef are idempotent: running them twice in a row will not break
your system

30. Develop and test on the same
environment as in production!
VM provisioning with Vagrant

31. Develop on local Virtual Machines
Vagrant
Vagrant is a tool to create local VirtualBox VMs, configured automatically by your
Chef recipe or Puppet manifest
It ensures you test on the same environment as your production server
It is VERY easy

32. All you need is:
Vagrant
A Puppet manifest
A few system config files
A Vagrant conf file

33. Demonstration
Vagrant
$ git clone git://github.com/fabriceb/sflive2011vm.git .
$ git clone git://github.com/fabriceb/sflive2011.git
$ vagrant up
http://127.0.0.1:2011/

34. Give developers the power to deploy
themselves
Scripted deployment

35. Deployment
Deployment is a very critical task usually done by admins
Remember Murphy's law: "If anything can go wrong, it will"
W hen things go wrong, most of the time developers have the solution
So give the developers the responsibility to deploy, rollback, correct and deploy
again!

36. Scripting deployment can be VERY easy
Simple Fabric script example
# fabfile.py
from fabric.api import *
env.hosts = ['theodo@myserver.com']
def deploy():
with cd('/theodo/sflive2011'):
run('git pull')
run('./symfony doc:build --all --no-confirmation')
run('./symfony cc')
$ fab deploy

37. A good practise: scripting a rollback
Another Fabric example
# fabfile.py
def deploy():
tag = "prod/%s" % strftime("%Y/%m-%d-%H-%M-%S")
local('git tag -a %s -m "Prod"' % tag)
local('git push --tags')
with cd(path):
run('git fetch')
tag = run('git tag -l prod/* | sort | tail -n1')
run('git checkout ' + tag)
def rollback(num_revs=1):
with cd(path):
run('git fetch')
tag = run('git tag -l prod/* | sort | tail -n' +
str(1 + int(num_revs)) + ' | head -n1')
run('git checkout ' + tag)

38. And why not let Jenkins deploy
himself?
Continuous deployment

39. The Holy Grail of Rapid App Development & Deployment:
Automate everything low value-added
and relax

40. Isn't it dangerous to trust a machine?
Errare humanum est
Of course you need continuous integration with MANY tests
Of course you need some serious monitoring on the production server
Of course you need some good rollback scripts
But aren't that good things to do anyway ?
Good continuous integration is more reliable than a human!

41. You need to separate dev, pre-prod and prod...
Continuous deployment howto
For example with git:
features/* branches for small projects
dev branch for merging team development
master branch for production-ready code
prod/* tags for production

42. And you need a deployment script + Jenkins
Continuous deployment howto
Deployment script using Fabric (for example)
Jenkins (formerly known as Hudson) to test and deploy

43. Create a new Jenkins project testing only branch master

44. Specify "Build other projects" in the post-build actions

45. Don't forget to activate Chuck Norris

46. Create a second Jenkins project to execute the deploy script

47. That's it!

48. Next step

49. Links
docs.puppetlabs.com
fabfile.org
vagrantup.com
github.com/fabriceb/sflive2011vm

50. DevOps meetups
groups.google.com/group/paris-devops
and many more devops meetups around the world

51. Many thanks to Samuel @smaftoul Maftoul, organiser of the
Paris DevOps meetup, who bootstrapped me on DevOps!

52. Questions?
@theodo
fabriceb@theodo.fr