http://jedi.be/blogPatrick Debois
OPSDEV

When I was a young
developer http://www.flickr.com/photos/ninahiironniemi/1118868590/sizes/l/

http://1000awesomethings.files.wordpress.com/2008/11/campfire.jpg I loved to develop
simple applications

http://www.rookgordijn.net/wp-content/uploads/2009/08/Texel_21.JPG I kept my own
little server going

http://www.foodgeekery.com/fullimg/ribfest09-cowboy2.jpg Most of the time
, the results were fine

http://www.flickr.com/photos/abbyladybug/2644163076/sizes/l/ Sometimes the applications would
crash and I didn’t really know why

http://www.motoer75.nl/images/BBQ2009030.jpg But overall my friends
seemed to like my work

http://www.bfeedme.com/wp-content/uploads/2007/07/legends-of-texas-bbq-cookbook-of-the-day-7-30-07.jpg I learned from the
pro’s

http://www.freepatentsonline.com/6557545-0-large.jpg
Read about
software
architecture

http://patiopalace.com/assets/images/gas-grills/webber/certifiedexperts3.jpg I even got my
Scrum certification

http://www.flickr.com/photos/tradica/3071394619/sizes/l/ Still I was experiencing
the occasional problems

I blamed the
operations guys

http://media.photobucket.com/image/bbq%20cowboy/NotleyQue/The%20BBQ%20Family/CowboyChili.jpg I decided to leave
the cowboys

http://www.flickr.com/photos/iimaj/3959147070/sizes/o/ And make plans of
my own

http://www.flickr.com/photos/brachiator/107510145/sizes/l/ I really wanted to
focus on the customer

http://www.dongeovanny.com/images/meatpage2kobe.jpg I had the developers
use the coolest programming languages

http://www.bbqsaucereviews.com/wp-content/uploads/2009/06/bbqsaucereview-001.jpg Use the best frameworks
available

http://jeffreysmeatmarketblog.files.wordpress.com/2009/06/picture-4-3-0412.jpg We had the DBA’s
take care of the database schemas

http://www.flickr.com/photos/jayneandd/4078134280/sizes/l/http://www.flickr.com/photos/bhamsandwich/3601018752/sizes/l/ Explained them that User
Interface is important

http://lovefeasttable.com/blog/wp-content/uploads/2009/05/may09c067.jpg Up until now we
had done only some manual checks.

http://www.flickr.com/photos/cookthink/210948100/sizes/l/ So we moved to
unit tests

http://blogs.suntimes.com/food/4-29_white_BBQ_1.jpg
Even Behavior Driven Development

We were still experiencing problems
with our application servers.

http://www.flickr.com/photos/80417459@N00/2056898218/ We brought in bigger
storage boxes

http://euroross.blogspot.com/bbq.jpg gave the sysadmins the
biggest servers

http://propanesmoker.org/wp-content/uploads/2009/11/Propane_smoker.jpg created virtual machines to
create different environments

http://www.bbqpits.com/people_of_bbq/greg_nj3.jpg Used the cloud to
solve some scalability problems

http://www.flickr.com/photos/theogeo/3317291208/sizes/l/in/photostream/ Still the customer wasn’t
happy

http://www.funnypictures.net.au/images/firefighter-bbq1.jpg And the IT department
was firefighting all the time

http://www.flickr.com/photos/atesca/2457127172/sizes/l/ Then there was a
complete power failure

http://www.flickr.com/photos/zanyflash/2278286656/sizes/o/ Nobody had checked the
power installation

http://www.flickr.com/photos/cirox/363284194/sizes/l/ Leaving the application in
an unknown state

http://badgerdiary.files.wordpress.com/2008/12/closed-sign-smer.jpg Resulting again in a
major outage

http://www.nonplusultra.nl/data/producten/12727-groot.jpg Our small fixes would
not help anymore

http://www.proexotics.com/faq_images/soaking_water_monitor.jpg All backups were corrupt
because we could only backup properly when the application was down

http://www.flickr.com/photos/jmv/16366498/sizes/o/ Luckily we had a
cold Standby Backup

kr.com/photos/tom-poes/60373623/ Time to take a
step back

http://allwecaneat.files.wordpress.com/2008/08/_igp7826_sm.jpg We worked hard on
minimizing technical debt in our code

http://www.eggcookers.net/images/pictures/progressive-stainless-steel-bbq-tongs-bbq505.jpg http://ecx.images-amazon.com/images/I/41wSaS%2BMy8L.jpg We started to
improve our deployment tools

http://www.flickr.com/photos/wiphey/66525468/sizes/o/ Put monitoring in our
development and test environment

http://3.bp.blogspot.com/_ySMs_t20v08/Sw8m5JjVg1I/AAAAAAAAAFQ/jr2nXJYSfbI/s1600/DSC_4835.jpg Reused it in the
production environment

http://weblog.greenpeace.org/makingwaves/welldone430.jpg Implemented scaling from the
ground up

http://www.universalbarbeque.com/images/untitled-2_copy.jpg
We made deployment reproducible

http://www.flickr.com/photos/cjscott69/400160562/sizes/l/ Improved the way to
collect metrics

http://www.flickr.com/photos/namealus/3114627540/ To offload some of
the appservers we installed a caching system

http://www.flickr.com/photos/benchilada/4231838748/sizes/l/ And loadbalancers solved the
webserver scaling problem

http://www.izzysburgerspa.com/images/uploaded/image/Delivery%20Truck%201.jpg
Content Delivery Networks

http://strongdogz.com/wp-content/uploads/2008/09/beauceron.jpg Made our application security
proof

http://www.flickr.com/photos/loop_oh/3086747855/sizes/o/
Comply to
regulations

http://www.mzmfarm.net/linked/img_3507.jpg Integrated backups on the
application level

http://carrotsncake.com/wp-content/uploads/2009/03/img_6705.jpg Solved the multi-user problem
problem

For some know states We
would automatically reset the system Self healing on our database http://assets.whirlpool.com/files/usr/970/FilterReset5.jpg

So you want to be
a great agile chef?

http://www.damniwish.com/wp-content/uploads/2009/12/IMG_1265.jpg Software Infrastructure Presentation Operations
Testing Project #1 care for the whole lifecycle #2 look at it from a business perspective #3 work together as a team

It’s not about the software
It’s not about the hardware

I like my servers, software
steak well done http://www.flickr.com/photos/spenceke/2250116189/sizes/l/

IT Tooling is changing

Virtualization • kvm • xen
• solaris zones • vmware • virtualbox • ...

Cloud showed us how to
scale

Abstraction API http://www.jclouds.org/ Adrian Cole
@jclouds http://libvirt.org/

Platform as a Service

Hudson as Service
http://www.cloudbees.com/

But who will manage your
IAAS platform?

First steps of Automation https://fedorahosted.org/cobbler/
http://spacewalk.redhat.com/ http://fai-project.org/

Cloning

Virtual Sprawl

Configuration Management Chef Pallet http://www.cfengine.org/
http://www.puppetlabs.com http://www.opscode.com/chef https://github.com/ hugoduncan/pallet

# tomcat.pp class tomcat {
! package { ! ! "java-1.6.0-sun-compat": ! ! ! ensure => "installed"; ! ! "tomcat6": ! ! ! require => [Package["java-1.6.0-sun-compat"], Package["ntc-tomcat-log4j"]], ! ! ! ensure => "installed"; ! ! "tomcat6-admin-webapps": ! ! ! ensure => "installed"; ! ! "log4j": ! ! ! ensure => "installed"; ! ! "tanukiwrapper": ! ! ! ensure => "installed"; ! ! "mysql-connector-java": ! ! ! ensure => "installed"; ! } ! file { ! ! "/etc/init.d/tomcat6": mode => "0755", owner => "root", group => "root", source => "puppet:///tomcat/tomcat6.init"; } ! user { ! ! "tomcat": ! ! ! ensure => "present", ! ! ! require => Package["tomcat6"]; ! } }! Infrastructure as code

Model Driven Infrastructure

https://github.com/lusis/vogeler John E.Vincent @lusis Definition
of your infrastructure

Version Control

Reusable
components
https://github.com/camptocamp

Code/Server Smells Private!Playground The sysadmin
uses the system as his toy playground, doesn't clean up. • /tmp & /var/tmp full of old install files • / full of files Gready!Server One server that does every function • combined mail and web and dns and fileshares • all users on the same system Root is the cause of all!evil • last show login all root • no sudo is activated • no sshd keys for logins • nfs share/root? • Chmod 777 • most processes run as root

Sharing Recipes/Code
http://forge.puppetlabs.com/
http://cookbooks.opscode.com/

Automation
without tests is

Lindsay Holmwood @auxesis Feature: google.com
It should be up And I should be able to search for things Scenario: Searching for things When I visit "http://www.google.com" And I fill in "q" with "wikipedia" And I press "Google Search" Then I should see "www.wikipedia.org" $ cucumber-nagios google.feature Critical: 0, Warning: 0, 4 okay | passed=4, failed=0 cucumber-nagios

Lindsay Holmwood @auxesis Scenario: Checking
/etc/passwd When I ssh to "example.org" with the following credentials: | username | password | keyfile | | jacob | spifeofstrife | /home/jacob/.ssh/id_dsa | And I run "cat /etc/passwd" Then I should see "jacob" in the output cucumber-ssh

Julian Simpson @builddoctor apache.feature Feature:
In order to deploy my application As a systems administrator I want to know that the config files are legal Scenario: Valid Apache config Given a config file template apache2.conf in etc/apache2 When I generate it Then there should be a file called apache2.conf in etc/ap And it should be valid cucumber-apache http://pastie.org/693713

Feature: cucumber-puppet In order to
run my puppet manifest's test suite As an admin I want the cucumber-puppet gem installed Scenario: Install cucumber-puppet Given a node of class "cucumber-puppet" When I compile the catalog Then gem "cucumber-puppet" should be "installed" https://github.com/nistude/cucumber-puppet cucumber-puppet Nikolay Sturm @nistude

Orchestration
http://controltier.com/ http://marionett-collective.org
@ripienaar@damonedwards

Vagrant http://vagrantup.com/ Allows an easy
way to use the production recipes in your own dev environment Mitchell Hashimoto @mitchellh

IAAS and PAAS blurring Pull
in both your recipes and your code

http://blog.crisp.se/henrikkniberg/2008/07/14/1216002720000.html Agile Infrastructure & Agile
Sysadmin

Comparing Agile and ITIL http://www.winstonind.com/images/brands/traditional_cooking_process.gif
Agile ITIL

Certifications

Process
Fundamentalism

Manifesto’s for the world Meat
Cloud Manifesto Agile Manifesto == Rules == On Infrastructure —————– There is one system, not a collection of systems. The desired state of the system should be a known quantity. The “known quantity” must be machine parseable. The actual state of the system must self-correct to the desired state. The only authoritative source for the actual state of the system is the system The entire system must be deployable using source media and text files. On Buying Software ——————- ..... James White Manifesto

Truck Factor The number of
people on your team who have to be hit with a truck before the project is in serious trouble The ability to take a random machine in your infrastructure , drop it from the 10th floor, and be able to restore your infrastructure to working within 5-10 minu 10th floor test

Backlog of items • User
Stories • Features • Tickets • Incidents • Problems

Requirements • Login/Logout • Print
order • CRUD order • Security • Capacity • Stability • Scalability Functional Non-Functional

BusinessValue Ordered by the value
gained Ordered by the value protected Increase Change Avoid Change

Proxy Customers Service Manager Understand
what the business wants

Support the team Scrum Manager
Service Delivery Manager

Product Owner1 Ideally one product
owner Product Owner2 Product Owner3 Development Team Project1 Project 2 Project 3 Operations ManagerScrum Master Operations Team

Outsourcing
Call Centers
Outsourced
Development

Specialist & Generalists Rockstar Mentality
Testers UI Designers Frontend Backend Analysts Security Network Storage Sysadmins

Global Ownership It’s the javascript
It’s the backend It’s the database It’s the network It’s the server It’s the storage

Sprint Review Meeting Change Advisory
Board Approve Changes

Iterations & Flow

Kanban

1day sprints
Operations

Sprint Velocity Service Levels Team
Approved Estimate Improves over time Story Points Uptime 99%

Continuous Improvement Lean Thinking Lean
ITIL Lean Software Development

Value Stream Mapping

Technical waste

What’s new? I know all
that

The map is not the
territory Alfred Korzybski

Systems Thinking Optimize the whole
not just the individual parts

People make it happen

Culture & Mindset

Collaboration Testers Analysts Developers Security
Architects UI Designers Network Sysadmins not only when things go wrong

No rock star mentality Mine
are bigger then yours

Craftsmanship

Trust

It’s about customer
value
http://www.flickr.com/photos/spenceke/2250116189/sizes/l/

Interesting Books

Want to know more? #devops
on twitter & irc http://devopsdays.org http://devopscafe.org http://planetdevops.net devops googlegroup devops-toolchain googlegroup

The End