"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
Hudson: Your robotic butler
1. Hudson
Your Robotic Butler
by @stevenmerrill
steven@treehouseagency.com
http://dgo.to/@smerrill
Saturday, July 24, 2010
2. Hudson
• Java-based continuous integration engine
• Used to automate any regular job
• Send notifications through a variety of channels
Saturday, July 24, 2010
3. Why Use It?
• Automate tasks that developers should run
• Get detailed console logs for jobs without
inserting into your Drupal database
• Run jobs on any number of servers and chain
them together contingent on the success
• Notify users in a variety of media
Saturday, July 24, 2010
5. Getting Hudson
• Hudson is packaged as a .war file
• Run it on a standard servlet container
• Hudson labs also maintains a .deb file and an
apt repository
Saturday, July 24, 2010
6. Setup Tips
• Run the Hudson server as a user with umask
002 and with the same primary group as your
web server
• This will make sure that permissions are correct
for files when DrupalWebTestCase::drupalGet()
or the like invoke your web server
• http://drupal.org/node/838852
Saturday, July 24, 2010
7. Setup, cont’d
• You can also use Apache (or nginx, etc) to
proxy Hudson so that you can run it without
having to use :8080 in the URL
• Use Apache’s mod_proxy and the instructions
at http://wiki.hudson-ci.org/display/HUDSON/
Running+Hudson+behind+Apache
Saturday, July 24, 2010
8. Configuring on Ubuntu
• The simplest way to get up and running with
Hudson is with Ubuntu and their apt repository
• http://pkg.hudson-labs.org/debian/
Saturday, July 24, 2010
9. Configuring on Ubuntu, cont’d
• wget -q -O - http://pkg.hudson-labs.org/debian/
hudson-labs.org.key | sudo apt-key add -
• sudo sh -c "echo 'deb http://pkg.hudson-labs.org/
debian binary/' >> /etc/apt/sources.list"
• sudo apt-get update
• sudo apt-get install hudson
• sudo make me a sandwich (this will take awhile)
Saturday, July 24, 2010
10. Configuring on Ubuntu, cont’d
• sudo usermod -G www-data hudson
• sudo nano -w /etc/init.d/hudson
• Add --umask=002 to DAEMON_ARGS
• sudo service hudson stop
• sudo service hudson start
Saturday, July 24, 2010
11. You now have a robotic butler at your command.
Saturday, July 24, 2010
12. Terminology
• Every task you wish to automate using Hudson will is
called a job
• The directory on the file system where a job’s source
files are checked out it the workspace
• You create numerous build steps to execute
whenever the job is run, which is called a build
• You can take any number of files produced by the
build and save them as artifacts
Saturday, July 24, 2010
13. Terminology, cont’d
• Each build has two status axes
• Failure or success
• Stable or unstable
Saturday, July 24, 2010
14. This is Drizzle’s build farm at http://hudson.drizzle.org/.
Saturday, July 24, 2010
15. Working with Jobs
Credit: http://www.flickr.com/photos/techshownetwork/2943427637/
Saturday, July 24, 2010
16. Job Configuration
• Each job has a unique name
• Name your jobs like Drupal variables -
lowercase and underscores
• On the Ubuntu install, a folder will be created in
/var/lib/hudson/jobs/[job name] .
Saturday, July 24, 2010
17. Source Control Integration
• Hudson can check source control for you and build
your project when there are changes
• Hudson will do a checkout and then grab any available
changes and start a build once it has grabbed them
• Other advantages include being able to have the
notification plugins send out a list of changes and the
ability to create Hudson users for each person who
commits to your repository
Saturday, July 24, 2010
18. Source Control, cont’d
• For the job where you’ll be running your unit
tests from, make sure you do not choose the
option to do a clean checkout every time.
• There are source control integration plugins for
nearly every VCS out there, including the open-
source standards cvs, svn, hg, bzr and git, and
proprietary ones such as Perforce, ClearCase
and even Team Foundation Server.
Saturday, July 24, 2010
19. Notifications
• Hudson has built-in notifications via email
• Via user integration, it can also email users who have
broken or fixed the build
• Many plugins exist for other notification media, such
as multi-protocol IM and IRC, suitable for running in
a team IRC room
• If you enter committers IRC nicks / IM handles, the
bot will praise or chastise them
Saturday, July 24, 2010
20. Slave Builds
• Hudson can install a Java-based slave via SSH
on any *nix machine
• The slave will run builds on the local machine
and report build status back to Hudson
• Jobs can be tied to any open slave (better for
jobs like running cron) or to a specific machine
(often used for deployments)
Saturday, July 24, 2010
21. Build Steps
Credit: http://www.flickr.com/photos/mbgrigby/3427948763
Saturday, July 24, 2010
22. Build Steps
• The meat of any job is the build steps that it
goes through
• When used with *nix machines, these build
steps are shell scripts
• Hudson sets a number of environmental
variables when these jobs are run
Saturday, July 24, 2010
23. Build Steps, cont’d
• $WORKSPACE - the directory with the VCS code
• $BUILD_URL - the URL to this specific build
• $BUILD_NUMBER - the integer build number
• In addition, you can create parameterized builds
where a default value is set, but users running
builds in the Hudson interface can specify a
different value on a per-build basis
Saturday, July 24, 2010
24. Build Steps, cont’d
• When you look in the console for a build that
runs, you’ll see that it runs sh -xe and then the
scripts that you have put into the build steps
• This option will make it so that if any step in the
process returns a non-zero exit code, the build
will fail
Saturday, July 24, 2010
25. Testing
• The most obvious use for testing is to run
Drupal SimpleTests on every checkin
• To do this, set up a job connected to your
source control and that checks out your site
• Install the site on the machine and make sure
that the SimpleTest module is enabled
Saturday, July 24, 2010
26. Testing, cont’d
• With the SimpleTest module enabled, you can
use the scripts/run-tests.sh script to run
SimpleTests from the command line
• These tests will be run as the Hudson user
when run through Hudson, so that’s where the
umask 002 we set up earlier is important
Saturday, July 24, 2010
27. Testing, cont’d
• The SimpleTest module in Pressflow 6 contains
a patch to allow the SimpleTest run-tests.sh
script to output test results in the JUnit XML
format using the --xml switch
• Hudson will show your test successes and
failure and to make a build unstable if tests fail
• http://drupal.org/node/602332
Saturday, July 24, 2010
28. Linking Jobs
Credit: http://www.flickr.com/photos/lopolis/2623060590/
Saturday, July 24, 2010
29. Other Scheduling Options
• In addition to having a build run whenever there
are new changes on a branch in source control,
you can schedule builds:
• Using a cron-like syntax
• To run when another completes successfully
• By hand or with a GET to /job/[job name]/build
Saturday, July 24, 2010
30. Other Scheduling, cont’d
• When you make a chain of builds, one failure in
a build in the chain will stop the whole thing
• You can also select to make a single unstable
build stop the chain
Saturday, July 24, 2010
31. Deployment
Credit: http://www.flickr.com/photos/mateus27_24-25/2760653277/
Saturday, July 24, 2010
32. Deployment
• Hudson is a great tool for automating
deployments
• A common workflow when working with DVCS
systems is to have an integration branch or
repository that changes get pushed to
• When new changes appear in the integration
repo, Hudson can check it and run tests
Saturday, July 24, 2010
33. Deployment, cont’d
• Using drush (and optionally its master/slave
support) it is quite easy to automate updating
other environments
• Provided that all important schema updates are
in code, drush can run all these automatically in
a deploy task
Saturday, July 24, 2010
34. Deployment, cont’d
• The following script can be used as a deploy
script on a local or remote machine:
• drush -y updb
• drush -y features-update-all
• drush -y cc all
Saturday, July 24, 2010
35. Other Cool Tricks
Credit: http://www.flickr.com/photos/nowhere77/2571139625/
Saturday, July 24, 2010
36. Run cron
• David Strauss of Four Kitchens similarly
elucidates on why Drush and Hudson rock and
how you can use them together
• http://fourkitchens.com/blog/2010/05/09/drop-
cron-use-hudson-instead
Saturday, July 24, 2010
37. View Status in NetBeans
• NetBeans includes a Hudson plugin in core.
Saturday, July 24, 2010
38. Track Back-End Performance
• Make a build that does some GET requests
against a production site
• Set “XDEBUG_PROFILE” in $_GET, $_POST or
$_COOKIE to get a cachegrind file
• Run the cachegrind file through xdebugtoolkit to
get a nice PDF to show where your execution
time goes and track it over time.
Saturday, July 24, 2010
39. Yellow items are called a lot, red items take a lot of time per call.
Magenta items are both.
Saturday, July 24, 2010
40. Track Front-End Performance
• Install ShowSlow from showslow.com
• Use xtightvnc to make a virtual display that
ShowSlow’s integration scripts can direct
Firefox at on a set schedule
• Compare your Google Page Speed and YSlow!
scores over time
Sorry, a magician doesn’t reveal his secrets, and I haven’t had enough time
to write up these instructions yet. But it works - Trust Me™.
Saturday, July 24, 2010
41. Thanks
• The whole @hudsonci team for their work in
creating this amazing tool.
• Steven Jones for his work on JUnit XML output
for SimpleTests.
• The Pressflow team for including the patch in
Pressflow 6.
Community plumbing, baby.
Saturday, July 24, 2010