Using Fabric for deploying and administering Django applications

1. Jonas Nockert
jonasnockert@gmail.com
@lemonad
Fabric
Django deployment
Stockholm Django User Group, November 9, 2009
http://jonasnockert.com/2009/11/fabric-django-deployment/
Photo credit

2. Great! Your Django app is done!

3. Now all you have to do is...
• fix bugs and implement new • keep track of local_settings
features
• reload wsgi files
• git archive
• Change symbolic links (yay,
• move files from development deployed!)
to staging and production
servers • empty and pre-warm cache
• install updates to django, • oops! forgot to migrate data!
reusable apps, pypi
packages, etc.
• run backups
(for the rest of your life)

4. Wouldn’t it be nice if all you had to
do was this?
$ fab staging deploy
$ fab production deploy (yay, backup’d
and deployed!)

5. Fabric is easy!
Photo credit

6. Basic building blocks (the core API)
• run — run a command on a remote host.
• sudo — run a sudoed command on a remote host.
• local — run a command on the local host.
• get/put — copy a file from/to a remote host.
• prompt — ask the user for information
• For everything else there’s Python.

7. Execute commands
• Get output and return code:
output = run(“command”)
• Chain commands:
run(“workon project && git pull origin master”)
• start pseudo-daemons:
run(“screen -d -m not-a-daemon”)
• Any return code except 0 is treated as an error and an exception is
thrown (unless warn_only is set)

8. Move files
• Put a file (upload):
put(“local-file”, “remote-file”, mode=0755)
• Get a file (download):
get(“remote-file”, “local-file”)
• If a problem occurs while uploading or downloading files, fabric
throws an exception.

9. Ask questions
• Validation:
relase_name = prompt(“What do you want to “
“name the new release?: ”,
validate=”^[a-zA-Z0-9]+$”)
proxy_port_number = prompt(“Proxy port: “,
validate=int)
• Default:
username = prompt(“Username: “,
default=”jonas”)

10. Keeping state (sort of)
• Directories:
with cd(“xmpp/”):
run(“git clone git://git/something.git”)
run(“mkvirtualenv --no-site-packages something“)
• For those running from trunk:
with prefix(“workon something”):
run(“pip install Django>=1.1”)
Observe that the name is not set in stone yet.
c.f. http://stackoverflow.com/questions/1180411/activate-a-virtualenv-via-fabric-as-deploy-user

11. Contrib
• Contrib is a collection of useful tools. Search and replace in remote
files amongst other things.
• However, it is not as evolved as the Fabric core API and contains, in
Jeff Forcier’s own words, “approaches that work for Jeff on two
different Linux distros and that’s about it.”

12. Append text to files
• Appends text unless it already exist in file as a discrete line:
from fabric.contrib.files import append
append(“DATABASE_NAME = 'hello.db'”,
“local_settings.py”)

13. Search and replace
• Use fabric’s sed function to modify content of files:
from fabric.contrib.files import sed
sed(“local_settings.py”,
“^DEBUG = True$”,
“DEBUG = False”)

14. Example
from fabric.api import run
def install_django():
run('workon myproject && pip install django')
$ fab -H localhost,djangodev install_django
This will install Django within the myproject virtualenv on both the local
machine and on the host djangodev

15. While Fabric is easy,
deployment is hard!
(but rewarding)
Photo credit

16. Rebuilding from scratch?
• Define scratch Depends to some degree on
what you want to achieve
Do you start from a given and if it’s limited to
virtual machine image? deployment purposes.
...or just your application and Perhaps you just want to be
its dependencies? able to easily test against a
new release of Ubuntu,
...or just your application? perhaps a new release of
Django.

17. What about the database?
• Do you migrate the current • Apply migrations to a copy?
one?
When you’ve deployed, the
You could backup and copy becomes the new
restore if something goes production DB.
wrong.
Make sure you have a way
But you can’t reliably keep of finding out the name of
the site up while migrating. your current production DB
(in order to deploy the next
time).
Database-level permissions
might need to be handled.

18. Testing?
• Cancel and revert deployment if the test suite fails!
• This is actually pretty simple:
run(“workon project && python manage.py test”)
Django returns the number of failed tests or 0 if all tests pass, and
run throws an exception if the return code is anything but 0.
All you have to worry about is reverting to the state before
deployment started.

19. What about the live site?
• Show an update banner on • Set database layer to read-
the site? only?
• Set Django application to Have you designed your
application to handle SQL
read-only?
errors nicely?
Have you separated code
that reads from the DB from • Shut down Apache? What if
code that writes? you have multiple
applications?
• You’ll probably need to work
on a separate DB while
updating and migrating,
right?

20. Trying to combine it all
1. Get source code
2. Create virtualenv
3. Use buildout to install dependencies
4. Copy local configuration for testing and production from local
repository (as to avoid checking in passwords)
5. Show upgrade banner on site
6. Close database for writing
7. Fetch the name of the current production database
8. Clone database
9. Modify Django settings to point at cloned database
10.Migrate using South
11.Run tests with testing configuration
12.Re-index site using your chosen search backend
13.Switch symlink from pointing to previous production site to new (site
should reload automatically since wsgi file is updated)
14.Open DB for writing

21. Good luck!
Photo credit