Using virtualenv, pip, and fabric

1. repeatable INSTALLS and DEPLOYMENTS
PIP, VIRTUALENV and FABRIC
IDAN GAZIT
PyWeb-IL 10, Nov 30th 2009

2. SO YOU HAVE A PROJECT
HOORAY.

3. NOW YOU HAVE A
NEW DEVELOPER
NEW SERVER
NEW VERSION

4. NOW YOU HAVE A HUNDRED
NEW DEVELOPERS
NEW SERVERS
NEW VERSIONS

5. MAKING SHIT
OVERHEAD

6. http://www.flickr.com/photos/derpunk/3875534682

7. THE OVERHEAD WILL KILL YOU
KILL IT FIRST

8. ONE CLICK
ANY MORE AND YOU'RE F*CKED

9. ELIMINATE OVERHEAD
THESE TOOLS WILL HELP

10. ISOLATION
PROTECT YOUR APP FROM OTHER APPS
REPEATABILITY
PULL TOGETHER ALL THE PARTS OF YOUR APP
SCALABILITY
O(1) DEPLOYMENTS

11. VIRTUALENV
by IAN BICKING

12. http://www.flickr.com/photos/fensterbme/145621388

13. LIKE GOGGLES FOR YOUR PYTHON
NOT REALLY.

14. ISOLATED PYTHON ENVIRONMENTS
"PRIVATE" SITE-PACKAGES

15. APP X APP Y
FOO 0.8 BLAH 2.2
BAR 1.23 BAR 1.23
BAZ 6.0c9 BLING 9.1b2

16. APP X APP Y
FOO 0.8 BLAH 2.2
BAR 1.23 BAR 1.24
BAZ 6.0c9 BLING 9.1b2

17. GLOBAL SITE-PACKAGES
LIKE A VIRGIN

18. $ python virtualenv.py ENV
CREATE THE VIRTUALENV

19. --no-site-packages
COMPLETE ISOLATION

20. ME, PERSONALLY?
A FEW GLOBAL PACKAGES (IPYTHON, IPDB)

21. $ source ENV/bin/activate
TO ENTER THE MATRIX

22. (ENV)$ easy_install kungfu
WOAH.

23. (ENV)$ python theone.py
I CAN DODGE BULLETS

24. (ENV)$ deactivate
OPERATOR, I NEED AN EXIT!

25. MORE USE CASES
DIFFERENT VERSIONS OF THE SAME APP

26. DEPLOYING?
SETUP THE SAME VIRTUALENV

27. WEB APPS
IT'S JUST PYTHONPATH

28. INDEPENDENT VIRTUALENV
YOU GOTTA KEEP'EM SEPARATED

29. http://www.flickr.com/photos/wendycopley/3025707754

30. VIRTUALENVWRAPPER
by DOUG HELLMANN

31. $ mkvirtualenv ENV
$ workon ENV

32. PIP
also by IAN BICKING

33. PIP INSTALLS PACKAGES
GOODBYE, EASY_INSTALL

34. SMARTER
NO PARTIAL INSTALLS
NON-SUCKY CONSOLE OUTPUT
INSTALL FROM VCS
REQUIREMENTS FILES
UNINSTALL

35. $(ENV) pip install -E ENV mypackage
PIP is VIRTUALENV-AWARE

36. $(ENV) pip freeze -E ENV > reqs.txt
BOOM. I HAZ REQUIREMENTZ FILE.

37. $(ENV) pip install -E ENV -r reqs.txt
SIT BACK AND RELAX

38. REQUIREMENTS FILE
FRAMEWORK==0.2
LIBRARY>=1.3
-e svn+http://myrepo/svn/MyApp#egg=MyApp

39. SUPPORTED VCS's
SUBVERSION
GIT
MERCURIAL
BAZAAR

40. FRESH DEPLOY
1. PULL FROM VCS
2. CREATE VENV
3. PIP INSTALL FROM REQUIREMENTS FILE
4. THERE IS NO STEP FOUR

41. ROCKSTAR.
http://www.flickr.com/photos/an-di/274127482

42. NOT SO FAST
4 STEPS IS 3 STEPS TOO MANY

43. FABRIC
maintained by JEFF FORCIER

44. AUTOMATE
REMOTE COMMANDS OVER SSH
(LOCAL COMMANDS TOO)

45. IT'S KIND OF LIKE ‘MAKE'
EXCEPT PYTHONIC AND DELICIOUS

46. FABFILE.PY

47. FABFILE.PY
from __future__ import with_statement
from fabric.api import *
env.hosts = ['server1.foo.com', 'server2.foo.com']
def test():
with settings(warn_only=True):
result = local('./manage.py test my_app', capture=False)
if result.failed:
abort("Aborting at user request.")
def pack():
local('tar czf /tmp/my_project.tgz .', capture=False)
def prepare():
test()
pack()
def deploy():
put('/tmp/my_project.tgz', '/tmp/')
with cd('/srv/django/my_project/'):
run('tar xzf /tmp/my_project.tgz')
run('touch app.wsgi')

48. $ fab prepare
RUNS test() AND pack()

49. $ fab prepare deploy
ONE STEP NIRVANA.

50. FABFILE.PY
from __future__ import with_statement
from fabric.api import *
def staging():
env.hosts = ['staging.foo.com']
def production():
env.hosts = ['prod1.foo.com', 'prod2.foo.com']
# ... more fab actions ...
def deploy():
put('/tmp/my_project.tgz', '/tmp/')
with cd('/srv/django/my_project/'):
run('tar xzf /tmp/my_project.tgz')
run('touch app.wsgi')

51. $ fab staging deploy
DEPLOY TO STAGING IN ONE LINE

52. FABRIC BUILTINS
LOCAL
RUN
SUDO
GET/PUT
PROMPT

53. VIRTUALENV
TO ISOLATE YOUR PROJECT
PIP
REPEATABLE DEPENDENCY INSTALLS
FABRIC
FOR AUTOMATED DEPLOYMENTS

54. VIRTUALENV
http://pypi.python.org/pypi/virtualenv
PIP
http://pip.openplans.org/
FABRIC
http://docs.fabfile.org/

55. QUESTIONS?

56. THANK YOU!
@IDANGAZIT
IDAN@PIXANE.COM
http://bit.ly/repeatable-deployments