This document discusses Python package management and different tools available. It notes that pip, the default package manager, has some issues around dependency isolation and replication. Virtualenv helps address this by isolating packages by project, but still lacks locked dependencies. Pipenv improves on this by automatically creating virtual environments, locking dependencies, and managing development dependencies. Poetry is also introduced as a more full-featured tool for packaging and publishing Python projects and libraries. Overall, pipenv or poetry are recommended for most Python projects to simplify dependency and environment management.

1. PACKAGING INPACKAGING IN
PYTHON?PYTHON?
DON'T ROLL THE DICE.DON'T ROLL THE DICE.

2. EFFECTIVE PACKAGEEFFECTIVE PACKAGE
MANAGEMENT IS HARD.MANAGEMENT IS HARD.

3. WHAT SHOULD WE EXPECTWHAT SHOULD WE EXPECT
FROM PACKAGE MANAGERS?FROM PACKAGE MANAGERS?

4. RECONCILES THE BEST VERSION OF EACHRECONCILES THE BEST VERSION OF EACH
PACKAGE FOR ALL REQUIREMENTS.PACKAGE FOR ALL REQUIREMENTS.
setuptools >= 36.2.1
+ setuptools == *
+ setuptools <= 37.0.0
= setuptools 36.7.2

5. WORKS TO OPERATING SYSTEMWORKS TO OPERATING SYSTEM
CONSTRAINTS.CONSTRAINTS.
(Like Window's default 260 character path limit)

6. ENSURES A PERFECTLY REPLICABLEENSURES A PERFECTLY REPLICABLE
DEPENDENCY TREE.DEPENDENCY TREE.
(Not just the packages you rely on — also the
packages those packages rely upon, ad nauseum!)

7. SECURE.SECURE.
... NOT OBVIOUSLY INSECURE?... NOT OBVIOUSLY INSECURE?
Use different techniques to make sure the package
you downloaded is the one you expected.
Doesn't help if you expected a malicious package.

8. Some package managers will automatically
execute any scripts attached to a package as
pre/post-install.
The rest just show warnings that a user clicks
through.

9. OPTIMAL DOWNLOAD EXPERIENCE.OPTIMAL DOWNLOAD EXPERIENCE.
Use a combination of techniques to make package
downloads fast and reliable.

10. EMOJI.EMOJI.
Because yarn said so.

11. SO... PYTHON.SO... PYTHON.

12. SO... PYTHON.SO... PYTHON.

13. Python's current recommended package manager,
pip ,

14. Python's current recommended package manager,
pip ,
(which isn't generally installed by default if you're
using a Linux distribution's package manager),

15. Python's current recommended package manager,
pip ,
(which isn't generally installed by default if you're
using a Linux distribution's package manager),
(and also isn't installed by default on macOS),

16. Python's current recommended package manager,
pip ,
(which isn't generally installed by default if you're
using a Linux distribution's package manager),
(and also isn't installed by default on macOS),
is available by default on Windows when you
install Python!

17. RIGHT. EASY.RIGHT. EASY.
(USE PYENV EVERYWHERE, AND YOU'LL(USE PYENV EVERYWHERE, AND YOU'LL
HAVE PIP, AT LEAST)HAVE PIP, AT LEAST)

18. LINUX DISTRIBUTIONS PATCHLINUX DISTRIBUTIONS PATCH
PIP IN THEIR REPOSITORIES.PIP IN THEIR REPOSITORIES.
The default behaviour of trying to install packages
to system-level directories is not helpful for the
average Python developer.
sudo pip install is evil!

19. WORKS TO OPERATING SYSTEMWORKS TO OPERATING SYSTEM
CONSTRAINTS.CONSTRAINTS.
(Though, the Linux distributions generally do the
same thing in their own repositories in a slightly
different way — the behaviour can be desirable!)

20. WHAT IF YOU NEED TO INSTALL TWOWHAT IF YOU NEED TO INSTALL TWO
VERSIONS OF DJANGO AT THE SAME TIME?VERSIONS OF DJANGO AT THE SAME TIME?
THAT'S NOT DJANGO 1.11.15!THAT'S NOT DJANGO 1.11.15!
$ pip3 install --user django==2.1
...
Installing collected packages: django
Successfully installed django-2.1
$ pip3 install --user django==1.11.15
...
Installing collected packages: django
Successfully installed django-2.1

21. ENSURES A PERFECTLY REPLICABLEENSURES A PERFECTLY REPLICABLE
DEPENDENCY TREE.DEPENDENCY TREE.

22. EMOJI ASIDE,EMOJI ASIDE,
THAT'S TWO MAJOR ISSUESTHAT'S TWO MAJOR ISSUES
WITH DEFAULT PIP USAGE.WITH DEFAULT PIP USAGE.

23. VIRTUALENVVIRTUALENV

24. virtualenv is a tool to create isolated
Python environments.

25. spammish-repetition
django == 1.11.15
spambot > 2.0
postgres == *
pip
project-antigravity
six == *
reverse > 2.0
requests < 42.1.3
pin == 1.2.3.4
pip
python

26. THE MAGICAL VIRTUALENVTHE MAGICAL VIRTUALENV
ACTIVATE SCRIPTACTIVATE SCRIPT
activate sandboxes the session into using a
specific (redirected) Python interpreter, and keeps
packages inside the sandbox. It can optionally let
you access packages installed elsewhere in the
system, or restrict access entirely.

27. A VIRTUALENV PER PROJECTA VIRTUALENV PER PROJECT
If you use a virtualenv per project, each project
can have completely independent dependencies,
without any accidental upgrades or operations
that affect another project.

28. PIP FREEZEPIP FREEZE
Using the pip freeze command, you can get a
list of all packages installed by pip .
Inside a virtualenv , that's limited to packages
you installed there.

29. autopep8==1.3.5
colorama==0.3.9
mock==2.0.0
nose==1.3.7
pycodestyle==2.4.0
pylint==1.9.2
rope==0.10.7
six==1.11.0
virtualenv==16.0.0
virtualenv-clone==0.3.0

30. ENSURES A PERFECTLY REPLICABLEENSURES A PERFECTLY REPLICABLE
DEPENDENCY TREE?DEPENDENCY TREE?
NOT QUITE.NOT QUITE.

31. TRANSITIVE DEPENDENCIES AREN'TTRANSITIVE DEPENDENCIES AREN'T
LOCKED.LOCKED.
A package you rely upon might have a package it
relies upon update, breaking compatibility.
Then your production deploy won't run, even
though the "same dependencies" work on your
machine.

32. And as a side note, if you're forgetful like me,
sometime you forget to activate your
virtualenv .
Repeatedly.

33. PIPENVPIPENV

34. Python Development Workflow for
Humans

35. AUTOMATED VIRTUALENVS.AUTOMATED VIRTUALENVS.

36. TRANSITIVE DEPENDENCYTRANSITIVE DEPENDENCY
LOCKING.LOCKING.

37. ONE COMMAND TO USE ANDONE COMMAND TO USE AND
REMEMBER.REMEMBER.

38. DEPENDENCIES FROM GITHUBDEPENDENCIES FROM GITHUB
ETC.ETC.

39. SERIOUSLY, IT DOES MORE COOL THINGSSERIOUSLY, IT DOES MORE COOL THINGS
THAN I REALISED!THAN I REALISED!

40. $ pipenv --three
Virtualenv location: /home/liamdawson/.virtualenvs/wat-txLatJkZ
$ pipenv install django
Adding django to Pipfile's [packages]...
Pipfile.lock not found, creating...
Installing dependencies from Pipfile.lock (4f9dd2)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv ru
$ pipenv run python -m 'django-admin' startproject helloworld
$ ls
helloworld/ Pipfile Pipfile.lock

41. NO NEED TO MAKE ORNO NEED TO MAKE OR
ACTIVATE VIRTUALENVS.ACTIVATE VIRTUALENVS.
They're still there and in use, though — pipenv -
-venv .

42. CHECK IN THE PIPFILE.LOCKCHECK IN THE PIPFILE.LOCK
(Unless you're building a reusable library)
Now you have a locked dependency graph for fully
replicable builds!

43. HOW HARD IS IT TO MIGRATEHOW HARD IS IT TO MIGRATE
TO PIPENV?TO PIPENV?

44. $ git clone https://github.com/shalakhin/check_btc_tx.git
$ cd check_btc_tx
$ pipenv --three
requirements.txt found, instead of Pipfile! Converting...
$ pipenv install
# Check the last donation to The Internet Archive
$ pipenv run python check 1Archive1n2C579dMsAu3iC6tWzuQJz8dN
TXID: a75fe75...
Status: Confirmed
Amount: 0.15997978

45. TESTS?TESTS?
Handled natively as a "dev-only" dependency.
$ pipenv install --dev --requirements requirements-dev.txt
$ pipenv install --dev nose
$ head --lines=-3 Pipfile | tail --lines=+5
[dev-packages]
nose = "*"
[packages]
click = "*"
requests = "*"
$ pipenv install # --dev

46. REVERSIBLE?REVERSIBLE?
pipenv lock -r > requirements.txt

47. EMOJI?EMOJI?
Just not on Windows, where I ran my code.
(Time to write a PR!)

48. WHEN A NEW DEVELOPERWHEN A NEW DEVELOPER
STARTS ON YOUR PROJECT...STARTS ON YOUR PROJECT...

49. PIPENV INSTALLPIPENV INSTALL

50. If they're using pyenv , it installs the right
Python version for them.
Sets up a virtualenv for the project.
Installs the requirements (and dev requirements
if you pass --dev ).

51. Can run scripts you set up, such as pipenv run
format .
Can still replicate activate behaviour from the
virtualenv via pipenv shell

52. PREVIOUS EXAMPLE BECOMESPREVIOUS EXAMPLE BECOMES
1. git clone
2. pipenv install --dev
3. pipenv run check
4.

53. WHERE DOES PIPENV FALLWHERE DOES PIPENV FALL
SHORT?SHORT?

54. PIPENV IS A TOP-NOTCH TOOL FORPIPENV IS A TOP-NOTCH TOOL FOR
MANAGING AN APPLICATION WHICH:MANAGING AN APPLICATION WHICH:
Runs on one version of Python.
Isn't used as a dependency.
Isn't packaged for PyPi etc.

55. PIPFILES DECLARE AN EXPECTED VERSIONPIPFILES DECLARE AN EXPECTED VERSION
OF PYTHONOF PYTHON

56. PIPFILE.LOCK RIGIDLY FIXES DEPENDENCYPIPFILE.LOCK RIGIDLY FIXES DEPENDENCY
VERSIONSVERSIONS
(which doesn't suit usage as a dependency)

57. PIPENV WON'T HELP WITH PACKAGEPIPENV WON'T HELP WITH PACKAGE
CONFIGURATIONCONFIGURATION

58. IF THOSE ASSUMPTIONS DON'TIF THOSE ASSUMPTIONS DON'T
SUIT YOU?SUIT YOU?

59. TRY POETRYTRY POETRY
(the tool)

60. POETRY IS A SOMEWHAT FULLER-POETRY IS A SOMEWHAT FULLER-
FEATURED TOOL FOR DEPENDENCYFEATURED TOOL FOR DEPENDENCY
MANAGEMENT.MANAGEMENT.

61. Poetry is a little more involved, and uses more
metadata.
This equips it to support packaging and
publishing, in addition to dependency
management.

62. IF YOU USE PYENV TO MANAGE YOURIF YOU USE PYENV TO MANAGE YOUR
PYTHON VERSIONSPYTHON VERSIONS
AND PIPENV OR POETRY TO MANAGEAND PIPENV OR POETRY TO MANAGE
YOUR PROJECTSYOUR PROJECTS

63. IT'S A LOT LESS BITE FOR EVERYONE.IT'S A LOT LESS BITE FOR EVERYONE.
Party danger noodle!

64. FIN.FIN.
Want to follow up later? I'm on Twitter:
@LIAMDAWS@LIAMDAWS

65. CREDITSCREDITS
Elephant photo by on
Unsplash
Original snake photo by on
Unsplash
Inbetween Architects
Boris Smokrovic