Dependency Management in Python and a look at - pip-tools - Pipenv - Poetry - Hatch

1. Python Dependency
Management
Patrick Muehlbauer (@tmuxbee)
2018-10-24

2. • Predicting bananas
• ...with Python
Blue Yonder

3. Python Dependency Management

4. banana_predictions.py

5. How to provide this to others?

6. $ python setup.py sdist
banana-predictions-1.0.0.tar.gz

7. $ python setup.py bdist_wheel
banana_predictions-1.0.0-py3-none-any.whl

8. How to upload to PyPI?

9. $ python setup.py upload
but no HTTPS

10. twine

11. How to specify dependencies?

12. setuptools’ install_requires

13. How to install other uploaded packages?

14. pip

15. Different versions of ‘banana_predictions’ in
different projects?

16. virtualenv
$ virtualenv –p python3.7 venv

17. Reproducible environments? (I)

18. $ pip freeze > requirements.txt
$ pip install –r requirements.txt

19. • multiple tools for the whole workflow
• no real dependency resolution in pip
• multiple files needed for reproducible environments
(requirements.txt, dev-requirements.txt, …)

20. Reproducible environments? (II)

21. Pipfile and Pipfile.lock
https://github.com/pypa/pipfile

22. distutils/setuptools limitations

23. pyproject.toml
PEP 517 -- A build-system independent format for source trees
PEP 518 -- Specifying Minimum Build System Requirements for Python Projects

24. Python Dependency Managers

25. pip-tools
Pipenv
poetry hatch
condaflit
Bento
Pants / pex

26. pip-tools
Pipenv
poetry hatch

27. pip-tools
• set of command line tools: pip-compile and pip-sync
• pip-compile for dependency resolution: generates requirements.txt
from setup.py or a requirements.in
• pip-sync for dependency synchronization

28. setup.py
pip-compile
requirements.txt

29. $ pip-compile --generate-hashes

30. $ pip-compile --upgrade

31. pip-tools brings dependency resolution and a
nicer pinning workflow

32. Pipenv
• wrapper around pip and virtual environments
• lots of convenience features (e.g. automatic virtualenv creation,
python installation)
• uses Pipfile and Pipfile.lock in addtion to setup.py
• main focus on applications rather than libraries

33. Pipfile
• separate sections for dev and
main dependencies instead of
requirements.txt and dev-
requirements.txt
• optional section to define
required Python version

34. Pipfile.lock

35. $ pipenv install scipy
$ pipenv uninstall scipy
update Pipfile and Pipfile.lock

36. Poetry
• tool to manage dependencies, build and publish packages
• automatic virtualenv creation
• single config file for everything: pyproject.toml, poetry.lock for
reproduciblity
• reliable dependency resolution
• project scaffolding

37. $ poetry new my-project

38. $ poetry add scipy

39. $ poetry version

40. $ poetry build
sdist and .whl

41. $ poetry publish

42. poetry.lock

43. hatch
• simplifies development workflow
• wraps multiple tools (pip, virtualenv, twine, pytest, coverage.py, …)
• setup.py + requirements.txt

44. $ hatch new my-project

45. $ hatch install

46. $ hatch test

47. $ hatch grow

48. $ hatch build
sdist and .whl

49. $ hatch release

50. So...which tool should I use?

52. It depends...

53. • building a library?
• or an application?
• does it fit my infrastructure?

54. Thanks