This document discusses using Python on Red Hat platforms. It covers:
- Deploying Python applications on OpenShift using Source-to-Image (S2I)
- Python base images for building applications
- Migrating existing apps from OpenShift 2 to 3
- Using Software Collections to run newer Python versions without impacting the system
- Modernizing the Python networking stack and defaulting HTTPS certificate verification
Developing Python Apps and Services on Red Hat Platforms
1. Developing in Python
on Red Hat platforms
Nick Coghlan
Senior Software Engineer
Graham Dumpleton
Developer Advocate for OpenShift
June 28th
2016
2. Using Python on Red Hat Platforms
● Python for Network Services
● Python for Applications
● Python for System Administration
3. Current Transitions in the Python Ecosystem
● Migration to Python 3
● Modernizing the Python 2.7 Network Security Stack
● Defaulting to HTTPS Certificate Verification
12. $ oc new-app https://gitlab.com/osevg/python-django-modwsgi.git
--> Found image 772dc19 (4 weeks old) in image stream "python" in project "openshift" under tag "3.4" for "python"
Python 3.4
----------
Platform for building and running Python 3.4 applications
Tags: builder, python, python34, rh-python34
* The source repository appears to match: python
* A source build using source code from https://gitlab.com/osevg/python-django-modwsgi.git will be created
* The resulting image will be pushed to image stream "python-django-modwsgi:latest"
* This image will be deployed in deployment config "python-django-modwsgi"
* Port 8080/tcp will be load balanced by service "python-django-modwsgi"
* Other containers can access this service through the hostname "python-django-modwsgi"
--> Creating resources with label app=python-django-modwsgi ...
imagestream "python-django-modwsgi" created
buildconfig "python-django-modwsgi" created
deploymentconfig "python-django-modwsgi" created
service "python-django-modwsgi" created
--> Success
Build scheduled, use 'oc logs -f bc/python-django-modwsgi' to track its progress.
Run 'oc status' to view your app.
$ oc expose svc/python-django-modwsgi
route "python-django-modwsgi" exposed
13. Source to Image (S2I)
https://github.com/openshift/source-to-image
14. $ s2i build https://gitlab.com/osevg/python-django-modwsgi.git centos/python-34-centos7 python-django-modwsgi
I0610 10:02:07.422470 84805 docker.go:352] Image "centos/python-34-centos7:latest" not available locally, pulling ...
I0610 10:02:36.501637 84805 clone.go:32] Downloading "https://gitlab.com/osevg/python-django-modwsgi.git" ...
I0610 10:02:38.831859 84805 install.go:251] Using "assemble" installed from "image:///usr/libexec/s2i/assemble"
I0610 10:02:38.831913 84805 install.go:251] Using "run" installed from "image:///usr/libexec/s2i/run"
I0610 10:02:38.831943 84805 install.go:251] Using "save-artifacts" installed from "image:///usr/libexec/s2i/save-artifacts"
I0610 10:02:38.832495 84805 environment.go:60] Setting 1 environment variables provided by environment file in sources
---> Copying application source ...
---> Installing dependencies ...
...
---> Collecting Django static files ...
I0610 10:03:00.876021 84805 environment.go:60] Setting 1 environment variables provided by environment file in sources
$ docker run --rm -p 8080:8080 python-django-modwsgi
---> Running application from Python script (app.py) ...
[Fri Jun 10 00:07:33.580013 2016] [mpm_event:notice] [pid 1:tid 139758878566464] AH00489: Apache/2.4.6 (CentOS)
mod_wsgi/4.5.2 Python/3.4.2 configured -- resuming normal operations
[Fri Jun 10 00:07:33.580149 2016] [core:notice] [pid 1:tid 139758878566464] AH00094: Command line: 'httpd (mod_wsgi-
express) -f /tmp/mod_wsgi-localhost:8080:1001/httpd.conf -D MOD_WSGI_MULTIPROCESS -D
MOD_WSGI_WITH_PROXY_HEADERS -D MOD_WSGI_MPM_ENABLE_EVENT_MODULE -D
MOD_WSGI_MPM_EXISTS_EVENT_MODULE -D MOD_WSGI_MPM_EXISTS_WORKER_MODULE -D
MOD_WSGI_MPM_EXISTS_PREFORK_MODULE -D FOREGROUND'
16. Migrating from OpenShift 2 to 3
● Converting of existing applications.
● Backward compatible S2I builder.
● Guidelines for porting applications.
● Templates to aid in porting applications.
19. Why *Not* Containers?
● Containers are the recommended option for network services
● However:
– Container support for rich desktop applications is currently limited
– Container runtime may impose unwanted overhead on dedicated systems
– Containers may give more isolation than is wanted
– Applications may require non-trivial modification to run as a privileged container
● Software Collections aim to offer “minimum viable runtime isolation”
– Add new executable directories to front of PATH
– Add new shared library directories to front of LD_LIBRARY_PATH
20. Why Software Collections for Python?
● Use newer Python runtimes without impacting system components
● Use a common Python runtime across multiple operating system versions
● Python 3 for Red Hat Enterprise Linux 6 & 7!
21. Red Hat Software Collections
● Platforms
– Red Hat Enterprise Linux 6 & Red Hat Enterprise Linux 7
– CentOS 6 and 7 (via softwarecollections.org)
– Basis for OpenShift language runtimes
● Available versions (as of RHSCL 2.2)
– Python 2.7.8 (+ selected backports)
– Python 3.5.1 (+ selected backports)
– Python 3.4.2 (+ selected backports)
– Python 3.3.2 (+ selected backports)
22. Python Virtual Environments
● Software Collections allow multiple runtimes to share a system without conflicting
● Virtual environments allow multiple Python applications to share a runtime
● Low or no runtime overhead: just add/replace directories in Python’s import path
● Cleanly isolate application dependencies from platform components
● Dependencies within the environment managed with pip
● Created via:
– python3 -m venv (Python 3.4+)
– virtualenv (Python 2.7, Python 3.x)
● Not included in the Red Hat Enterprise Linux System Python
23.
24. Constructing a Layered Application
● Base platform (via system package manager):
– Operating system (e.g. kernel, C runtime)
– Language runtime (from Software Collections)
– Other external dependencies (e.g. OpenSSL)
● Modify shared library loading (via enabled Software Collection)
● Modify Python import configuration (via virtualenv or standard library’s venv)
● Inside the virtual environment:
– Application dependencies (managed via pip)
– Application source code (managed via pip, or direct from source control)
25. Deploying a Layered Application (1)
● Application RPM with generated artifacts in SRPM:
– Create full installation in representative environment
– Bundle entire virtualenv and other desired components into SRPM
– Also include scripts to appropriately activate SCL and virtual environment
● Some generated files will end up in the SRPM
– Pre-compiled Python files
– Executable wrappers for pip managed Python scripts
26. Deploying a Layered Application (2)
● Application RPM with source-only SRPM:
– SRPM contains source for application and any application level dependencies
– virtual environment created and configured during RPM build process
● Not yet fully supported in pip
– some pip generated metadata will incorrectly include RPM buildroot paths
– shouldn’t matter for most RPM based deployment use cases
27. Managing Application Dependencies
● The Python Package Index is not an App Store!
● Designed to minimize barriers to publication:
– No pre-publication review
– Publisher Terms of Service ensure right to redistribute, not to run or modify
– Publishers may delete (but not replace) previously published versions
● Recommendation: use caching proxies and a component review process
– Commercially supported options: JFrog Artifactory, Sonatype Nexus
– Community/self-supported options: devpi, Python plugin for Pulp
– Check licensing, export restrictions, project governance, ...
● Security monitoring & response also becomes a dev team responsibility!
29. Why *Not* Software Collections?
● Containers are the recommended option for network services
● Software Collections are recommended for applications
● However:
– some platform bindings are only installed in the main System Python
– the System Python is available on all systems by default
● Some system administration tasks are best handled with the System Python
30. System Python
● Red Hat Enterprise Linux 6
– Python 2.6.6 (+ selected backports)
● Red Hat Enterprise Linux 7
– Python 2.7.5 (+ selected backports)
● Fedora 23 and later
– Recent Python 3.x (rebases rather than backports)
– Recent Python 2.7 available as system package, may not have all system bindings
31. Caveats and Challenges
● Restricted to features of System Python on oldest supported platform
● Community maintained libraries and frameworks often require newer runtimes
● Conflict between supporting:
– Red Hat Enterprise Linux 5 (Python 2.4 lacks Py3 forward compatibility features)
– Fedora 23+ (Python 3.x as System Python)
● May want to consider higher level system abstractions like Ansible
33. Python 2.7 Support Timeline
● Anticipated community end-of-life for Python 2.7 in January 2020
– https://docs.python.org/devguide/#status-of-python-branches
● Supported in Red Hat Enterprise Linux 7 until June 2024
– https://access.redhat.com/support/policy/updates/errata
● Anticipate community project support for Python 2 declining sharply post-2020
– Already seeing new community projects starting as Python 3 only
34. Python 3 Migration Techniques
● General “refactoring enablement” techniques:
– automated regression testing frameworks
– static structural analysis tools
● Recommended approach for applications and network services:
– Migrate to the Python 2.7 SCL or OpenShift image (if using the system Python)
– Follow https://docs.python.org/3/howto/pyporting.html
– Migrate to the latest Python 3.x SCL or OpenShift image
● Recommended approach for system administration tools:
– Consider using Fedora 23+ to look for potential pain points
35. Python 3 Migration Notes
● Common subset of Python 2.6+ and 3.3+ is quite large
● Many deprecated idioms can be updated automatically
● Key data & workload driven pain points
– Explicit bytes/unicode separation
– Removal of implicit cross-type comparisons
● Automated refactoring and compatibility testing tools continue to improve
37. New Security Features in Python 2.7
● https://docs.python.org/2/whatsnew/2.7.html#pep-466-network-security-enhancements-
for-python-2-7
● Constant-time comparison (hmac.compare_digest())
● Password storage hashing (hashlib.pbkdf2_hmac())
● ssl module rebase on Python 3.4 implementation
– Server Name Indication support
– SSLContext for SSL configuration
– Configuration support for TLS 1.x
– Access to system certificate stores
– ...
38. Availability in Red Hat Products
● Red Hat Enterprise Linux 7.2
– backported to System Python
● Red Hat Software Collections 2.0+
– default in Python 3.4
● Red Hat Software Collections 2.2+
– backported to Python 2.7
– default in Python 3.5
39. Third Party Module Compatibility
● ssl module rebase changed several private implementation details
● Some third party libraries had used internal APIs instead of requesting public ones
● Backport offers greater compatibility than upstream rebase
● Testing before upgrading is still recommended
● Report problems through the usual channels
41. What does “HTTPS” Mean?
● Historical Python standard library answer:
– “HTTP connection with SSL/TLS enabled”
– didn’t check certificate validity or remote host identification
● Modern Python standard library answer:
– “What web browsers say it means”
– still a HTTP connection with SSL/TLS enabled
– also checks for certificate validity
– also checks remote host identification against system certificate store
42. Verifying HTTPS Certificates
● https://docs.python.org/2/whatsnew/2.7.html#pep-476-enabling-certificate-verification-by-
default-for-stdlib-http-clients
● Default behavior of standard library HTTPS clients in:
– Python 2.7.9+
– Python 3.4.3+
– Python 3.5.0+
● Turns a silent security failure into a noisy connection failure
● Potential problems:
– Self-signed internal certificates
– Expired certificates
– Internal CAs not configured on client system
43. On Red Hat Platforms
● File-based opt-in:
– Config setting in /etc/python/cert-verification.cfg
– Red Hat Enterprise Linux 7.2+ System Python
– Red Hat Software Collections 2.2+ Python 2.7 collection
● Default behavior:
– Red Hat Software Collections Python 3.5 collection
● Details in Knowledge Base
– https://access.redhat.com/articles/2039753
44. Future Configuration Options
● https://docs.python.org/2/whatsnew/2.7.html#pep-493-https-verification-migration-tools
-for-python-2-7
● Adds new Python 2.7 specific configuration options
– ssl._https_verify_certificates() API
– PYTHONHTTPSVERIFY environment variable
● Can be used to revert Python 2.7.12+ to Python 2.7.8 behavior
● Python 2.7 only, not supported by any version of Python 3
47. Python
● Red Hat Software Collections
– http://developers.redhat.com/products/softwarecollections/
– Access: https://access.redhat.com/solutions/472793
● Software Collections Upstream
– https://wiki.centos.org/SpecialInterestGroup/SCLo
– https://www.softwarecollections.org
48. Related DevNation 2016 Sessions
● OpenShift
– OpenShift Enterprise 3 walk-through with Docker and Kubernetes
● Container Development Kit
– CDK 2.0: Docker, Kubernetes, and OSE on your desk
– Container development for command line developers
● Software Collections
– Software Collections: Easy access to the cutting edge
