Pyramid patterns for building medium and large applications.

1. Patterns for building large
Pyramid applications
Carlos de la Guardia

2. Pyramid is a very flexible framework
”A la carte”, use-only-what-you-need features. You can
start small and grow your application as needed.
No persistence back end enforced.
Ability to use multiple templating languages.
Many pluggable points and extensibility mechanisms.
We love that and are proud about this.
But for new users, sometimes flexibility means too
many options.

3. Please tell me what to do!

4. Decisions, decisions, decisions
Pyramid tries to help you decide in some ways:
Scaffolds make some decisions for you.
Everything is documented so you can make informed
choices.
Some third party packages make yet more decisions.
The scope of your project can also be a factor that
influences how you layout your project and what tools
you use.
Knowing how others do things can also help decide.

5. Some suggestions
Project layout.
Tips for choosing a persistence mechanism.
Assembling your toolkit.
What is traversal and do I need it?
View configuration.
Deployment.
Extensible applications.

6. KARL
Some of these suggestions come from my experience
working as a member of KARL's development team.
KARL is an open source web system for collaboration,
organizational intranets, and knowledge management.
Used by many international organizations, such as
OXFAM GB, OSF, and more.
5000+ users (no anonymous access).
More than 75,000 Pages of content.
Nearly 100,000 lines of code
http://karlproject.org

7. Project layout
Pyramid does not enforce any layout.
Small or 'one-of' applications can simply use a single
file or a few files on the same directory.
For small to medium size applications, all Pyramid
scaffolds suggest the same layout.

8. Scaffold layout
MyProject/
|-- CHANGES.txt
|-- development.ini
|-- MANIFEST.in
|-- myproject
| |-- __init__.py
| |-- static
| | |-- favicon.ico
| | |-- logo.png
| | `-- pylons.css
| |-- templates
| | `-- mytemplate.pt
| |-- tests.py
| `-- views.py
|-- production.ini
|-- README.txt
|-- setup.cfg
`-- setup.py

9. Larger project layout
MyLargerProject/
|-- CHANGES.txt, etc.
|-- mylargerproject
| |-- __init__.py
| |-- static
| | `-- project.css
| |-- templates
| | `-- mytemplate.pt
| |-- models
| | |-- somemodel.py
| | `-- othermodel.py
| |-- someotherpackage
| | `-- somestuff.py
| |-- views
| | |-- someview.py
| | `-- otherview.py
| |-- tests
| | `-- test_something.py
`-- setup.py, etc.

10. Do I have to use python packaging?
Love it or hate it, it's what we have.
You probably want your distributions on PyPi.
Plus, packaging tools are being improved right now by
people at this conference.

11. Assembling your toolkit
Persistence back end:
SQLAlchemy.
Other ORMs.
Direct db driver connection.
MongoDB (pyramid_mongodb).
ZODB (Pyramid_zodbconn).
Multiple transactional backends? Use pyramid_tm.
What tools are you familiar with?
The most important question is how does your data look
like?

12. Assembling your toolkit
Templating language:
Chameleon.
Mako.
Jinja2 (pyramid_jinja2).
Other.
Forms:
pyramid_deform.
pyramid_formalchemy.
pyramid_formish.
pyramid_simpleform.

13. Assembling your toolkit
Authentication:
Write your own with Pyramid's out of the box facilites.
pyramid-openid.
pyramid_basicauth.
pyramid_who.
Velruse.
Development:
pyramid_debugtoolbar (a must).
buildout.

14. URLDispatch and traversal
Pyramid offers two mechanisms for finding views
associated with your data.
URLDispatch is the classic, route-based mechanism.
URLDispatch is great for table-like data.
Traversal allows you to create a tree of resources, each
of which can be addressed by one or more URLs. It's
great for tree-like data (think filesystem).
It's also a perfect fit for arbitrarily extendable sites and
sites that require very granular security.
Like with relational backends, the key to choosing is
how your data will look like.

15. View configuration
Use declarative configuration.
Lets framework configuration statements live next to the
code they configure.
Pyramid's view configuration decorators are inert. They don't
change the view's input or output.
They have to be activated from the main configuration with
scan().
Use renderers.
Easy to change implementation.
Facilitates testing.

16. View configuration
Use predicates:
In Pyramid, a single route can correspond to one or more views.
Predicates allow the view finding mechanism to find the correct
view.
route_name='home'
request_method=POST
xhr=True
Lots of others, plus custom predicates.
Easy HTTP caching:
@view_config(http_cache=3600) # 60 minutes
Pyramid will add appropriate Cache-Control and Expires headers
to responses generated when this view is invoked.

17. View configuration
Many applications have similar view data that needs to
be passed to the renderers every time:
You can generate that information on every view.
You can have a method or class that generates a dictionary
with the common values.
You can make your views methods of a single class with a
common initialization.
If you don't have alergic reactions to any mention of a
particular forbidden 'Z' word, there are other ways.

18. View configuration
Class method views example:
from pyramid.response import Response
from pyramid.view import view_config
class Aview(object):
def __init__(self, request):
self.request = request
@view_config(route_name='view_one')
def view_one(self):
return Response('one')
@view_config(route_name='view_two')
def view_two(self):
return Response('two')

19. Extensibility
Requirements depend on kind of project.
Pyramid provides some easy to use mechanisms that
allow you to compose or extend applications.
Strong, extensible configuration mechanism with conflict
detection and the ability to add directives.
Structured includes.
Asset specifications.
Tweens (Pyramid specific middleware).
ZCA-based overriding mechanisms.

20. Extensibility
Asset specification example:
my.package:static/baz.css
identifies the file named baz.css in the static subdirectory of the
my.package Python package.
Using asset overriding:
config.override_asset(
to_override='some.package:templates/mytemplate.pt',
override_with='another.package:othertemplates/anothertemplate.pt')
config.override_asset(to_override='some.package',
override_with='another.package')

21. Test, test, test
What do you call 100% coverage? A good start.
Unit tests. Make sure that nothing breaks.
Functional tests. Make sure it does what it's supposed
to do.
Tools:
Nose
Coverage
WebTest
Selenium
Use whatever you want, just test.

22. Document
If possible, use documentation to guide application
design.
Write or sketch how things should work before
implementing them.
Share with the team to get input, understanding.
Tools:
Class or method doc strings.
Sphinx.
Balsamiq.

23. Deployment with buildout
Buildout is a Python system for assembling applications
from multiple parts in a repeatable manner.
Recipes are used to define what each part of the
buildout will install and/or setup. There are many
available recipes on PyPI.
A buildout can have different configurations. For
example, deployment and production.
Buildout can be used to setup a Pyramid application
and it's dependencies easily.

24. Deployment with buildout
Sample buildout:
[buildout]
parts =
myapp
mkrelease
Develop = src/mypackage
index = http://example.github.com/myapp/staging/index/
[myapp]
recipe = zc.recipe.egg
eggs =
pyramid_debugtoolbar
nose
mypackage
interpreter = py

25. Deployment with supervisor
Supervisor is a client/server system that allows its users
to monitor and control a number of processes on
UNIX-like operating systems.
We can use supervisor to control our pyramid
applications, either alone or together with other
applications or services.
A very easy way to configure it is to simply add the
supervisor egg to our buildout and include a
configuration file

26. Deployment with supervisor
[inet_http_server]
port=127.0.0.1:9001
[supervisord]
logfile=%(here)s/var/supervisord.log
logfile_maxbytes=50MB
logfile_backups=10
loglevel=info
pidfile=%(here)s/var/supervisord.pid
[rpcinterface:supervisor]
supervisor.rpcinterface_factory =
supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=http://127.0.0.1:9001
[program:core]
command = %(here)s/bin/paster serve %(here)s/src/inav2_core/development.ini
redirect_stderr = true

27. Deployment with nginx
More information at:
http://docs.pylonsproject.org/projects/pyramid_cookbook/dev/deployment/nginx.html
upstream myapp-site {
server 127.0.0.1:5000;
server 127.0.0.1:5001;
}
server {
server_name example.com;
access_log /home/example/env/access.log;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 60s;
proxy_send_timeout 90s;
proxy_read_timeout 90s;
proxy_buffering off;
proxy_pass http://myapp-site;
proxy_redirect off;
}
}

28. Deployment with mod_wsgi
We need to create an app that will call our application with configuration.
Then we set up apache to call this app.
More information at:
http://docs.pylonsproject.org/projects/pyramid/1.2/tutorials/modwsgi/index.html
#pyramid.wsgi
from pyramid.paster import get_app
application = get_app('/mydir/modwsgi/env/myapp/production.ini', 'main')
#apache.conf
WSGIApplicationGroup %{GLOBAL}
WSGIPassAuthorization On
WSGIDaemonProcess pyramid user=cguardia group=staff threads=4
python-path=/mydir/modwsgi/env/lib/python2.6/site-packages
WSGIScriptAlias /myapp /mydir/modwsgi/env/pyramid.wsgi
<Directory /mydir/modwsgi/env>
WSGIProcessGroup pyramid
Order allow,deny
Allow from all
</Directory>

29. Thanks!
@cguardia on Twitter and IRC.
Join us at the #pyramid IRC channel for encouragement
and support.
There's also the Pyramid mailing lists at Google
Groups:
http://groups.google.com/group/pylons-devel