Beaker is probably the most widespread cross-framework solution to manage sessions and caching in the python web ecosystem.
Born in 2005 from Pylons author has been historically maintained together with the Pylons framework. Since Pylons has been deprecated in favour of Pyramid, the Pylons Project team decided to write a custom session management solution for Pyramid and let the user handle more advanced backends.
Since 2015 beaker maintenance has been passed to the TurboGears project who ported it to a fully native Python3 solution and it’s stille the de-facto standard for Sessions and Caching on Bottle.
The talk will cover the advantages and drawbacks of the architectural decisions behind the 10 years of history of Beaker, the drawbacks we discovered while porting it to Python3 and which have been the “wrong choices” and how we plan to solve them and make it shine again as the best whole-around solution for Sessions and Caching on WSGI.
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
PyConIT6 - MAKING SESSIONS AND CACHING ROOMMATES
1. BEAKER - MAKING SESSIONS AND
CACHING ROOMMATES
Alessandro Molina
@__amol__
amol@turbogears.org
2. Who am I
● CTO @ AXANT.it, mostly Python company
(with some iOS and Android)
● TurboGears2 dev team member
● Took over Beaker maintenace in 2015
● Mostly contributed to web python libraries:
Formencode, MING MongoDB ODM, ToscaWidgets2
3. What’s Beaker
● Framework for handling Caching and
Sessions in web applications
● Used by many different frameworks:
TurboGears, Bottle, Pyramid, etc...
● Created by Pylons Author to solve the
dogpile effect.
5. The Good
● Handles the DogPile effect
● Provides many backends: Memcache, File,
MongoDB, Redis, SQLAlchemy
● A single tool to handle both caching and
sessions with common backends.
6. The Bad
● File based synchronization, what about
distributed solutions?
● Session as a big BLOB of data doesn’t
perform well in case of big BLOBs of data
● Updated for too long without breaking
backward compatibility: Too many APIs
7. Getting on project: PY3 Support
● Beaker supported Python3 using 2to3
● This lead to some bugs (not everything got
properly converted)
● Made really hard to run the test suite and
maintain the project.
15. Porting to PY3: #2 Session Cookies?
● Session ID is stored in cookies
● CookieSession stores even the whole
session in a cookie
○ Great idea, makes really simple to scale
○ HTTP Headers are plain text
○ We are pickling data and base64 so already ASCII
○ Just need to encrypt it to avoid people from
messing with them.
16. HTTP Cookies
HTTP has allowed field content with text
in the ISO-8859-1 charset [ISO-8859-1],
supporting other charsets only through
use of [RFC2047] encoding. In practice,
most HTTP header field values use only a
subset of the US-ASCII charset [USASCII].
Newly defined header fields SHOULD limit
their field values to US-ASCII octets
17. Encoding/Encripting data on Py3
● Beaker expected to be able to perform
text operations on the result of base64
● Relied on AES to encrypt cookies, pbkdf2
to generates keys and base64/binascii
to make it text.
● Reads the encrypt_key from config file
18. 4 lines of code == 10 doubts
● Even Python has no clear idea of what is
text and what are bytes:
○ Both BASE64 and BINASCII work with ASCII
○ BINASCII accepts unicode (text)
○ BASE64 accepts bytes
○ Between Python3.2 and Python3.3 the binascii
changed behaviour of accepting unicode vs bytes
○ ConfigParser returns text on py3, bytes on py2
20. Let’s go for something simpler...
● Review patch to caching decorator
● To cache a function, just apply a decorator
that calls function and caches the result
● Caching decorator generates cache key
○ Cache Key from the function name
○ convert parameters to strings
○ add parameters to cache key
21. Getting the Cache Key
def cache(self, *args, **kwargs):
"""Decorate a function to cache itself with supplied parameters
:param args: Used to make the key unique for this function, as in region()
above.
:param kwargs: Parameters to be passed to get_cache(), will override defaults
Example::
# Assuming a cache object is available like:
cache = CacheManager(dict_of_config_options)
def populate_things():
@cache.cache('mycache', expire=15)
def load(search_term, limit, offset):
return load_the_data(search_term, limit, offset)
return load('rabbits', 20, 0)
"""
return _cache_decorate(args, self, kwargs, None)
23. Unknown parameters
● Decorator gets *args and **kwargs
● Makes sense... doesn’t know the real
function arguments
● Not so much in fact… func(1) and func
(a=1) end up with two different cache
keys
24. Avoid the doubt
● Beaker solved this by accepting only
positional arguments on cached functions
.. note::
The function being decorated must only be called with
positional arguments.
● Users not so happy, breaks people code
when they introduce caching:
○ https://github.com/bbangert/beaker/issues/62
25. What to do?
● Leave it as is… As been like that for years
● Add **kwargs and just document the issue
● Add **kwargs and try to be smart using
inspect.getargspec / inspect.
getcallargs but that would be slow
27. Has been a great trip!
● Making sessions and caching in a single
solution seemed simple, but has actually a
lot of corner case
● Beaker Co-Author thrown in the towel and
created dopgile.cache which is based on
beaker code but is much simpler and
doesn’t provide sessions
28. Still is incredibily convenient
● There are more shared features than
differences (especially in cookie based
marshalled sessions).
● It’s really convenient, write backends once
and have support for both sessions and
caching on them.