Elegant Solutions For Everyday Python Problems - PyCon Canada 2017

Elegant Solutions For Everyday
Python Problems
Nina Zakharenko
@nnja
bit.ly/elegant-python-ca
ℹ There are links in these slides. Follow along ^

This talk is for you if:
— You're an intermediate python programmer
— You're coming to python from another language
— You want to learn about fancy features like: magic
methods, iterators, decorators, and context managers
slides: bit.ly/elegant-python-ca
@nnja

How do we make code elegant?
We pick the right tool for the job!
Resources for converting from Python 2 -> 3

Beauty is
in the eye of
the beholder

You're used to implementing __str__ and __repr__ --but
there's a whole other world of powerful magic methods!
By implementing a few straightforward methods,
you can make your objects behave like built-ins such as:
— numbers
— lists
— dictionaries
— and more...
@nnja

class Money:
currency_rates = {
'$': 1,
'€': 0.88,
}
def __init__(self, symbol, amount):
self.symbol = symbol
self.amount = amount
def __repr__(self):
return '%s%.2f' % (self.symbol, self.amount)
@nnja

class Money:
# defined currency_rates, __init__, and repr above...
def convert(self, other):
"""Convert other amount to our currency"""
new_amount = (
other.amount / self.currency_rates[other.symbol]
* self.currency_rates[self.symbol])
return Money(self.symbol, new_amount)
@nnja

__repr__ in action
>>> soda_cost = Money('$', 5.25)
>>> soda_cost
$5.25
>>> pizza_cost = Money('€', 7.99)
>>> pizza_cost
€7.99
@nnja

class Money:
def __add__(self, other):
""" Add 2 Money instances using '+' """
new_amount = self.amount + self.convert(other).amount
return Money(self.symbol, new_amount)
@nnja

>>> soda_cost + pizza_cost
$14.33
More on Magic Methods: Dive into Python3 - Special Method Names

>>> soda_cost + pizza_cost
$14.33
>>> pizza_cost + soda_cost
€12.61
More on Magic Methods: Dive into Python3 - Special Method Names
@nnja

some magic methods map to built-in functions
class Alphabet:
letters = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
def __len__(self):
return len(self.letters)
>>> my_alphabet = Alphabet()
>>> len(my_alphabet)
26
@nnja

Making classes iterable
— In order to be iterable, a class needs to implement
__iter__()
— __iter__() must return an iterator
— In order to be an iterator a class needs to implement
__next__() which must raise StopIteration when there
are no more items to return
or next() in python2
^ can be confusing at ﬁrst, but remember these guidelines for making classesGreat explanation of iterable vs. iterator vs. generator

class IterableServer:
services = [
{'active': False, 'protocol': 'ftp', 'port': 21},
{'active': True, 'protocol': 'ssh', 'port': 22},
{'active': True, 'protocol': 'http', 'port': 21},
]
def __init__(self):
self.current_pos = 0
def __iter__(self): # can return self, because __next__ implemented
return self
def __next__(self):
while self.current_pos < len(self.services):
service = self.services[self.current_pos]
self.current_pos += 1
if service['active']:
return service['protocol'], service['port']
raise StopIteration
next = __next__ # optional python2 compatibility
@nnja

>>> for protocol, port in IterableServer():
print('service %s is running on port %d' % (protocol, port))
service ssh is running on port 22
service http is running on port 21
... not bad
@nnja

tip: use a generator
when your iterator doesn't need to
maintain a lot of state
@nnja

class Server:
services = [
{'active': False, 'protocol': 'ftp', 'port': 21},
{'active': True, 'protocol': 'ssh', 'port': 22},
{'active': True, 'protocol': 'http', 'port': 21},
]
def __iter__(self):
for service in self.services:
if service['active']:
yield service['protocol'], service['port']
@nnja

Why does this work?
use single parenthesis ( ) to create a generator
comprehension
^ technically, a generator expression but I like this term better, and so does Ned Batchelder
>>> my_gen = (num for num in range(1))
>>> my_gen
<generator object <genexpr> at 0x107581bf8>
@nnja

An iterator must implement __next__()
>>> next(my_gen) # remember __len__() mapped to built-in len()
0
and raise StopIteration when
there are no more elements
>>> next(my_gen)
... StopIteration Traceback (most recent call last)
For more tools for working with iterators, check out itertools

alias methods
class Word:
def __init__(self, word):
self.word = word
def __repr__(self):
return self.word
def __add__(self, other_word):
return Word('%s %s' % (self.word, other_word))
# Add an alias from method __add__ to the method concat
concat = __add__
@nnja

When we add an alias from __add__ to concat because
methods are just objects
>>> # remember, concat = __add__
>>> first_name = Word('Max')
>>> last_name = Word('Smith')
>>> first_name + last_name
Max Smith
>>> first_name.concat(last_name)
Max Smith
>>> Word.__add__ == Word.concat
True
@nnja

Dog class
>>> class Dog:
sound = 'Bark'
def speak(self):
print(self.sound + '!', self.sound + '!')
>>> my_dog = Dog()
>>> my_dog.speak()
Bark! Bark!
read the docs

getattr(object, name, default)
>>> class Dog:
sound = 'Bark'
def speak(self):
print(self.sound + '!', self.sound + '!')
>>> my_dog = Dog()
>>> my_dog.speak()
Bark! Bark!
>>> getattr(my_dog, 'speak')
<bound method Dog.speak of <__main__.Dog object at 0x10b145f28>>
>>> speak_method = getattr(my_dog, 'speak')
>>> speak_method()
Bark! Bark!
read the docs

Example: command line tool with dynamic commands
class Operations:
def say_hi(self, name):
print('Hello,', name)
def say_bye(self, name):
print ('Goodbye,', name)
def default(self, arg):
print ('This operation is not supported.')
if __name__ == '__main__':
operations = Operations()
# let's assume error handling
command, argument = input('> ').split()
getattr(operations, command, operations.default)(argument)
read the docs

Output
$ python getattr.py
> say_hi Nina
Hello, Nina
> blah blah
This operation is not supported.
✨
additional reading - inverse of getattr() is setattr()

functool.partial(func, *args, **kwargs)
— Return a new partial object which behaves like func
called with args & kwargs
— if more args are passed in, they are appended to args
— if more keyword arguments are passed in, they extend
and override kwargs
read the docs on partials

functool.partial(func, *args, **kwargs)
>>> from functools import partial
>>> basetwo = partial(int, base=2)
>>> basetwo
functools.partial(<class 'int'>, base=2)
>>> basetwo('10010')
18
read the docs on partials

library I
!
: github.com/jpaugh/agithub
agithub is a (badly named) REST API client with
transparent syntax which facilitates rapid prototyping
— on any REST API!
— Implemented in 400 lines.
— Add support for any REST API in ~30 lines of code.
— agithub knows everything it needs to about protocol
(REST, HTTP, TCP), but assumes nothing about your
upstream API.
@nnja

deﬁne endpoint url & other connection properties
class GitHub(API):
def __init__(self, token=None, *args, **kwargs):
props = ConnectionProperties(
api_url = kwargs.pop('api_url', 'api.github.com'))
self.setClient(Client(*args, **kwargs))
self.setConnectionProperties(props)
then, start using the API!
>>> gh = GitHub('token')
>>> status, data = gh.user.repos.get(visibility='public', sort='created')
>>> # ^ Maps to GET /user/repos
>>> data
... ['tweeter', 'snipey', '...']
github.com/jpaugh/agithub

black magic!
but, how?...
@nnja

class API:
def __getattr__(self, key):
return IncompleteRequest(self.client).__getattr__(key)
__getitem__ = __getattr__
class IncompleteRequest:
def __getattr__(self, key):
if key in self.client.http_methods:
htmlMethod = getattr(self.client, key)
return partial(htmlMethod, url=self.url)
else:
self.url += '/' + str(key)
return self
__getitem__ = __getattr__
class Client:
http_methods = ('get') # ...
def get(self, url, headers={}, **params):
return self.request('GET', url, None, headers)
github.com/jpaugh/agithub source: base.py

given a non-existant path:
>>> status, data = this.path.doesnt.exist.get()
>>> status
... 404
& because __getitem__ is aliased to __getattr__:
>>> owner, repo = 'nnja', 'tweeter'
>>> status, data = gh.repos[owner][repo].pulls.get()
>>> # ^ Maps to GET /repos/nnja/tweeter/pulls
>>> data
.... # {....}
github.com/jpaugh/agithub

Context
Managers
& new in python 3: async context managers

When should I use one?
Need to perform an action before and/or after an
operation.
Common scenarios:
— Closing a resource after you're done with it (ﬁle,
network connection)
— Perform cleanup before/after a function call
@nnja

Example Problem: Feature Flags
Turn features of your application on and off easily.
Uses of feature ﬂags:
— A/B Testing
— Rolling Releases
— Show Beta version to users opted-in to Beta Testing
Program
More on Feature Flags

Example - FeatureFlags Class
class FeatureFlags:
""" Example class which stores Feature Flags and their state. """
SHOW_BETA = 'Show Beta version of Home Page'
flags = {
SHOW_BETA: True
}
@classmethod
def is_on(cls, name):
return cls.flags[name]
@classmethod
def toggle(cls, name, on):
cls.flags[name] = on
feature_flags = FeatureFlags()
@nnja

How do we temporarily turn features on and off when
testing ﬂags?
Want:
with feature_flag(FeatureFlags.SHOW_BETA):
assert '/beta' == get_homepage_url()
@nnja

Using Magic Methods __enter__ and __exit__
class feature_flag:
""" Implementing a Context Manager using Magic Methods """
def __init__(self, name, on=True):
self.name = name
self.on = on
self.old_value = feature_flags.is_on(name)
def __enter__(self):
feature_flags.toggle(self.name, self.on)
def __exit__(self, *args):
feature_flags.toggle(self.name, self.old_value)
See: contextlib.contextmanager

The be!er way: using the contextmanager decorator
from contextlib import contextmanager
@contextmanager
def feature_flag(name, on=True):
old_value = feature_flags.is_on(name)
feature_flags.toggle(name, on)
yield
feature_flags.toggle(name, old_value)

The be!er way: using the contextmanager decorator
@contextmanager
""" The easier way to create Context Managers """
feature_flags.toggle(name, on) # behavior of __enter__()
yield
feature_flags.toggle(name, old_value) # behavior of __exit__()

Note: yield?
@contextmanager
""" The easier way to create Context Managers """
feature_flags.toggle(name, on) # behavior of __enter__()
yield
feature_flags.toggle(name, old_value) # behavior of __exit__()

either implementation
def get_homepage_url():
""" Method that returns the path of the home page we want to display. """
if feature_flags.is_on(FeatureFlags.SHOW_BETA):
return '/beta'
else:
return '/homepage'
def test_homepage_url_with_context_manager():
# saw the beta homepage...
assert get_homepage_url() == '/beta'
with feature_flag(FeatureFlags.SHOW_BETA, on=False):
# saw the standard homepage...
assert get_homepage_url() == '/homepage'
@nnja

either implementation
""" Method that returns the path of the home page we want to display. """
if feature_flags.is_on(FeatureFlags.SHOW_BETA):
return '/beta'
else:
return '/homepage'
def test_homepage_url_with_context_manager():
print('seeing the beta homepage...')
with feature_flag(FeatureFlags.SHOW_BETA, on=False):
assert get_homepage_url() == '/homepage'
print('seeing the standard homepage...')
@nnja

Decorators
The simple explanation:
Syntactic sugar that allows modiﬁcation of an underlying
function.
@nnja

Recap!
Decorators:
— Wrap a function in another function.
— Do something:
— before the call
— after the call
— with provided arguments
— modify the return value or arguments
@nnja

def say_after(hello_function):
def say_nice_to_meet_you(name):
hello_function(name)
print('It was nice to meet you!')
return say_nice_to_meet_you
def hello(name):
print('Hello', name)
>>> hello('Nina')
Hello Nina
>>> say_after(hello)('Nina')
Hello Nina It was nice to meet you!
— say_after(hello) returns the function
say_nice_to_meet_you
— then we call say_nice_to_meet_you('Nina')
@nnja

def say_nice_to_meet_you(name):
print('It was nice to meet you!')
return say_nice_to_meet_you
@say_after
def hello(name):
>>> hello('Nina')
Hello Nina It was nice to meet you!
— calling the decorated function hello(name)
— is the same as calling an undecorated hello with
say_after(hello)('Nina')
@nnja

closure example
def multiply_by(num):
def do_multiplication(x):
return x * num
return do_multiplication
multiply_by_five = multiply_by(5)
>>> multiply_by_five(4)
20
@nnja

decorators that take arguments
def greeting(argument):
def greeting_decorator(greet_function):
def greet(name):
greet_function(name)
print('It was %s to meet you!' % argument)
return greet
return greeting_decorator
@greeting('bad')
def aloha(name):
print ('Aloha', name)
@nnja

decorators that take arguments
def say_this_after(argument):
def say_after_meeting(name):
print('It was %s to meet you' % argument)
return say_after_meeting
return say_after
@say_this_after('bad')
def hello(name):
Is the same as calling this on an undecorated function:
say_after_bad = say_this_after('bad')(hello)
say_after_bad('Nina')
@nnja

losing context with a decorator
def say_bye(func):
def wrapper(name):
func()
print('Bye', name)
return wrapper
@say_bye
def my_name():
""" Say my name"""
print('Nina')
>>> my_name.__name__
'wrapper'
>>>> my_name.__doc__
# ... empty
@nnja

solution: use wraps, or wrapt library!
from contextlib import wraps
def say_adios(func):
@wraps(func) # pass in which function to wrap
def wrapper():
func()
print('Adios!')
return wrapper
@say_adios
def say_max():
""" Says the name Max"""
print('Max')
>>> say_max.__name__
'say_max'
>>> say_max.__doc__
' Says the name Max'
@nnja

Decorators: Common uses
— logging
— timing
— validation
— rate limiting
— mocking/patching
@nnja

ContextDecorators
ContextManagers
+ Decorators combined.
@nnja

As of python 3.2 ContextDecorators are in the standard
library. They're the best of both worlds!
— By using ContextDecorator you can easily write classes
that can be used both as decorators with @ and
context managers with the with statement.
— ContextDecorator is used by contextmanager(), so you
get this functionality
✨
automatically .
— Alternatively, you can write a class that extends from ContextDecorator or uses
ContextDecorator as a mixin, and implements __enter__, __exit__ and __call__
— If you use python2, a backport package is available here: contextlib2
@nnja

Remember @contextmanager from earlier?
@contextmanager
feature_flags.toggle(name, on)
yield
feature_flags.toggle(name, old_value)
@nnja

use it as a context manager
beta_flag_on = feature_flags.is_on(FeatureFlags.SHOW_BETA)
return '/beta' if beta_flag_on else '/homepage'
or use as a decorator
@feature_flag(FeatureFlags.SHOW_BETA, on=False)
def get_profile_page():
beta_flag_on = feature_flags.is_on(FeatureFlags.SHOW_BETA)
return 'beta.html' if beta_flag_on else 'profile.html'
assert get_profile_page() == 'profile.html'
@nnja

library I
!
: freezegun lets your python tests ❇ travel
through time! ❇
from freezegun import freeze_time
# use it as a Context Manager
def test():
with freeze_time("2012-01-14"):
assert datetime.datetime.now() == datetime.datetime(2012, 1, 14)
assert datetime.datetime.now() != datetime.datetime(2012, 1, 14)
# or a decorator
@freeze_time("2012-01-14")
def test():
assert datetime.datetime.now() == datetime.datetime(2012, 1, 14)
read the source sometime, it's mind-bending!
@nnja

NamedTuple
Useful when you need lightweight representations of
data.
Create tuple subclasses with named ﬁelds.
@nnja

Example
from collections import namedtuple
CacheInfo = namedtuple(
"CacheInfo", ["hits", "misses", "max_size", "curr_size"])
@nnja

Giving NamedTuples default values
RoutingRule = namedtuple(
'RoutingRule',
['prefix', 'queue_name', 'wait_time']
)
(1) By specifying defaults
RoutingRule.__new__.__defaults__ = (None, None, 20)
(2) or with _replace to customize a prototype instance
default_rule = RoutingRule(None, None, 20)
user_rule = default_rule._replace(prefix='user', queue_name='user-queue')
@nnja

NamedTuples can be subclassed and extended
class Person(namedtuple('Person', ['first_name', 'last_name'])):
""" Stores first and last name of a Person"""
__slots__ = ()
def __str__(self):
return '%s %s' % (self.first_name, self.last_name)
>>> me = Person('nina', 'zakharenko')
>>> str(me)
'nina zakharenko'
>>> me
Person(first_name='nina', last_name='zakharenko')
@nnja

Tip
Use __slots__ = () in your NamedTuples!
— It prevents the creation of instance dictionaries.
— It lowers memory consumption.
— Allows for faster access
@nnja

"Perfection is achieved, not when
there is nothing more to add, but
when there is nothing left to take
away."
— Antoine de Saint-Exupery
@nnja

New Tools
— Magic Methods
— make your objects behave like builtins (numbers,
list, dict, etc)
— Method ❇Magic❇
— alias methods
— * getattr
— functool.partial
@nnja

— ContextManagers
— Close resources
— Decorators
— do something before/after call, modify return value
or validate arguments
— ContextDecorators
— ContextManagers + Decorators combined!
@nnja

— Iterators & Generators
— Loop over your objects
— yield
— NamedTuple
— Lightweight classes
@nnja

Don't be
a mindless
Code
Monkey
@nnja

Use these
tools to be
an elegant
Pythonista!
@nnja

Thanks!
@nnja
nina@nnja.io
bit.ly/elegant-python-ca
@nnja

Elegant Solutions For Everyday Python Problems - PyCon Canada 2017

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Elegant Solutions For Everyday Python Problems - PyCon Canada 2017

Ähnlich wie Elegant Solutions For Everyday Python Problems - PyCon Canada 2017 (20)

Mehr von Nina Zakharenko

Mehr von Nina Zakharenko (7)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Elegant Solutions For Everyday Python Problems - PyCon Canada 2017