pyCologne 15 May

1. asyncio/aiohttp
Anton Kasyanov
Some slides adopted from A. Svetlov

2. Who?
• Python enthusiast
• RWTH Aachen Master student
• Contributed into aiohttp, aiozmq, aioes, aiohttp-session
• Using asyncio in production for >1 year

3. Asynchronous programming
Sync.
Images from http://krondo.com/

4. Asynchronous programming
Sync. Threaded
Images from http://krondo.com/

5. Asynchronous programming
Sync. Threaded Real world
Images from http://krondo.com/

6. Asynchronous programming
Async
Images from http://krondo.com/

7. Asynchronous programming
Async
Images from http://krondo.com/
I/O only!

8. Asynchronous programming
• Non-blocking IO
• Superior (comparing to threads) speed of web servers

9. Asynchronous programming
• Non-blocking IO
• Superior (comparing to threads) speed of web servers
• But: not elegant code, callbacks
fs.readdir(source, function(err, files) {
if (err) {
console.log('Error finding files: ' + err)
} else {
files.forEach(function(filename, fileIndex) {
console.log(filename)
gm(source + filename).size(function(err, values) {
if (err) {
console.log('Error identifying file size: ' + err)
} else {
console.log(filename + ' : ' + values)
aspect = (values.width / values.height)
widths.forEach(function(width, widthIndex) {
height = Math.round(width / aspect)

10. Two years ago
• twisted
• tornado
• gevent
• etc.

11. One year ago
• asyncio came out
• no http
• no database drivers

12. Hello world
import asyncio
import time
@asyncio.coroutine
def sleepy():
print("before sleep", time.time())
yield from asyncio.sleep(5)
print("after sleep", time.time())
asyncio.get_event_loop().run_until_complete(sleepy())

13. Today
• aiohttp
• aiopg
• aiomysql
• aioredis
• aiozmq
• aioes
• aiomemcache
• aiohttp_jinja2
• aiohttp_session
• aiocouchdb
• aiogreen
• aiokafka

14. aiohttp: client API

15. aiohttp: client
import asyncio
import aiohttp
@asyncio.coroutine
def fetch_page(url):
response = yield from aiohttp.request('GET', url)
body = yield from response.read()
return body
content = asyncio.get_event_loop().run_until_complete(
fetch_page('http://python.org'))
print(content)

16. aiohttp: client
>>> session = aiohttp.ClientSession()
>>> yield from session.get(
... 'http://httpbin.org/cookies/set/my_cookie/my_value')
>>> r = yield from session.get('http://httpbin.org/cookies')
>>> json = yield from r.json()
>>> json['cookies']['my_cookie']
'my_value'

17. aiohttp: server API

18. aiohttp: simple server
import asyncio
from aiohttp import web
@asyncio.coroutine
def handle(request):
text = “Hello, world!”
return web.Response(body=text.encode(‘utf-8'))
app = web.Application()
app.router.add_route('GET', '/', handle)

19. aiohttp: simple server
loop = asyncio.get_event_loop()
f = loop.create_server(app.make_handler(),
'127.0.0.1', 8080)
srv = loop.run_until_completed(f)
print(‘Server started on’,srv.sockets[0].getsockname())
try:
loop.run_forever()
except KeyboardInterrupt:
pass

20. aiohttp: upload files
<form action="/store_mp3" method="post"
accept-charset="utf-8"
enctype="multipart/form-data">
<label for="mp3">Mp3</label>
<input id="mp3" name="mp3"
type="file" value="" />
<input type="submit" value="submit" />
</form>

21. aiohttp: upload files
@asyncio.coroutine
def store_mp3_view(request):
data = yield from request.post()
filename = data['mp3'].filename
input_file = data['mp3'].file
content = input_file.read()
return web.Response(body=content,
headers=MultiDict(
{'CONTENT-DISPOSITION':
input_file})

22. aiohttp: websockets
@asyncio.coroutine
def websocket_handler(request):
ws = web.WebSocketResponse()
ws.start(request)
while True:
data = yield from ws.receive_str()
if data == 'close':
yield from ws.close()
else:
ws.send_str(data + '/answer')
return ws

23. aiohttp: more
• middleware
• aiohttp_sessions
• aiohttp_jinja2

24. benchmarks
• around the same as tornado
• twisted.web is a bit faster
• WSGI is slower
• WIP

25. What’s next?

26. aiopg
import sqlalchemy as sa
from aiopg.sa import create_engine
metadata = sa.MetaData()
tbl = sa.Table('tbl', metadata,
sa.Column('id', sa.Integer, primary_key=True),
sa.Column('val', sa.String(255)))
app = web.Application()
app[‘db’] = yield from create_engine(…)

27. aiopg
@asyncio.coroutine
def handler(request):
with (yield from request.app['db']) as conn:
yield from conn.execute(
tbl.insert().values(val=‘abc’))
res = yield from conn.execute(tbl.select())
for row in res:
print(row.id, row.val)

28. aiozmq
import asyncio
from aiozmq import rpc
class Handler(rpc.AttrHandler):
@rpc.method
def remote(self, arg1, arg2):
return arg1 + arg2

29. aiozmq
@asyncio.coroutine
def go():
server = yield from rpc.serve_rpc(
Handler(),
bind='tcp://127.0.0.1:5555')
client = yield from rpc.connect_rpc(
connect='tcp://127.0.0.1:5555')
ret = yield from client.call.remote(1, 2)
assert ret == 3

30. PEP 492
async def read_data(db):
data = await db.fetch('SELECT ...')
...
async for row in Cursor():
print(row)
async with session.transaction():
...
await session.update(data)
...

31. Links
• https://docs.python.org/3/library/asyncio.html
• http://aiohttp.readthedocs.org/
• http://aiozmq.readthedocs.org/
• http://aiopg.readthedocs.org

32. Questions?
antony.kasyanov@gmail.com
http://antonkasyanov.com