Python 开源异步并发框架的未来 fantix @ OSTC 2014

1. Python 开源异步并发
框架的未来
About Those
Python Asynchronous Concurrency Frameworks

2. def about(self):
http://about.me/fantix

3. Python 3.4
胶水语言，内含电池

4. 并行 vs. 并发
__name__ ==
„__main__‟

7. 多个窗口
并行处理

8. 照相1
照相2
照相3
取号2
取号1
办理1 办理2 办理3 办理4

9. 照相1
照相2
照相3
取号2
取号1
办理1 办理2 办理3 办理4
17
23
16
5 7 3 6
12
15

10. 最多
同时容纳

11. 并行

12. 并发
服务器
?

13. 多进程／多线程
• “多进程” not in self.topic
• Python GIL
• PyPy STM

14. {
select(2),
epoll(7),
kqueue(2),
}

15. 事件驱动
while True:
张三 = select(“10万客户端”)
try:
data = 张三.recv()
张三.send(data)
except EAGAIN:
pass

16. sock = socket.socket()
sock.setblocking(0)
sock.bind((“”, 80))
sock.listen(128)
def on_conn(fd, events):
conn, address = sock.accept()
conn.send(b‟Hello‟)
io_loop = ioloop.IOLoop.instance()
io_loop.add_handler(sock.fileno(), on_conn, io_loop.READ)
io_loop.start()

17. from twisted.internet import protocol, reactor
class Echo(protocol.Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoFactory(protocol.Factory):
def buildProtocol(self, addr):
return Echo()
reactor.listenTCP(1234, EchoFactory())
reactor.run()
另外...

18. lol, Deferred

19. generator
@defer.inlineCallbacks
def main(endpoint, username="alice", password="secret"):
endpoint = endpoints.clientFromString(reactor, strport)
factory = protocol.Factory()
factory.protocol = imap4.IMAP4Client
try:
endpoint.connect(factory)client = yield
yield client.login(username, password)
yield client.select('INBOX')
info = yield client.fetchEnvelope(imap4.MessageSet(1))
print 'First message subject:', info[1]['ENVELOPE'][1]
except:
print "IMAP4 client interaction failed"
failure.Failure().printTraceback()
task.react(main, sys.argv[1:])

20. greenlet
@defer.inlineCallbacks
def main(endpoint, username="alice", password=“secret”):
endpoint = endpoints.clientFromString(reactor, strport)
factory = protocol.Factory()
factory.protocol = imap4.IMAP4Client
try:
client = yield endpoint.connect(factory)
yield client.login(username, password)
yield client.select('INBOX')
info = yield client.fetchEnvelope(imap4.MessageSet(1))
print 'First message subject:', info[1]['ENVELOPE'][1]
except:
print "IMAP4 client interaction failed"
failure.Failure().printTraceback()
task.react(main, sys.argv[1:])
def main(endpoint, username="alice", password=“secret”):
endpoint = endpoints.clientFromString(reactor, strport)
factory = protocol.Factory()
factory.protocol = imap4.IMAP4Client
try:
client = endpoint.connect(factory)
client.login(username, password)
client.select('INBOX')
info = client.fetchEnvelope(imap4.MessageSet(1))
print 'First message subject:', info[1]['ENVELOPE'][1]
except:
print "IMAP4 client interaction failed"
failure.Failure().printTraceback()
task.react(main, sys.argv[1:])

21. Eventlet / Gevent
from gevent import monkey
monkey.patch_all()
def handle(socket, address):
url = socket.recv()
data = urllib.urlopen(url).read()
socket.send(data)
server = StreamServer(('127.0.0.1', 1234), handle)
server.serve_forever()

22. https://glyph.twistedmatrix.com/2014/02/unyielding.html

23. __feature__
callback generator greenlet Python 3
Twisted ✅ ✅ 57%
Tornado ✅ ✅ ✅
Gevent ✅ ✅ 80%

24. activity in day * 30
issues mails commits
Twisted 122 40 19 (258)
Tornado 31 36 17
Gevent 12 5 26
2014.03.18

25. 互操作性－今天
Twisted
Tornado
Eventlet
/
Gevent
Stackless

26. 互操作性－今天
redis postgresql mysql
Twisted txredisapi txpostgres txmysql
Tornado tornado-redis momoko amysql
Gevent <monkey> psycogreen greenify

27. from __future__

28. tulip / asyncio
by Guido van Rossum

29. Python 3.4
胶水语言，内含电池

30. 像Twisted

31. callback
import asyncio
def print_and_repeat(loop):
print('Hello World')
loop.call_later(2, print_and_repeat, loop)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
print_and_repeat(loop)
loop.run_forever()

32. protocol && transport
class MyServerUdpEchoProtocol:
def connection_made(self, transport):
print('start', transport)
self.transport = transport
def datagram_received(self, data, addr):
print('Data received:', data, addr)
self.transport.sendto(data, addr)
def error_received(self, exc):
print('Error received:', exc)
def connection_lost(self, exc):
print('stop', exc)

33. Future
In [1]: from asyncio.futures import Future
In [2]: import asyncio
In [3]: loop = asyncio.get_event_loop()
In [4]: f = Future()
In [5]: loop.call_later(2, f.set_result, 'OK')
Out[5]: TimerHandle(74709.18277206, <bound method
Future.set_result of Future<PENDING>>, ('OK',))
In [6]: loop.run_until_complete(f)
Out[6]: 'OK'

34. Task && Coroutine
import asyncio
from asyncio.tasks import Task
@asyncio.coroutine
def greet_every_two_seconds():
while True:
print('Hello World')
yield from asyncio.sleep(2)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
t = Task(greet_every_two_seconds())
loop.run_until_complete(t)

35. PEP 3156
asyncio
Event Loop
RI
apps libs
Third-party
Event Loop
adapter

36. 原厂发动机
asyncio
Event Loop
RI
Twisted gevent
adapter adapter

37. “组装”发动机
asyncio
Twisted
Event Loop
Twisted
gevent
adapter

38. 互操作性－明天
Twisted
Tornado
Eventlet
/
Gevent
Stackless
asyncio
txredisapi
txmysql
momokoamysql psycogreen

39. 互操作性－明天
框架 适配 asyncio
Twisted 讨论中
Tornado 实验中
Gevent https://github.com/decentfox/gevent3

40. 谢谢！
http://about.me/fantix
http://github.com/fantix
http://weibo.com/fantix
http://twitter.com/fantix
http://www.linkedin.com/in/fantix