Lightning talk on kamaelia at Europython 2010 in Birmingham leeds

1. Embracing
Concurrency
for Fun, Utility & Simpler Code
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

2. Why? Hardware finally going
massively concurrent
Opportunity!
...
.... PS3, high end servers, trickling down to desktops, laptops)
“many hands make light
work” but Viewed as Hard
... do we just have crap tools?
“And one language to in
Problems
the darkness bind them”
... can just we REALLY abandon 50 years of code for Erlang, Haskell
and occam?
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

3. Missing Something?
Fundamental Control Structures
... in imperative languages number greater than 3! (despite what you get taught...!)
Control Structure Traditional Abstraction Biggest Pain Points
Sequence Function Global Var
Selection Function Global Var
Iteration Function Global Var
Parallel Thread Shared Data
Usually Skipped Lost or duplicate update
This slide raises the point that
are most common bugs
teaching of imperative languages
often misses out concurrency as a
Michael Sparks
fundamental construct, leaving a
BBC R&D, http://www.kamaelia.org/Home.html
conceptual gap for most developers

4. Regarding using
concurrency, what
sort of applications
are we talking about
here?
Desktop gsoc
Media Novice
APPS trainee
Network 3rd Party
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

5. Think backend Speak 'n Write
P2P Whiteboard Programming
needed for Simple
ER DB Modeller (logo) P2P Radio
youtube/flickr Kids Games
Torrent
type systems 3D Systems
Compose Realtime Music
UGC Desktop gsoc Paint App
Backend P2P Web Server
Secure “phone”
Transcoder Social Net Vis
Media Novice ...
Shot Change
Detection APPS trainee
MiniAxon
ScriptReader
Mobile DVB MediaPreview
Reframing on Mobile
Reliable
Macro
Podcasts
Network 3rd Party Multicast
Sedna
“record
everything” XMPP XMLDB
Email & AWS pubsub
Spam Web (Amazon)
SMTP IRC Qt
Serving Gtk
Greylisting
Pop3Proxy ClientSide AIM microblogging
Spam Tools
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

6. Core Approach:
Concurrent things with comms points
Generally send messages
Keep data private, don't share
outbox inbox outbox
inbox
signal control signal
control
... ... ...
...
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

7. But I must share data?
Use Software Transactional Memory
ie version control for variables.
1. Check out the collection
of values you wish to
work on
2. Change them
3. Check them back in
4. If conflict/clash, go
back to 1
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

8. Perspectives in APIs! (1/2)
If you have concurrency it becomes
natural to think in terms of 1 st 2 nd and 3 rd
person. This affects an API's structure,
and can be vital for unde rstanding it!
1st Person - I change my state
This is one we've found that makes sense
2nd Person – YOU 3rd Person –
want to me to do Bla should
something do something
(you send outbox (I send a message)
inbox
me a message)
control signal
... ...
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

9. Perspectives in APIs! (2/2)
If you have concurre ncy it become s
natural to think in terms of 1 st 2 nd and 3 rd
person. This affects an API's structure ,
and can be vital for understanding it!
private real methods
This is one we've found that makes sense
Messages Messages sent
from public to public outboxes
inboxes
inbox outbox
control signal
Also, think ... ... Also, think
about stdout
about stdin
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

10. Actor Systems
Distinction can be unclear,
potential source of ambiguity* private real methods
Messages
No outbox concept
from public
Possible issues with
inboxes
rate limiting*
inbox
Hardcodes recipient
control
in the sender
...
*system dependent issue
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

11. Advantages of outboxes
No hardcoding of recipient
allows:
- Late Binding
- Dynamic rewiring
inbox outbox
Concurrency Patterns as
control signal Reusable Code
... ... ... a concurrency DSL
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

12. A Core Concurrency DSL
Pipeline(A,B,C)
Graphline(A=A,B=B, C=C, linkages = {})
Tpipe(cond, C)
Seq(A,B,C), PAR(), ALT()
Backplane(“name”), PublishTo(“name”), SubscribeTo(“name”)
Carousel(...)
PureTransformer(...)
StatefulTransformer(...)
PureServer(...)
MessageDemuxer(...)
Source(*messages)
NullSink
Some of these are work in progress –
they've been identified as useful, but
not implemented as chassis, yet
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

13. Pipeline Example
Pipeline(
MyGamesEventsComponent(up="p", down="l", left="a", right="s"),
BasicSprite("cat.png", name = "cat", border=40),
).activate() MyGames
Events
Component
Basic
Sprite
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

14. Graphline Example
Graphline(
NEXT = Button(...),
PREVIOUS = Button(...), PREVIOUS NEXT
FIRST = Button(...), (button) (button)
LAST = Button(...),
CHOOSER = Chooser(...),
IMAGE = Image(...), FIRST LAST
(button) (button)
...
).run()
Chooser
Image
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

15. Server Example
data
from
user data
Main Socket handler
Server Core to
user
Created at
runtime to
Protocol Handler Factory
handle the Remote
Protocol handler connection User
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

16. Server Example
You
therefore
Main need to
Server Core
provide
this bit.
Protocol Handler Factory
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

17. Server Example
from Kamaelia.Chassis.ConnectedServer import ServerCore
from Kamaelia.Util.PureTransformer import PureTransformer
def greeter(*argv, **argd):
return PureTransformer(lambda x: "hello" +x)
class GreeterServer(ServerCore):
protocol=greeter
port=1601
GreeterServer().run()
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

18. Backplane Example
# Streaming Server for raw DVB of Radio 1
Backplane(“Radio”).activate()
Pipeline(
DVB_Multiplex(850.16, [6210], feparams), # RADIO ONE
PublishTo("RADIO"),
).activate()
def radio(*argv,**argd):
return SubscribeTo(“RADIO”)
ServerCore(protocol=radio, port=1600).run()
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html

19. Thank you for listening!
If you have questions, grab me later :-)
Kamaelia is open source
Kamaelia Tutorial (from Europython 09) :
- http://tinyurl.com/kamaelia
Michael Sparks
BBC R&D, http://www.kamaelia.org/Home.html