Discussion on distributed apps and the cloud resources available to support them. Some discussion on the XMPP/Jabber based messaging system we use at Koordinates. Part of the seminar series for the Wellington Summer of Code programme.

1. Distributed-ness
Robert Coup, Koordinates
http://rob.coup.net.nz/
robert.coup@koordinates.com

2. Me

3. What is it?
“Distributed computing deals with hardware and
software systems containing more than one processing
element or storage element, concurrent processes, or
multiple programs, running under a loosely or tightly
controlled regime.” - Wikipedia
http://en.wikipedia.org/wiki/Distributed_computing

5. What is it?
Application architecture
Independent components
Dynamic resourcing

6. What is it?
Distributed computing is not scaling
Distributed computing can help you scale
There are easier ways to scale short-term

7. Easier ways

8. Distributed problems
Break up my big problem into small chunks which can
be worked on in parallel or asynchronously.

9. Distributed applications
Single application with a bunch of components
Inter-dependency
Components “load-up” differently

10. catinthehat.biz
“great hats for your cats”

12. Little catinthehat.biz
Load balancer
Cache Media
App
Storage
DB
Worker

13. Bigger catinthehat.biz
Load balancer
Cache Media
App App App +
Cache Media
DB DB
Worker
Storage +
Worker
+

14. Talking

15. Talking
Components of a distributed app need to talk

16. Talking
Components of a distributed app need to talk
But should have minimal knowledge of each other

17. Talking
Components of a distributed app need to talk
But should have minimal knowledge of each other

18. Talking
Components of a distributed app need to talk
But should have minimal knowledge of each other
Just like in code modules! “Decoupling”

19. Messaging
Point to point:
Needs configuration
Web services

20. Messaging
Queues:
Publish-subscribe
Amazon SQS
Lots of others
http://aws.amazon.com/sqs/

21. Messaging
Peer-to-peer
Jabber / XMPP
Persistent
connections
Presence

22. Jabber at Koordinates
Brainz
manages the work
Korrew
does the work

23. Jabber @ Koordinates
Data imports have
20-25 inter-related
tasks
“Package” defines the
dependencies and
input data

24. Task Packages

25. Task Packages
Kerrows & Brainzs connect via XMPP

26. Task Packages
Kerrows & Brainzs connect via XMPP
Brainz publishes tasks via PubSub

27. Task Packages
Kerrows & Brainzs connect via XMPP
Brainz publishes tasks via PubSub
Kerrow negotiates for tasks, then does them

28. Task Packages
Kerrows & Brainzs connect via XMPP
Brainz publishes tasks via PubSub
Kerrow negotiates for tasks, then does them
Brainz notified on task completion/error

29. Task Packages
Kerrows & Brainzs connect via XMPP
Brainz publishes tasks via PubSub
Kerrow negotiates for tasks, then does them
Brainz notified on task completion/error
If Kerrows go offline, tasks are re-assigned

30. Bots
Via IM, we can connect to
Brainz/Kerrow
Ask for status, cancel,
new tasks, …
And it can message us:
errors, info

31. Live Status
Keep a live eye on whats
going on
Danga apps have
terminal consoles (telnet)
Otherwise you’re
debugging via logs

32. Bigger catinthehat.biz
Load balancer
Cache Media
App App App +
Cache Media
DB DB
Worker
Storage
Worker +
+

33. Storage
Dump files
Get them back
Reliably
Quickly
In bulk
Backups

34. Amazon S3
- Simple Storage Service
Unlimited storage
Cheap!
US$0.15 / GB / month
US$0.10 / GB in & US$0.17 / GB out
http://aws.amazon.com/s3/

35. Amazon S3
Not a hard disk or filesystem
Data is organised into namespaces (buckets)
hatdesigns.catinthehat.biz
Within that: key-value pairs
Access via HTTP
Authentication / access-control
Open source version - mogilefs
http://www.danga.com/mogilefs/

36. Amazon S3 - downsides
Eventual consistency
99.99% reliable = 1/10K requests fail
Will return errors

37. Amazon S3 - uses
Uses for catinthehat.biz?
Customer photos of hats on cats
Customer hat designs
Story videos
Manufacturing design files
Backups

38. Bigger catinthehat.biz
Load balancer
Cache Media
App App App +
Cache Media
DB DB
Worker
Storage
Worker +
S3
+

39. Compute power
Supply & demand
Supply costs
Demand is hard to
manage

40. Amazon EC2
- Elastic Compute Cloud
Virtual servers on demand
From US$0.10 - US$0.80 / hour
Linux & Windows, 1-8 cores, 1.7-15GB memory,
160GB-1.7TB local storage, 32/64bit
Permanent storage from US$0.10 / GB / month
http://aws.amazon.com/ec2/

41. Amazon EC2
Turn capacity on & off at will
Ideal for batch processing
Ideal for dynamic loads

42. Amazon EC2
Not cheapest - US$70+/month for static server
Instances can be terminated at any time!
Organise configuration - Puppet, RightScale, Scalr
Need an app that is architected to handle it
http://slicehost.com/
http://puppet.reductivelabs.com/
http://www.rightscale.com/
http://code.google.com/p/scalr/

43. Amazon EC2 - uses
Uses for catinthehat.biz?
converting customer designs
creating story videos
application servers

44. Bigger catinthehat.biz
Load balancer
Cache Media
App App App +
EC2
Cache Media
DB DB
Worker
Storage
Worker +
EC2 S3
+

45. Google AppEngine
Auto-scaling web applications
Google hosts and runs
Access to BigTable, Image/Email/Cache/HTTP APIs
Restricted Python environment
Free to get started
http://code.google.com/appengine/

46. Google AppEngine
Still in beta, no way of buying “extra” capacity
No offline/background processing
Time limits on requests
No file storage
Datastore isn’t SQL
Lock-in

47. Google AppEngine
Uses at catinthehat.biz?
Facebook application?
Prototypes?

48. MapReduce
Map Phase Reduce Phase
Take a problem Combine all the
answers to the chunk
Chop it up into chunks
to get the real answer
Distribute chunks to
lots of workers to do
http://en.wikipedia.org/wiki/MapReduce

49. MapReduce
Small atomic chunks of work
Run across acres of machines on masses of data
Easy to write (although problems need to “fit”)
Can be chained together
Open source versions - Hadoop, others
http://en.wikipedia.org/wiki/MapReduce
http://hadoop.apache.org/

50. MapReduce
Use at catinthehat.biz?
Find most popular non-English words in user stories:

51. MapReduce
Use at catinthehat.biz?
Find most popular non-English words in user stories:
def map(document):
for word in document:
if not isEnglishWord(word):
yield (word,1)

52. MapReduce
Use at catinthehat.biz?
Find most popular non-English words in user stories:
def map(document):
for word in document:
if not isEnglishWord(word):
yield (word,1)
def reduce(word, partialCounts):
return sum(partialCounts)

53. So
De-couple application components
Figure out a messaging strategy
Monitor your apps live
Vertical scaling is cheaper short-term

54. So
On demand storage (S3) & compute power (EC2)
Google App Engine for simple apps
Lots of tools available

55. “If you never did, you
should. These things
are fun, and fun is
good.” - Dr. Seuss