Monica Wilkinson and Andy Piper present how to use Cloud Foundry and how to deploy an Activity Stream Engine on Cloud Foundry written in Node.js using MongoDB and Redis on the back end.
2. Andy Piper social bridgebuilder, techie
Developer Advocate @ Cloud Foundry
15 years in enterprise middleware & messaging
social web enthusiast
OSS supporter / contributor
excited by “what’s next”, Internet of Things, etc.
@andypiper
apiper@vmware.com
CONFIDENTIAL
2
Wednesday, June 20, 12
3. About Monica Wilkinson Loves the web and data portability.
Developer Advocate @ Cloud Foundry
12 years development experience.
Last 5 years in Social Web
Open Web Standards Advocate
Contact Me: @ciberch
mwilkinson@vmware.com
CONFIDENTIAL
3
Wednesday, June 20, 12
4. Agenda
1.Overview of Cloud Foundry
2.What is Activity Streams ?
3.Building an Activity Stream Engine
CONFIDENTIAL
4
Wednesday, June 20, 12
5. About Cloud Foundry The first Open PaaS
Multi(n) Languages, Frameworks,
Services & Clouds
Open Source
CONFIDENTIAL
5
Wednesday, June 20, 12
6. Traditional web app architecture
! Browser client
Web Application
! Monolithic WAR / other
deployment artefact StoreFront
! RDBMS
AccountingService
MySQL
Desktop Database
Apache
Browser
InventoryService
“easy” to...
ShippingService
! develop
! test
Container (e.g. Tomcat)
! deploy
! scale ...apart from: provisioning, setup, need to replicate, shard, scale to massive numbers of
mobile clients, provide polyglot support, run multiple dev teams, collaborate...
CONFIDENTIAL
6
Wednesday, June 20, 12
7. Need to handle massive loads and the data explosion
Scalable architectures!
! Application tier:
•Replicated/clustered servers
•Modular so that components can be scaled differently
•Asynchronous architecture - communication via a message broker
! Database tier:
•Replication
•Sharding
•Polyglot persistence: Relational, NoSQL, NewSQL databases
CONFIDENTIAL
7
Wednesday, June 20, 12
8. Modern application architecture
billing web application
Desktop Browser Accounting
Service
StoreUI
Native Mobile NodeJS
NodeJS inventory web application
Application
front-end MySQL
StoreUI
application
StoreUI InventoryService
HTML5 mobile
application
StoreUI Redis shipping web application
RabbitMQ
Mongo
ShippingService
CONFIDENTIAL
8
Wednesday, June 20, 12
9. Platform-as-a-Service is the solution
Deployment Services
Easy deployment SQL databases
Application management NoSQL databases
+
Easy scaling up and down Message Broker
CONFIDENTIAL
9
Wednesday, June 20, 12
10. The Open
Platform as a Service
“Deploy and scale applications in
seconds, without locking yourself into a
single cloud or vendor”
,
xible
n, Fle
Ope e
ple, l
Si m Sc alab
CONFIDENTIAL
10
Wednesday, June 20, 12
11. Cloud Foundry open PaaS - Choice of clouds
OSS community
Private(
Ap
I)
Data Services
CP
Clouds(
pli
vFabric
(
ca
ce
Postgres
tio
r fa
nS
e
Int
er
Public
v ic
r
ide
eI
Msg Services
ov
Clouds
nt e
Pr
r fa
vFabric RabbitMQTM
d
ou
ce
Cl
Micro
Other Services
Clouds to
h: d eploy c)
! bos et
N EW PIs (AWS
n at e C
Additional partner services … alter
CONFIDENTIAL
Wednesday, June 20, 12
12. Multi-cloud flexibility is critical to long-term success
! Make use of both public and private clouds without rewriting your applications
! Protect against vendor lock-in
! Meet different compliance and geographical needs
! Accommodate peak loads while optimizing costs
! Manage your growth and changing needs over time
CONFIDENTIAL
12
Wednesday, June 20, 12
19. What is Activity Streams ?
CONFIDENTIAL
19
Wednesday, June 20, 12
20. http://activityStrea.ms
! Activity Streams is a simple open specification used to describe social
actions around the web.
! The goal of Activity Streams is data portability
! Some publishers of Activity Streams: Socialcast, MySpace, Facebook
and G+
! The default format is JSON but Atom is also supported. The JSON
format was added after we saw some initial limitations with JSON
! Core concepts are: Actor, Verb, Object and Target
CONFIDENTIAL
20
Wednesday, June 20, 12
24. What is an Activity Stream Engine ?
! An activity stream engine allows you to publish events and
subscribe to events.
! Many activity stream engines support aggregation of events
via Streams.
! Many activity stream engines support fanning out activities
to subscribers.
CONFIDENTIAL
24
Wednesday, June 20, 12
25. So how did we build this Activity Stream
Engine on Cloud Foundry ?
CONFIDENTIAL
25
Wednesday, June 20, 12
27. What should we use to store Activities ?
Key-Value Column Document Graph
Redis, Cassandra, MongoDB Neo4J
Riak HBase
CONFIDENTIAL
27
Wednesday, June 20, 12
28. Mongo DB schema-less documents are perfect
for Activities
Key-Value Column Document Graph
MongoDB
(so easy)
CONFIDENTIAL
28
Wednesday, June 20, 12
29. About MongoDB
1. No-SQL database development : {
2. Stores JSON-style tools: many,
documents with language_support: superb,
embedded documents
=
agility: high
3. Horizontally scalable },
production: {
4. Full Indexing Support
speed: fast
5. Open Source so great fault_tolerance: true
ORMs and drivers.
scalability: high
CONFIDENTIAL
29
Wednesday, June 20, 12
32. Step 2 - Expose helpers for the queries
! The most important is to be able to list the activities in
descending order by published time.
! It is also important to hydrate any objects for which we
have references
! With Mongoose you can do:
! this.getActivityStream = function(n, fx)
{ Activity.find().sort('published',
'descending').limit(n).populate('target').run
(fx);
}
! https://github.com/ciberch/activity-streams-mongoose
CONFIDENTIAL
32
Wednesday, June 20, 12
33. How do we make our engine faster and more
scalable ?
! Add support for PubSub
! In a PubSub model as soon as an event is published it
gets sent to all the subscribers.
! The Publisher doesn’t know who the subscribers are, it
simply knows where to publish.
! Redis has nice support for Pub Sub http://redis.io/topics/
pubsub
CONFIDENTIAL
33
Wednesday, June 20, 12
34. Using Socket.io
! Server Side:
• Subscribe to a Stream
• When there is a new event send it to the client as JSON
! Client Side
• Add support to publish activities to the Stream
• Change the client rendering to use ActivityStrea.ms
! Benefits
• Richer messaging
• Activity Syndication
CONFIDENTIAL
34
Wednesday, June 20, 12