Keynote talk at the 2005 International Symposium on Distributed Objects and Applications (DOA 2005), 31 October 2005.

1. Content-Based
Publish/Subscribe:
A Re-Assessment
David S. Rosenblum
London Software Systems
University College London
OTM/DOA 2005 31 October 2005

2. Acknowledgments
 AlexanderL. Wolf University of Lugano
Antonio Carzaniga
 Costin Raiciu University College London
OTM/DOA 2005 31 October 2005

3. The ‘Fire Hose’
OTM/DOA 2005 31 October 2005

4. Controlling the Fire Hose
OTM/DOA 2005 31 October 2005

5. Controlling the Fire Hose
OTM/DOA 2005 31 October 2005

6. Publish/Subscribe
symbol ==MSFT
symbol IBM
price = 29.34
83.47
30.17
symbol == MSFT
&&
pri
sym ce =
bo 30.
l= 17 price > 30.00
MS
FT
OTM/DOA 2005 31 October 2005

7. Publish/Subscribe Features
 Asynchronous delivery
 Multi-way delivery
 Content-driven interaction
 Anonymity
 Strong decoupling
Many applications are a natural fit
OTM/DOA 2005 31 October 2005

8. Some Ancient History
YEAST
 Pub/sub for LANs of UNIX workstations
 Centralised server implementation
 Novelty: Applications
.cpp
 Process awareness
 Office automation
 Telco feature deployment
 Many others
.h
OTM/DOA 2005 31 October 2005

9. Some Ancient History
YEAST
 Pub/sub for LANs of UNIX workstations
 Centralised server implementation
 Novelty: Applications
OTM/DOA 2005 31 October 2005

10. Some Ancient History
YEAST
 Pub/sub for LANs of UNIX workstations
 Centralised server implementation
 Novelty: Applications
.cpp
 Process awareness
.h
OTM/DOA 2005 31 October 2005

11. Some Ancient History
YEAST
 Pub/sub for LANs of UNIX workstations
 Centralised server implementation
 Novelty: Applications
 Process awareness
212-555-8076
 Office automation
OTM/DOA 2005 31 October 2005

12. Some Ancient History
YEAST
 Pub/sub for LANs of UNIX workstations
 Centralised server implementation
 Novelty: Applications
 Process awareness
212-555-8076
 Phone call awareness
 Telco feature deployment
 Several others
OTM/DOA 2005 31 October 2005

13. Some More Recent History
SIENA
 Wide-area content-based publish/subscribe
 Decentralised overlay network of
publish/subscribe ‘routers’
 Routing and forwarding based on
subscription and notification content
 Novelty:
Algorithms, Protocols, Architectures
Assumed that the applications
would naturally appear!
OTM/DOA 2005 31 October 2005

14. Most Recently
PreCache
 Sony-funded startup to commercialise
content-based publish/subscribe
 Survived 2.5 years
 Successful technology development
 Less successful business development
 Video-on-demand (???)
 Anti-virus updates
 Travel alerts
OTM/DOA 2005 31 October 2005

15. So What Are the Killer
Applications?
 Many research projects
 Many novel research results
 No significant deployments yet
Need to take a closer look
at some proposed approaches
OTM/DOA 2005 31 October 2005

16. SIENA Content-Based Routing
Subscription Forwarding
s1:1
s1:1
s1 s1: “price < 700”
a s1:a
s1:a 2
1
s1:2
s1:2
s1:2
s1:2 3
5
s1:1
s1:1 4 s1:3
s1:3 6
s1:3
s1:3
7
8
s1:5
s1:5
s1:6
s1:6
9
OTM/DOA 2005 31 October 2005

17. SIENA Content-Based Routing
Subscription Merging
s1 covers s2 s1:1
s1:1
s2:covers s< 600”
s1 “price 2
s1:1
s1:1
s2:5
s2:5
s1:a
s1:a
a s1:a
s1:a 2
1 s2:2
s2:2
s1:2 s1:2
s1:2
s1:2
s1:2
s1:2
s2:8 3
s2:8
5
s1:1
s1:1 4 s1:3
s1:3 6
s1:3
s1:3
7
s2
s1:5
s1:5 8
b s1:5
s1:5
s2:b
s2:b 9
s1:6
s1:6
OTM/DOA 2005 31 October 2005

18. SIENA Content-Based Routing
Notification Delivery
s1:1
s1:1
s2:5
s2:5 n1: “price = 550”
s1:a
s1:a
a 2
1 s2:2
s2:2
s1:2 s1:2
s1:2
s1:2
s2:8
s2:8 3
5
s1:1
s1:1 4 s1:3
s1:3 6
s1:3
s1:3
7
n1
s1:5
s1:5 8
b s2:b
s2:b 9
s1:6
s1:6
OTM/DOA 2005 31 October 2005

19. Implications of SIENA’s Design
 Notifications can be very frequent
 But subscriptions should be relatively
infrequent
 Yet there should be a lot of subscription
variation
 But there should be some similar
subscriptions
 And the similar subscriptions should come
from the same part of the network
Which applications are like this?
OTM/DOA 2005 31 October 2005

20. Other Approaches
 Gryphon
 Subscription flooding over tree of clusters
 Applicable if subscriptions are few and stable
 Hermes
 Rendezvous nodes allocated to content types
 Applicable if load is spread evenly by type
 PreCache
 Trie- and kd-tree-based subscription storage
 Applicable if unsubscription occurs very infrequently
All of these limit application suitability
OTM/DOA 2005 31 October 2005

21. Publish/Subscribe Features
Conceptual Features Infrastructure Features
 Asynchronous delivery  Message flooding
 Multi-way delivery  Subscription merging
 Content-driven interaction  Tree-based routing
 Anonymity  Localised forwarding
 Strong decoupling  Content partitioning
Few applications can naturally exploit these features
OTM/DOA 2005 31 October 2005

22. Example
Stock Quotes vs Online Gaming
Stock Quotes Online Gaming
 Message flooding  Message flooding
? Subscription merging ? Subscription merging
 Tree-based routing  Tree-based routing
 Localised forwarding ? Localised forwarding
 Content partitioning  Content partitioning
One size infrastructure does not fit all
OTM/DOA 2005 31 October 2005

23. Matching Applications with
Infrastructures
Application ??? Infrastructure
Characteristics Characteristics
 Notification size  Number of routers
 Notification throughput  Number of routing hops
 Notification latency  Path redundancy
 Notification variability  Subscription replication
 Subscription selectivity  Matching complexity
 Subscription stability  Matching accuracy
 Locality
 …
 …
OTM/DOA 2005 31 October 2005

24. Example
Stock Quotes vs Online Gaming
Stock Quotes Online Gaming
 Notification size  Notification size
 Notification frequency  Notification frequency
 Notification variability  Notification variability
 Notification latency  Notification latency
 Subscription selectivity  Subscription selectivity
 Subscription stability  Subscription stability
 Locality ? Locality
How do we translate these to design decisions?
OTM/DOA 2005 31 October 2005

25. Additional Complications
 Mobility
 Of publishers
 Of subscribers
 Of routers
 Firewalls
 Edge Fanout
 Security
OTM/DOA 2005 31 October 2005

26. The Value of Information
Can we do secure content-based routing
over an OTM/DOA 2005
untrusted infrastructure? 2005
31 October

27. Security in Content-Based
Publish/Subscribe
 Encryption used to implement many security goals
 Authentication
 Confidentiality
 Integrity
 But content-based routing intrinsically requires
some transparency of content
 Infrastructure must be able to determine if a subscription
matches notification
 Existing approaches have limited applicability
 In large part due to need to secure multiple messages
OTM/DOA 2005 31 October 2005

28. A Cryptographic Protocol Based
on Yao’s Garbled Circuits
 Subscriptions transformed to Boolean
circuits and then garbled based on shared
secret
 Notifications encrypted with shared secret
 Router evaluates circuit on encrypted
notification
 Router knows result but not content!
Weak but inexpensive security
OTM/DOA 2005 31 October 2005

29. A Cryptographic Protocol Based
on PSM
 PSM = Private Simultaneous Messages (Feige et al.)
 Subscription matching transformed to graph
reachability
 Notifications and subscriptions transformed to
subgraphs and encrypted based on shared secret
 Router sums adjacency matrices for subgraphs
 Router checks rank of resulting matrix for match
 Router knows result but not content!
Better security but very expensive
OTM/DOA 2005 31 October 2005

30. Inherent Security Limitations
(1)
 Must provide confidentiality of both
notifications and subscriptions
 Range of plaintext notifications can be matched
against confidential subscription
 Range of plaintext subscriptions can be matched
against confidential notification
 Router must know outcome of match
 This alone can sometimes be useful information
 Example: Battlefield Awareness
OTM/DOA 2005 31 October 2005

31. Inherent Security Limitations
(2)
 Router can determine subscription coverage
over time
 Again, this may be useful information
 Router can determine Euclidean distance
between notifications over time
 Studied protocols require sharing of secret
among potentially large number of
publishers and subscribers
OTM/DOA 2005 31 October 2005

32. Inherent Limitations of Possible
Security Solutions
 Cryptographic group membership protocols
 Too expensive with high subscription volatility
 Padding notification stream with dummy messages
 Reduces throughput and increases latency of
infrastructure
 Defeats the whole purpose of the infrastructure!
 Proxy publishers and subscribers
 Increases latency of messages
 Trusted infrastructure
 Can be expensive to deploy for each application
OTM/DOA 2005 31 October 2005

33. A Generic Architecture for
Content-Based Matching
Cluster
Cluster
2
3
Cluster
1
Cluster
Cluster 4
C
 Separates matching from routing
 Fully-connected mesh of N nodes in C clusters
 Full connectivity simulated on DHT with minimal overhead
 Choose 2 of 3 configuration parameters
 Subscription replication rate R (= N/C)
 Notification routing hops H (1 ≤ H ≤ C)
 Load-balancing factor B 2005
OTM/DOA 31 October 2005

34. Conclusion
The Past
 There have been many innovations in wide-
area content-based publish/subscribe
 But researchers have ignored application
characteristics for too long
 A universal infrastructure shared by all
applications is probably not feasible
 Security is very difficult to achieve over an
untrusted infrastructure
OTM/DOA 2005 31 October 2005

35. Conclusion
The Future
 We need to understand better the
relationship between application
requirements and infrastructure design
 Andwe need to explore further the limits of
security in content-based publish/subscribe
OTM/DOA 2005 31 October 2005

36. Questions?
Prof. David S. Rosenblum
London Software Systems
University College London
d.rosenblum@cs.ucl.ac.uk
http://www.cs.ucl.ac.uk/staff/D.Rosenblum/
OTM/DOA 2005 31 October 2005