In this lunchtalk we discuss p2p-based online social networks

1. P2P-based Monitoring Mechanism
and LifeSocial.KOM
Diensteebene: Monitoring
Soziales Wissensnetz
QuaP2P
Improving the quality of P2P systems
DFG research group 733
KOM - Multimedia Communications Lab
Prof. Dr.-Ing. Ralf Steinmetz (Director)
Dept. of Electrical Engineering and Information Technology
Dept. of Computer Science (adjunct Professor)
Dipl.Math., Dipl.Inform Kalman Graffi TUD – Technische Universität Darmstadt
Rundeturmstr. 10, D-64283 Darmstadt, Germany
Kalman.Graffi@KOM.tu-darmstadt.de Tel.+49 6151 166150, Fax. +49 6151 166152
Tel.+49 6151 164959 www.KOM.tu-darmstadt.de
Dagstuhl_DigitaleSozialeNetze_2010___Kalman_Graffi___LifeSocial_v2.ppt 17. Februar 2011
© 2009 author(s) of these slides including research results from the KOM research network and TU Darmstadt. Otherwise it is specified at the respective slide

2. QuaP2P 2 Overview
KOM – Multimedia Communications Lab 2

3. Qualität von P2P-System- und
Dokumentenmanagement
P2P-Diensteebene:
Zusatzdienste für die P2P-Anwendung
Kritisches Anwendungsszenario Soziales Wissensnetz
(Soziales Wissensnetz) umfasst:
Komplexe Dokumentnetze
Breites Spektrum an
Dienstgüteanforderungen
Wechselwirkungen der Mechanismen
Ziel: Dokumentverwaltung im H(„ my data“ )
stabilen P2P System = 3107
709
1008 1622 2011
2207
Systemweite Validität der ?
Dokumentenzustände und –beziehungen 611
3485
2906
Erfassung und Erhaltung der Dienstgüte 12.5.7.31
peer-to-peer.info
berkeley.edu planet-lab.org 89.11.20.15
95.7.6.10
86.8.10.18 7.31.10.25
KOM – Multimedia Communications Lab 3

4. Systemmanagement und Selbstoptimierung
Ziel: Bessere Verwaltbarkeit und garantierte Dienstgüte des P2P-Systems
Herausforderungen: Vorgehen:
Wechsel der Dienstgüteprioritäten im Statistische Erfassung des
dynamischen System Systemzustandes (Monitoring)
z.B. Skalierbarkeitsmechanismen Interpretation des Systemzustandes und
weniger wichtig im Netz mit 50 Knoten Ableiten einer Dienstgütestrategie
z.B. Lastverteilung erst wichtig, wenn Umsetzen und Evaluieren der
Knoten überlastet sind Dienstgütestrategie
Systemweite Dienstgüte im verteilten
System erreichen
z.B. koordiniertes Erreichen von
geringen Antwortzeiten
KOM – Multimedia Communications Lab 4

5. Benchmarking of Monitoring Solutions
Function: Get Global system Metrics:
statistics Precision: absolut and relative
Statistical information: monitoring error
avg, min, max, standard dev., sum,.. Traffic costs: msg/sec, bandwidth
E.g. on CPU usage, bandwidth consumption: avg, max, stddev
utilization, hop count, messages sent / Max. difference of age of view
received, number of peers, Message Information age
sizes according to message type,…
…’
Assumptions / Dependencies:
P2P Overlay with KBR functionality
Metriken, SkyEye,
Bewertung
4,
Anforderungen PushSum
6
Basis-Benchmarking-
Szenario
KOM – Multimedia Communications Lab 5

6. SkyEye.KOM – Architecture Design Decisions
Integrated vs. new layer
New layer allows wider applicability
Set on top of KBR-compatible structured p2p overlays
Reactive vs. proactive
System state information is continuously interesting for all users
Allows for fast queries
Monitoring topology: bus, ring, star, mesh, tree
Tree structure alleviate information aggregation
Fixed out and in degree
Position assignment: dynamic vs. deterministic
Deterministic IDs used in topology, dynamically resolved with DHT
For all structured P2P overlays
Covered by DHT-function: route(msg, key), lookup(key)
Usable by all functional layers/modules in the P2P system
K. Graffi et al. “SkyEye.KOM: An Information Management Over-Overlay for Getting the Oracle View on Structured P2P Systems”Communications Lab
KOM – Multimedia 6
IEEE International Conference on Parallel and Distributed Systems (IEEE ICPADS ‘08), December 2008

7. Topology of SkyEye.KOM
Coordinator_ID 0,5
Concept of Over-overlay C0
Built on underlying structured overlay C_ID 0,25 C_ID 0,75
1 1
Unified ID space [0,1] decouples C C
C_ID 0,125 C_ID 0,625 C_ID 0,875
from specific DHT implementation 2 2
C2 C_ID 0,375 C C
Communicates via common API
C2
route(msg, key)
C_ID 0,3125
C3
Information Domains:
0,09 0,2 0,31 0,4 0,5 0,6 0,75 0,9
Peer ID determines position in tree 0 1
Receive information from children nodes
Sends aggregated information to father
50 1
node (Coordinator) 45
10
DHT
15
40 20
30
Protocols for monitoring
System-specific information
Peer-specific information
K. Graffi et al. “SkyEye.KOM: An Information Management Over-Overlay for Getting the Oracle View on Structured P2P Systems”Communications Lab
KOM – Multimedia 7
IEEE International Conference on Parallel and Distributed Systems (IEEE ICPADS ‘08), December 2008

8. Gathering System-specific Information
Some design decisions
Equal roles for all peers
Load similar for all peers in all positions
Aggregation of statistics
Sum, min, max, average
Standard deviation, count
[µ,σ,σ²,Σ,
Statistic updates min,max]
Periodically sent to parent peer
[µ,σ,σ²,Σ,
Aggregated in each node ( same size) min,max]
Global view in root
Every update is ACKed with global [µ,σ,σ²,Σ,
view from above min,max]
KOM – Multimedia Communications Lab 8

9. Gathering System-specific Information
Some design decisions
Equal roles for all peers
Load similar for all peers in all positions
Aggregation of statistics
Sum, min, max, average
Standard deviation, count
[µ,σ,σ²,Σ,
Statistic updates min, max]
Periodically sent to parent peer
Aggregated in each node ( same size) [µ,σ,σ²,Σ,
min, max]
Global view in root
Every update is ACKed with global
[µ,σ,σ²,Σ,
view from above min, max]
KOM – Multimedia Communications Lab 9

10. Some Remarks on SkyEye.KOM and
Monitoring System Statistics
Why is it generally applicable on DHTs?
Unified ID space, using core DHT functions
(Key based Routing API)
Coordinator_ID
C 0 0,5
Why is it robust against churn? C_ID 0,25 C_ID
1 1 0,75
If peer fails: automatically replaced in the DHT C C
Updates are routed to new peer for aggregation C_ID 0,125 C_ID 0,625 C_ID 0,875
2 2
C2 C_ID 0,375 C C
2
C
Why are costs low?
C_ID
One update: ~1kb, 0,3125 C3
Out + in degree = 1 + tree degree (2 or 4)
Independent of position in the tree! 0,09 0,2 0,31 0,4 0,5 0,6 0,75 0,9
0 1
Age of information:
50 1
Limited by tree depth, O(log (N)) 10
45 DHT
Influenced by update period 15
40 20
30
Just two message types: Update, ACK
Assumed functions:
route(msg, key), amIresponsible(key)
KOM – Multimedia Communications Lab 10

11. Benchmarking Approach
Workload: artificial or based on gaming / social network / catastrophe
Scalability criteria: traffic load on peer = O(1)
Efficiency criteria:
Fixed costs: Maximum bandwidth consumption = 1 Kb/s
Comparative performance: minimize relative monitoring error
Stability:
Action: remove 50% of peers
Metric: Time until measurement error is <5% for at least 5 “rounds”
Freshness (Coherency):
Average age monitoring data in the peers
Multilateral Interdependencies:
Vary metrics, see changes in the quality properties, describe interdependencies
KOM – Multimedia Communications Lab 11

12. QuaP2P 2 Overview
KOM – Multimedia Communications Lab 12

13. What do I focus on?
Social implications, social sciences
Users and their interactions
Usage scenarios
Apps
Technical Platform
Data storage Infrastructure
Bits and Bytes
KOM – Multimedia Communications Lab 13

14. Plethora of Online Social Networks
KOM – Multimedia Communications Lab 14

15. Online Social Networks
What are ‘Online Communities’ technically?
Web-based applications (StudiVZ, Facebook, MySpace, Xing)
Provide different services for community members
Personal Events
information
and photos
Plugin
Games
architecture
Friends Social
interaction
KOM – Multimedia Communications Lab 15

16. Goals and Motivations
Users want System providers want
Storing and searching for content High profit
Profiles, friend lists, … Many users
Pictures, shared “Wall” editing, … Personalized advertisements
User to user interaction Low operational costs
Chatting, VoIP, … For servers, electricity, cooling …
Games For personnel, legal issues
Security Controlled Quality of Service
Access control on their data To attract and keep users
Secure, confidential communication Providing reliable, high quality services
Fun! Money!
Our goal: all of the above following another IT paradigm
KOM – Multimedia Communications Lab 16

17. How do they work?
What is the architecture beneath?
KOM – Multimedia Communications Lab 17

18. Current IT Paradigm: Client / Server
Web-based solution
Lots of operational costs!
Rough estimation: 1$/y per user
Facebook: 350M users !
KOM – Multimedia Communications Lab 18

19. Alternatives? – Peer-to-Peer based Platforms
Idea: Platforms:
Use capacities of user devices (Moore’s law!) LifeSocial.KOM
Interconnect users with p2p-overlay SafeBook, PeerSon
Provide all functionality in a distributed way
Shift the load and costs to
the users
KOM – Multimedia Communications Lab 19

20. Our Solution: LifeSocial.KOM
Researched since end of 2007
Ca. 10 diploma / bachelor theses on this topic
Ca. 20 students programming plugins / GUIs in “Praktika” / project seminars
See: www.lifesocial.org
KOM – Multimedia Communications Lab 20

21. How does it look like?
What can you do?
KOM – Multimedia Communications Lab 21

22. Screenshots
See: www.lifesocial.org KOM – Multimedia Communications Lab 22

23. Screenshots
See: www.lifesocial.org KOM – Multimedia Communications Lab 23

24. Screenshots
See: www.lifesocial.org KOM – Multimedia Communications Lab 24

25. How does this work?
What is the architecture beneath?
KOM – Multimedia Communications Lab 25

26. Architecture Overview on LifeSocial.KOM
Extendable framework for user interface
components User Interface
Stand-alone applications, core functionality
and optional functionality of the system. Optional Plugins
Extendable.
Mandatory Plugins
Caching of data objects and messages Information Cache Monitoring
Monitoring of the quality of service
Low-delay user-to-user communication Secure Storage Secure Message
Storage (store, modify, retrieve, delete) and Dispatcher Dispatcher
Storage and
Replication
Distributed storage and replication
Organization of nodes in an overlay network Peer-to-Peer Overlay
Standard Internet protocols Internet
KOM – Multimedia Communications Lab 26

27. Categories of Peer-to-Peer Systems
Unstructured P2P Structured P2P
Centralized P2P Pure P2P Hybrid P2P DHT-Based
1.Central entity is 1.Any terminal entity 1. Any terminal entity can be 1. Any terminal entity
necessary to provide can be removed with- removed without loss of can be removed without
the service out loss of functionality functionality loss of functionality
2.Central entity is some 2. No central entities, 2. Dynamic central entities for 2. “Fixed” connections in the
faster search overlay network
kind of index database fully distributed
3.Search costs: variable 3.Lookup costs: O(log n)
3.Search costs: O(1) 3.Search costs: O(n)
4.Costs for state: variable 4.Costs for state: O(log n)
4.Costs for state: O(n) 4.Costs for state: O(1)
5.For: Searches 5.For: Lookup
5.For: Searches 5.For: Searches
For keyword-based Search: For Lookup:
used query mechanism: flooding, random-walk… Routing
1st Generation 2nd Generation KOM 3rd Generation
– Multimedia Communications Lab 27
from R.Schollmeier and J.Eberspächer, TU München

28. FreePastry – Most Used Academic DHT
FreePastry – based on Pastry, DHT User Interface
Documents are mapped to peers: Mandatory Plugins
Optional Plugins
for every Document-ID there is a responsible peer Information Cache Monitoring
all document owners and requesters contact this peer Storage Dispatcher Message Dispatcher
FreePastry routes to responsible peer Storage and
Replication
Functions: void put(key, Object), Object get(key) Peer-to-Peer Overlay
Internet
Node 1008
queries item 3000 Use shortcuts/fingers…
1008 1622 2011
1009-1622
709 710-1008 1623-2011 2207 Responsible for
660-709 2012-2207 1008 + 1024
1 2682
2208-2682
3 2
Responsible 659 2906
peer found 612-659 611
3486-…
3485 2683-2906
2907-3485 Responsible for
0-611 Responsible for
2207 + 512
3000
KOM – Multimedia Communications Lab 28

29. DHT used for Storing Social Data
Functionality: Application areas:
Totally distributed storage E.g. filesharing using Kademlia
Reliable through replication and self- Proven to be scalable for low costs
organization
Using only user devices
?
KOM – Multimedia Communications Lab 29

30. What kind of data can you store?
And where?
Is it secure?
KOM – Multimedia Communications Lab 30

31. Document Types, Obvious Storage Keys
User Albums User album A Image x Profile
storage key = storage key p =
storage key a storage key x
„user name“+“album“ “User_Kalman_Graffi”
List of user albums: List of images: image
1. storage key a 1. storage key x Name: Kalman
2. storage key b 2. storage key y Age: 27
3. storage key c 3. storage key v University:
Image y
4. storage key d 4. storage key r Technische
... ... Universität
storage key y Darmstadt
User album D Image n
image
storage key d storage key n
List of images: image High granularity of stored data objects
1. storage key n
2. storage key m Better load balancing of the resources
3. storage key k Image m
4. storage key l
Used for
... Atomic data: profiles, login info, “emails”
storage key m
Linked lists: friend lists, groups, multicast
image
Allows for complex data structures
See: K. Graffi et al., “A Distributed Platform for Multimedia Online Communities” KOM – Multimedia Communications Lab 31
In: IEEE International Symposium on Multimedia '08 (IEEE ISM’08), December 2008.

32. Data Positioning in the Network
KOM – Multimedia Communications Lab 32

33. Simple Idea of Distributed Access Control
How to provide Access Control in a distributed environment?
Goal: Assign read-rights on objects to privileged users
Mechanism: Sym. encrypted objects, asym. encrypted sym. keys
For
See: K. Graffi et al., “Practical Security in P2P-based Social Networks” KOM – Multimedia Communications Lab 33
In: IEEE Local Computer Networks '09 (IEEE LCN’09), October 2009.

34. Detailed Idea of Distributed Access Control
Step 1:
PeerID SharedItem [userID A] =
3 Pub
objectID Header User A
= UserID
Privileged users userIDs [userID B] =
= PublicKey are public Pub
User B
1 keys
= f(username, Payload
extract
wrap symmetric key 4
passphrase) with public key
Serialized and encrypted with Symmetric Key
Secure comm.: 2
symmetic key
- encrypted
Signed CryptedItem Encrpyted
Pub
- signed 5 with User A
objectID
Key list
Symmetric Key
Byte array userID A – key A
containing userID B – key B
userID C – key C
encrypted
Encrpyted
…
SharedItem Pub
with User B
See: K. Graffi et al., “Practical Security in P2P-based Social Networks” KOM – Multimedia Communications Lab 34
In: IEEE Local Computer Networks '09 (IEEE LCN’09), October 2009.

35. What are the applications?
Where are the limitations?
KOM – Multimedia Communications Lab 35

36. Architecture Components
Plugins implement the funcitonality of
User Interface
online social networks (and more)
Information Cache: Optional Plugins
Enables the Plugins to reuse the data Mandatory Plugins
Monitoring:
Information Cache Monitoring
Provides statistics on system behavior
Secure Storage Secure Message
Secure Message Dispatcher: and Dispatcher Dispatcher
Sending: for low-delay user-to-user
communication, secure and authenticated Storage and
Receiving: dispatches incoming messages to Replication
addressed Plugins
Peer-to-Peer Overlay
Secure Storage Dispatcher:
Storage and retrieval of data objects with
distributed access control Internet
KOM – Multimedia Communications Lab 36

37. Plugin Architecture Overview
Everything is a Plugin
Plugins are stand-alone applications
Mandatory or optional
Plugins implement common interfaces
Every Plugin has an unique Identifier
Plugins and User Interfaces are decoupled
Easy Plugin-to-Plugin communication
Over shared storage
Over Plugin ID based messaging
See: K. Graffi et al., “LifeSocial.KOM: A P2P-based Platform for Secure Social Online Networks” KOM – Multimedia Communications Lab 37
submitted to IEEE Networking ‘10, January 2010.

38. Relevant for QuaP2P
Plugins are “Services”
Provide functionality, define dependencies
Relation to Anwendungsebene:
OSGi: can be loaded in runtime, service
repository in network possible
Mobile code is there mobile state is needed
Additional functionality for support:
Reliable resource reservation
Based on SkyEye.KOM:
Monitoring peer-specific resources
Additional reservation scheme:
Give me 5 peers with 50kb/s, 10MB for 2
months for deploying my service
Evaluation:
100% successful resource reservation under
churn possible
reservaation cost : 5-15 redundant peers
Max 100kb overhead per service
KOM – Multimedia Communications Lab 38

39. LifeSocial.KOM as huge testbed for
Diensteebene and Anwendungsebene
KOM – Multimedia Communications Lab 39

40. Monitoring and Evaluation
Integration of a monitoring solution
Totally distributed, precise and cheap
Global system statistics
Statistics on
CPU / bandwidth usage
Data retrieval delays
Messages sent / received
Number of peers
Objects in Cache
Friends and clustering coefficient
…
Statistical information:
avg, min, max, standard dev., sum,...
See: K. Graffi et al., “Monitoring and Management of Structured Peer-to-Peer Systems” KOM – Multimedia Communications Lab 40
In: IEEE Peer-to-Peer Computing '09 (IEEE P2P’09), September 2009.

41. Summary
IT solutions for social networks Analysis of needs:
Currently centralized and very costly Users want
Scales only with high monetary invests Storing and searching for content
User to user interaction
Distributed, p2p-based platforms Security
Data storage is totally distributed
Costs are shared among the users System provider want
Low operational costs
LifeSocial.KOM Controlled quality of service
Operational prototype High profit
Secure, reliable storage and messaging
Monitoring mechanism to observe (and Next steps:
control) the quality of service Integrate management mechanisms
Rich, extendable functionality through Run Internet-wide beta-test
Plugin-based architecture
Deploy
See videos on www.lifesocial.org
KOM – Multimedia Communications Lab 41

42. Questions?
KOM
Have a look at:
www.lifesocial.org
www.skynet-project.com
www.kom.tu-darmstadt.de
KOM – Multimedia Communications Lab 42