3. Introduction
Approaches
Social Graph
Consists of Nodes and Edges
Describes Entities and their Relation
Used by Facebook, Google, Amazon etc
About 100+ million nodes and 10+ billion edges
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
4. Introduction
Approaches
Problems and Motivation
amount of data exceeds capability of a single machine
necessary to distribute data and computation
data access managed by framework
different requirements (latency, throughput)
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
6. Introduction
Approaches
TAO: Data Model
data identified by 64-bit integer
Objects (id) → (otype, (key → value)*)
Associations (id1, atype, id2) → (time, (key → value)*)
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
7. Introduction
Approaches
TAO: API
fixed set of queries
assoc add, assoc delete, assoc change type
assoc get, assoc count, assoc range, assoc time range
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
8. Introduction
Approaches
TAO: Architecture
data divided into shard (via hashing)
each server handles one or more shard
objects and their associations are in the same shard
an object never changes the shard
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
9. Introduction
Approaches
TAO: Architecture
servers divided in leaders and followers
clients always communicate with followers
cache misses and writes redirected to leader
slave servers support master servers if necessary
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
11. Introduction
Approaches
TAO: Fault Tolerance and Performance
efficiency and availability > consistency
global mark for down server
followers are interchangeable
slave databases promoted to master, if master crashes
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
12. Introduction
Approaches
Conclusion
TAO: Fault Tolerance and Performance
Figure: Write Access Latencies
https://www.facebook.com/download/273893712748848/atc13-bronson.pdf
Effziente Verarbeitung von grossen Datenmengen Teil II
14. Introduction
Approaches
Horton: Data Model
similar to TAO
divided in partitions
additional data can be attached (e.g. key-value-pairs)
directed edges stored at source and target
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
16. Introduction
Approaches
Horton: Architecture
Graph Client Library translates query to regular expression
Graph Coordinator translates regular expression to finite state
machine and finds most effective execution plan
Graph Partitions executes the finite state machine and traverses
the graph
Graph Manager provides an interface to administrate the graph
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
21. Introduction
Approaches
Pregel: Basic Work Flow
1. copy task to worker machines, one is promoted to master
2. master assigns one or more partitions to each worker
3. master invokes supersteps
4. save graph after computation
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
22. Introduction
Approaches
Conclusion
Pregel: Fault Tolerance and Performance
workers save their progress at checkpoint supersteps
worker failure detected using ping
reassign partitions failed servers to available workers
reload state of the most recent available checkpoint superstep
process termination if master failed
Effziente Verarbeitung von grossen Datenmengen Teil II
23. Introduction
Approaches
Pregel: Fault Tolerance and Performance
Figure: varying number of worker on 1 billion vertex binary tree
http://kowshik.github.io/JPregel/pregel paper.pdf
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
24. Introduction
Approaches
Trinity
developed by Microsoft
flexible in data and computation
supports online query processing and offline computation
on top well-connected cluster (memory cloud)
based on TFS (similar to HDFS)
Strength low latency and high throughput (not at the same
time)
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
25. Introduction
Approaches
Conclusion
Trinity: Data Model
key-value-store
one table for nodes
one table for each type of relation
relations represented by id-pairs in the specific table
customisation possible with Trinity Structure Language (TSL)
data backed up in persistent file system
Effziente Verarbeitung von grossen Datenmengen Teil II
26. Introduction
Approaches
Trinity: API
Trinity Desktop Environment (TDE)
supports query requests (similar to Horton/SQL)
supports offline computation (similar to pregel)
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
27. Introduction
Approaches
Trinity: Architecture
Slaves Stores a part of the data, processes tasks and
messages.
Proxies Optional middle tier between slaves and clients.
Handles messages, does not store data.
Clients Responsible for user interaction with the cluster.
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
29. Introduction
Approaches
Conclusion
Trinity: Fault Tolerance and Performance
no ACID support, but atomicity of operations
dead machines are replaced by alive ones, reload memory from
TFS
requesting machine will wait till the dead machine is replaced
recovering the state of the most recent checkpoint superstep
(similar to pregel)
Effziente Verarbeitung von grossen Datenmengen Teil II
30. Introduction
Approaches
Trinity: Fault Tolerance and Performance
Figure: Response time of subgraph match queries
Effziente Verarbeitung von grossen Datenmengen Teil II
https://research.microsoft.com/pubs/161291/trinity.pdf
Conclusion
31. Introduction
Approaches
Unicorn
in-memory social graph-aware indexing system
search offering backend of Facebook
based on Hadoop
Strength Typeahead
Good performance on complex queries.
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
32. Introduction
Approaches
Unicorn: Data Model
sharded data (similar to Facebooks TAO)
indices built and converted using custom Hadoop pipeline
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
33. Introduction
Approaches
Conclusion
Unicorn: API
Queries in Unicorn Query Language
e.g. (term likers:104076956295773))
≈ 6M Likers of ”Computer Science”
apply allows to query a (truncated) set of id and then use
those to construct a new query
extract attaches matches as metadata within the forward
index of the query set
Effziente Verarbeitung von grossen Datenmengen Teil II
34. Introduction
Approaches
Conclusion
Unicorn: Architeture
top-aggregator dispatches the query to one rack-aggregator of
each rack, combines and returns result
rack-aggregator forwards the query to all index servers of its rack
(high bandwidth), combines results
index server about 40-80 machines per rack, stores adjacency
lists, performs operations
Effziente Verarbeitung von grossen Datenmengen Teil II
35. Introduction
Approaches
Unicorn: Fault Tolerance and Performance
sharding and replication
automatically replacing machines
serving incomplete results is strongly preferable to serving
empty results
Effziente Verarbeitung von grossen Datenmengen Teil II
Conclusion
36. Introduction
Approaches
Conclusion
Unicorn: Fault Tolerance and Performance
(apply friend: likers:104076956295773) ≈ Friends of Likers of
”Computer Science”
https://www.facebook.com/download/138915572976390/UnicornVLDBfinal.pdf
Effziente Verarbeitung von grossen Datenmengen Teil II