1. PREGEL
A
Systems
for
Large
Scale
Graph
Processing
by Iuliia Proskurnia
ber 2012
ovem
6 th of M

2. GRAPHS
ARE
EVERYWHERE
Graphs
Examples
2

3. GRAPHS
ARE
EVERYWHERE
Algorithms
3

4. Outline
Mo#va#on
Basic
Concepts
in
design
Implementa#on
Details
Evalua#on
Conclusions
4

5. MOTIVATION
Large
Graph
Processing
No Such System Exist
Single Machine
Algorithms
Parallel Solution
5

6. MAPREDUCE
LIKE
SOLUTION
MapReduce
is
great
:)
6

7. MAPREDUCE
LIKE
SOLUTION
MapReduce
is
great
:)
BUT
6

8. MAPREDUCE
LIKE
SOLUTION
MapReduce
is
great
:)
BUT
Pregel
6

9. Inspira@on
Valiant’s
Bulk
Synchronous
Parallel
Model
7

10. BASIC
DESIGN
CONCEPTS
8

11. VERTEX-­‐CENTRIC
APPROACH
Uses
supersteps
for
computa@on
Send/Receive Messages
Change the state
Modify topology
VERTEX
Termination?
9

12. MAXIMUM
VALUE
EXAMPLE
Chicken
Chicken
• Dotted Arrows -
messages
• Grey Nodes -
InActive
10

13. API
DETAILS
Combiners
11

14. API
DETAILS
Aggregators
12

15. IMPLEMENTATION
DETAILS
13

16. IMPLEMENTATION
Master
is
chosen
14

17. IMPLEMENTATION
Master
is
chosen
Cluster
Management
System’s name
service
14

18. IMPLEMENTATION
Par@@on
hash(VertexID) mod R
hash(VertexID) mod R
hash(VertexID) mod R
15

19. IMPLEMENTATION
Reading
the
input
16

20. IMPLEMENTATION
Reading
the
input
GFS,
BigTable
16

21. IMPLEMENTATION
SuperStep
17

22. IMPLEMENTATION
SuperStep
17

23. IMPLEMENTATION
SuperStep
Termination
if (all VoteToHalt) {
terminate();}
17

24. IMPLEMENTATION
Saving
the
results
Save graph state
18

25. FAULT-­‐TOLERANCE
CheckPoin@ng.
Chicken
Chicken.
19

26. FAULT-­‐TOLERANCE
CheckPoin@ng.
Chicken
Chicken.
19

27. EVALUATION
20

28. EVALUATION
Number
of
Worker
Tasks
300 PCs
Multicore
Billion Vertices
Binary Tree
21

29. EVALUATION
Number
of
Ver@ces
300 PCs
Multicore
Tree with 127
average node
degree
22

30. Conclusion
Vertex-­‐Centric
Approach
Computa#on
over
SuperSteps
Usability
and
Scalability
Fault
Tolerance
with
checkpoints
Performance
-­‐
almost
linear
to
the
size
of
the
graph
23