KONECT Cloud – Large Scale Network Mining in the Cloud
•
0 gefällt mir•483 views
In the Winter 2011/2012 run at the Future SOC Lab, we used the KONECT framework (Koblenz Network Collection) to compute ten different network statistics on a large collection of downsampled versions of a large network dataset, with the goal of determining whether sampling of a large network can be used to reduce the computational effort needed to compute a network statistic. Preliminary results show that this is indeed the case.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie KONECT Cloud – Large Scale Network Mining in the Cloud
Ähnlich wie KONECT Cloud – Large Scale Network Mining in the Cloud (6)
Mehr von Jérôme KUNEGIS
Mehr von Jérôme KUNEGIS (19)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
KONECT Cloud – Large Scale Network Mining in the Cloud
- 1. KONECT Cloud Large Scale Network Mining in the Cloud Jérôme Kunegis Future SOC Lab Day, 18.04.2012 1
- 2. Networks are Everywhere ip r sh tho Au ip dsh Fr ien t Trus n tio n i ca mu e Co m re nc c n r c tio c cu ter a Co-o In
- 3. Social Networks friend
- 4. Trust Networks tru st
- 5. Friend/Enemy Network en d em en y fri
- 6. Interaction Network listen
- 7. KONECT – Koblenz Network Collection 148 network datasets 26 are undirected 38 are directed 84 are bipartite 59 have unweighted edges 77 allow multiple edges 04 have signed edges 08 have ratings as edges 78 have edge arrival times konect.uni-koblenz.de
- 8. Largest Network Directed “who follows who” network 0 041 652 230 users 1 468 365 182 edges konect.uni-koblenz.de/networks/twitter
- 9. 148 Network Datasets authorship communication co-occurrence features folksonomy interaction physical ratings reference semantic social trust
- 10. What We Computed Connected components Network diameter ← at Future SOC Lab Clustering coefficients Degree distributions Spectral distribution Eigenvector centrality Graph drawing Temporal Analysis Link prediction
- 12. 90 Percentile Effective Diameter 5
- 13. 90 Percentile Effective Diameter 3
- 14. 90 Percentile Effective Diameter 3.75
- 15. Computing the Effective Diameter for each node i { |V| count hops needed to reach 90% |E| } Total runtime: |E| × |V|
- 16. Graph Sampling Keep X% of edges
- 17. Computation × 1 000 vertices (sampled) × 120 840 391 edges × 20 sample sizes (5%, 10%, …, 100%) × 50 random samplings Evaluation on single machine: 1 TiB memory 64 cores Matlab 64 bit
- 18. Results
- 19. Thank You! Dr. Jérôme Kunegis konect.uni-koblenz.de kunegis@uni-koblenz.de west.uni-koblenz.de