FluxGraph: a time-machine for your graphs
Upcoming SlideShare
Loading in...5
×
 

FluxGraph: a time-machine for your graphs

on

  • 3,803 Views

 

Statistiken

Views

Gesamtviews
3,803
Views auf SlideShare
3,729
Views einbetten
74

Actions

Gefällt mir
5
Downloads
28
Kommentare
0

4 Einbettungen 74

https://twitter.com 63
http://www.linkedin.com 7
https://si0.twimg.com 3
http://twitter.com 1

Zugänglichkeit

Kategorien

Details hochladen

Uploaded via as Adobe PDF

Benutzerrechte

© Alle Rechte vorbehalten

Report content

Als unangemessen gemeldet Als unangemessen melden
Als unangemessen melden

Wählen Sie Ihren Grund, warum Sie diese Präsentation als unangemessen melden.

Löschen
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Ihre Nachricht erscheint hier
    Processing...
Kommentar posten
Kommentar bearbeiten

FluxGraph: a time-machine for your graphs FluxGraph: a time-machine for your graphs Presentation Transcript

  • FluxGraph: A time-machine for your graphs Davy Suvee Michel Van Speybroeck Janssen Pharmaceutica
  • about me who am i ... ➡ working as an it lead / software architect @ janssen pharmaceutica • dealing with big scientific data sets • hands-on expertise in big data and NoSQL technologies ➡ founder of datablend • provide big data and NoSQL consultancy Davy Suvee • share practical knowledge and big data use cases via blog @DSUVEE
  • graphs and time ...➡ graphs are continuously changing ...
  • graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 31. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  • graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 3 copy and relink semantics ๏ graph size ๏ object identity ๏ mixing data-model and time-model1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  • FluxGraph ...➡ towards a time-aware graph ...
  • FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic
  • FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic➡ make FluxGraph fully time-aware ★ travel your graph through time ★ time-scoped iteration of vertices and edges ★ temporal graph comparison
  • travel through timeFluxGraph fg = new FluxGraph();
  • travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”);
  • travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”); PeterVertex peter = ...
  • travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”); PeterVertex peter = ...Vertex michael = ... Michael
  • travel through timeFluxGraph fg = new FluxGraph(); Davy kn owVertex davy = fg.addVertex(); sdavy.setProperty(“name”,”Davy”); PeterVertex peter = ...Vertex michael = ...Edge e1 = Michael fg.addEdge(davy, peter,“knows”);
  • travel through time DavyDate checkpoint = new Date(); kn ow s Peter Michael
  • travel through time DavidDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter Michael
  • travel through time DavidDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter knEdge e2 = ow fg.addEdge(davy, michael,“knows”); s Michael
  • travel through timetime kn Davy ow s Peter Michael
  • travel through timetime kn Davy ow s checkpoint Peter Michael
  • travel through timetime kn Davy ow David Davy s kn ow checkpoint s Peter Peter kn ow s Michael Michael
  • travel through timetime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  • travel through time by defaulttime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  • travel through timetime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael fg.setCheckpointTime(checkpoint);
  • time-scoped iteration t1 Davy
  • time-scoped iteration t1 t2 change Davy Davy’
  • time-scoped iteration t1 t2 t3 change change Davy Davy’ Davy’’
  • time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’
  • time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’ ➡ how to find the version of the vertex you are interested in?
  • time-scoped iteration t1 t2 t3 tcurrrent Davy Davy’ Davy’’ Davy’’’
  • time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous
  • time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previousVertex previousDavy = davy.getPreviousVersion();
  • time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion();Iterable<Vertex> allDavy = davy.getNextVersions();
  • time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
  • time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();Iterable<Vertex> selDavy = davy.getPreviousVersions(filter); Interval valid = davy.getTimerInterval();
  • time-scoped iteration➡ When does an element change?
  • time-scoped iteration➡ When does an element change?➡ vertex: ★ setting or removing a property ★ add or remove it from an edge ★ being removed
  • time-scoped iteration➡ When does an element change?➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed
  • time-scoped iteration➡ When does an element change?➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed➡ ... and each element is time-scoped!
  • temporal graph comparisonDavidDavy Davy kn kn ow ow s s Peter what changed? Peterkn ow s Michael Michael current checkpoint
  • temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!
  • temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!
  • temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph! David difference ( , )= kn ow s
  • FluxGraph ...➡ available on github http://github.com/datablend/fluxgraph
  • use case: longitudinal patient data t1 t2 t3 t4 t5 smoking smoking deathpatient patient patient patient patient cancer cancer
  • use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
  • use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)➡ example analysis: ★ if a male patient is no longer smoking in 2005 ★ what are the chances of getting lung cancer in 2010, comparing patients that smoked before 2005 patients that never smoked
  • use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
  • use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()
  • use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()while (males.hasNext()) { Vertex p2005 = males.next(); boolean smoking2005 = p2005.getEdges(OUT,"smokingStatus").iterator().hasNext();}
  • use case: longitudinal patient data➡ which patients were smoking before 2005?boolean smokingBefore2005 = ((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() { public TimeAwareElement filter(TimeAwareVertex element) { return element.getEdges(OUT, "smokingStatus").iterator().hasNext() ? element : null; } }).iterator().hasNext();
  • use case: longitudinal patient data➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());
  • use case: longitudinal patient data➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());➡ extract the patients that have an edge to the cancer node
  • gephi plugin for fluxgraph 2010
  • gephi plugin for fluxgraph 2001
  • gephi plugin for blueprints! 1 ➡ available on github http://github.com/datablend/gephi-blueprints-plugin ➡ Support for neo4j, orientdb, dex, rexter, ...1. Kudos to Timmy Storms (@timmystorms)
  • Questions?