FluxGraph: a time-machine for your graphs

FluxGraph: A time-machine for your graphs Davy Suvee Michel Van Speybroeck Janssen Pharmaceutica

about me who am i ... ➡ working as an it lead / software architect @ janssen pharmaceutica • dealing with big scientific data sets • hands-on expertise in big data and NoSQL technologies ➡ founder of datablend • provide big data and NoSQL consultancy Davy Suvee • share practical knowledge and big data use cases via blog @DSUVEE

graphs and time ...➡ graphs are continuously changing ...

graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 31. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html

graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 3 copy and relink semantics ๏ graph size ๏ object identity ๏ mixing data-model and time-model1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html

FluxGraph ...➡ towards a time-aware graph ...

FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic

FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic➡ make FluxGraph fully time-aware ★ travel your graph through time ★ time-scoped iteration of vertices and edges ★ temporal graph comparison

travel through timeFluxGraph fg = new FluxGraph();

travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”);

travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”); PeterVertex peter = ...

travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”); PeterVertex peter = ...Vertex michael = ... Michael

travel through timeFluxGraph fg = new FluxGraph(); Davy kn owVertex davy = fg.addVertex(); sdavy.setProperty(“name”,”Davy”); PeterVertex peter = ...Vertex michael = ...Edge e1 = Michael fg.addEdge(davy, peter,“knows”);

travel through time DavyDate checkpoint = new Date(); kn ow s Peter Michael

travel through time DavidDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter Michael

travel through time DavidDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter knEdge e2 = ow fg.addEdge(davy, michael,“knows”); s Michael

travel through timetime kn Davy ow s Peter Michael

travel through timetime kn Davy ow s checkpoint Peter Michael

travel through timetime kn Davy ow David Davy s kn ow checkpoint s Peter Peter kn ow s Michael Michael

travel through timetime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael

travel through time by defaulttime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael

travel through timetime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael fg.setCheckpointTime(checkpoint);

time-scoped iteration t1 Davy

time-scoped iteration t1 t2 change Davy Davy’

time-scoped iteration t1 t2 t3 change change Davy Davy’ Davy’’

time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’

time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’ ➡ how to find the version of the vertex you are interested in?

time-scoped iteration t1 t2 t3 tcurrrent Davy Davy’ Davy’’ Davy’’’

time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous

time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previousVertex previousDavy = davy.getPreviousVersion();

time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion();Iterable<Vertex> allDavy = davy.getNextVersions();

time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);

time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();Iterable<Vertex> selDavy = davy.getPreviousVersions(filter); Interval valid = davy.getTimerInterval();

time-scoped iteration➡ When does an element change?

time-scoped iteration➡ When does an element change?➡ vertex: ★ setting or removing a property ★ add or remove it from an edge ★ being removed

time-scoped iteration➡ When does an element change?➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed

time-scoped iteration➡ When does an element change?➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed➡ ... and each element is time-scoped!

temporal graph comparisonDavidDavy Davy kn kn ow ow s s Peter what changed? Peterkn ow s Michael Michael current checkpoint

temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!

temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph! David difference ( , )= kn ow s

FluxGraph ...➡ available on github http://github.com/datablend/fluxgraph

use case: longitudinal patient data t1 t2 t3 t4 t5 smoking smoking deathpatient patient patient patient patient cancer cancer

use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)

use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)➡ example analysis: ★ if a male patient is no longer smoking in 2005 ★ what are the chances of getting lung cancer in 2010, comparing patients that smoked before 2005 patients that never smoked

use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());

use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()

use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()while (males.hasNext()) { Vertex p2005 = males.next(); boolean smoking2005 = p2005.getEdges(OUT,"smokingStatus").iterator().hasNext();}

use case: longitudinal patient data➡ which patients were smoking before 2005?boolean smokingBefore2005 = ((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() { public TimeAwareElement filter(TimeAwareVertex element) { return element.getEdges(OUT, "smokingStatus").iterator().hasNext() ? element : null; } }).iterator().hasNext();

use case: longitudinal patient data➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());

use case: longitudinal patient data➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());➡ extract the patients that have an edge to the cancer node

gephi plugin for fluxgraph 2010

gephi plugin for fluxgraph 2001

gephi plugin for blueprints! 1 ➡ available on github http://github.com/datablend/gephi-blueprints-plugin ➡ Support for neo4j, orientdb, dex, rexter, ...1. Kudos to Timmy Storms (@timmystorms)

Questions?

https://twitter.com	45
https://si0.twimg.com	3
http://twitter.com	1

FluxGraph: a time-machine for your graphs

von datablend am 07. November 2012

Zugänglichkeit

Kategorien

Details hochladen

Benutzerrechte

3 Einbettungen 49

Statistiken

FluxGraph: a time-machine for your graphs Presentation Transcript