FluxGraph: A time-machine for your graphs                   Davy Suvee              Michel Van Speybroeck                J...
about me                 who am i ...                 ➡ working as an it lead / software architect @ janssen pharmaceutica...
graphs and time ...➡ graphs are continuously changing ...
graphs and time ...       ➡ graphs are continuously changing ...       ➡ graphs and time ...          ★ neo-versioning by ...
graphs and time ...       ➡ graphs are continuously changing ...       ➡ graphs and time ...          ★ neo-versioning by ...
FluxGraph ...➡ towards a time-aware graph ...
FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic
FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic➡ make FluxGraph f...
travel through timeFluxGraph fg = new FluxGraph();
travel through timeFluxGraph fg = new FluxGraph();                                   DavyVertex davy = fg.addVertex();davy...
travel through timeFluxGraph fg = new FluxGraph();                                   DavyVertex davy = fg.addVertex();davy...
travel through timeFluxGraph fg = new FluxGraph();                                   DavyVertex davy = fg.addVertex();davy...
travel through timeFluxGraph fg = new FluxGraph();                                     Davy                               ...
travel through time                                DavyDate checkpoint = new Date();                                      ...
travel through time                                    DavidDate checkpoint = new Date();                                 ...
travel through time                                       DavidDate checkpoint = new Date();                              ...
travel through timetime                        kn       Davy                  ow                                  s       ...
travel through timetime                        kn       Davy                  ow                                  s       ...
travel through timetime                        kn       Davy                  ow                            David         ...
travel through timetime                        kn       Davy                  ow                            David         ...
travel through time                                           by defaulttime                        kn       Davy         ...
travel through timetime                         kn       Davy                   ow                            David       ...
time-scoped iteration       t1     Davy
time-scoped iteration       t1               t2            change     Davy            Davy’
time-scoped iteration       t1               t2               t3            change           change     Davy            Da...
time-scoped iteration       t1               t2               t3                 tcurrrent            change           cha...
time-scoped iteration         t1               t2               t3                 tcurrrent              change          ...
time-scoped iteration      t1        t2         t3        tcurrrent    Davy     Davy’      Davy’’   Davy’’’
time-scoped iteration      t1                 t2                 t3                   tcurrrent             next          ...
time-scoped iteration       t1                 t2                 t3                   tcurrrent              next        ...
time-scoped iteration         t1                 t2                 t3                   tcurrrent                next    ...
time-scoped iteration            t1                 t2                 t3                   tcurrrent                   ne...
time-scoped iteration            t1                 t2                 t3                   tcurrrent                   ne...
time-scoped iteration➡ When does an element change?
time-scoped iteration➡ When does an element change?➡ vertex:   ★ setting or removing a property   ★ add or remove it from ...
time-scoped iteration➡ When does an element change?➡ vertex:                             ➡ edge:   ★ setting or removing a...
time-scoped iteration➡ When does an element change?➡ vertex:                                ➡ edge:   ★ setting or removin...
temporal graph comparisonDavidDavy                                          Davy                                          ...
temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!
temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!
temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!                   David  di...
FluxGraph ...➡ available on github      http://github.com/datablend/fluxgraph
use case: longitudinal patient data    t1        t2        t3        t4        t5          smoking   smoking             d...
use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)➡ example a...
use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate(...
use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate(...
use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate(...
use case: longitudinal patient data➡ which patients were smoking before 2005?boolean smokingBefore2005 =  ((FluxVertex)p20...
use case: longitudinal patient data➡ which patients have cancer in 2010                                       working set ...
use case: longitudinal patient data➡ which patients have cancer in 2010                                       working set ...
gephi plugin for fluxgraph   2010
gephi plugin for fluxgraph   2001
gephi plugin for blueprints!                                          1     ➡ available on github      http://github.com/d...
Questions?
FluxGraph: a time-machine for your graphs
FluxGraph: a time-machine for your graphs
Nächste SlideShare
Wird geladen in ...5
×

FluxGraph: a time-machine for your graphs

4,295

Published on

Published in: Technologie

FluxGraph: a time-machine for your graphs

  1. 1. FluxGraph: A time-machine for your graphs Davy Suvee Michel Van Speybroeck Janssen Pharmaceutica
  2. 2. about me who am i ... ➡ working as an it lead / software architect @ janssen pharmaceutica • dealing with big scientific data sets • hands-on expertise in big data and NoSQL technologies ➡ founder of datablend • provide big data and NoSQL consultancy Davy Suvee • share practical knowledge and big data use cases via blog @DSUVEE
  3. 3. graphs and time ...➡ graphs are continuously changing ...
  4. 4. graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 31. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  5. 5. graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 3 copy and relink semantics ๏ graph size ๏ object identity ๏ mixing data-model and time-model1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  6. 6. FluxGraph ...➡ towards a time-aware graph ...
  7. 7. FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic
  8. 8. FluxGraph ...➡ towards a time-aware graph ...➡ implement a blueprints-compatible graph on top of Datomic➡ make FluxGraph fully time-aware ★ travel your graph through time ★ time-scoped iteration of vertices and edges ★ temporal graph comparison
  9. 9. travel through timeFluxGraph fg = new FluxGraph();
  10. 10. travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”);
  11. 11. travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”); PeterVertex peter = ...
  12. 12. travel through timeFluxGraph fg = new FluxGraph(); DavyVertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”); PeterVertex peter = ...Vertex michael = ... Michael
  13. 13. travel through timeFluxGraph fg = new FluxGraph(); Davy kn owVertex davy = fg.addVertex(); sdavy.setProperty(“name”,”Davy”); PeterVertex peter = ...Vertex michael = ...Edge e1 = Michael fg.addEdge(davy, peter,“knows”);
  14. 14. travel through time DavyDate checkpoint = new Date(); kn ow s Peter Michael
  15. 15. travel through time DavidDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter Michael
  16. 16. travel through time DavidDate checkpoint = new Date(); kn ow sdavy.setProperty(“name”,”David”); Peter knEdge e2 = ow fg.addEdge(davy, michael,“knows”); s Michael
  17. 17. travel through timetime kn Davy ow s Peter Michael
  18. 18. travel through timetime kn Davy ow s checkpoint Peter Michael
  19. 19. travel through timetime kn Davy ow David Davy s kn ow checkpoint s Peter Peter kn ow s Michael Michael
  20. 20. travel through timetime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  21. 21. travel through time by defaulttime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  22. 22. travel through timetime kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael fg.setCheckpointTime(checkpoint);
  23. 23. time-scoped iteration t1 Davy
  24. 24. time-scoped iteration t1 t2 change Davy Davy’
  25. 25. time-scoped iteration t1 t2 t3 change change Davy Davy’ Davy’’
  26. 26. time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’
  27. 27. time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’ ➡ how to find the version of the vertex you are interested in?
  28. 28. time-scoped iteration t1 t2 t3 tcurrrent Davy Davy’ Davy’’ Davy’’’
  29. 29. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous
  30. 30. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previousVertex previousDavy = davy.getPreviousVersion();
  31. 31. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion();Iterable<Vertex> allDavy = davy.getNextVersions();
  32. 32. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
  33. 33. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();Iterable<Vertex> selDavy = davy.getPreviousVersions(filter); Interval valid = davy.getTimerInterval();
  34. 34. time-scoped iteration➡ When does an element change?
  35. 35. time-scoped iteration➡ When does an element change?➡ vertex: ★ setting or removing a property ★ add or remove it from an edge ★ being removed
  36. 36. time-scoped iteration➡ When does an element change?➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed
  37. 37. time-scoped iteration➡ When does an element change?➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed➡ ... and each element is time-scoped!
  38. 38. temporal graph comparisonDavidDavy Davy kn kn ow ow s s Peter what changed? Peterkn ow s Michael Michael current checkpoint
  39. 39. temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!
  40. 40. temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph!
  41. 41. temporal graph comparison➡ difference (A , B) = union (A , B) - B➡ ... as a (immutable) graph! David difference ( , )= kn ow s
  42. 42. FluxGraph ...➡ available on github http://github.com/datablend/fluxgraph
  43. 43. use case: longitudinal patient data t1 t2 t3 t4 t5 smoking smoking deathpatient patient patient patient patient cancer cancer
  44. 44. use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
  45. 45. use case: longitudinal patient data➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)➡ example analysis: ★ if a male patient is no longer smoking in 2005 ★ what are the chances of getting lung cancer in 2010, comparing patients that smoked before 2005 patients that never smoked
  46. 46. use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
  47. 47. use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()
  48. 48. use case: longitudinal patient data➡ get all male non-smokers in 2005fg.setCheckpointTime(new DateTime(2005,12,31).toDate());Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()while (males.hasNext()) { Vertex p2005 = males.next(); boolean smoking2005 = p2005.getEdges(OUT,"smokingStatus").iterator().hasNext();}
  49. 49. use case: longitudinal patient data➡ which patients were smoking before 2005?boolean smokingBefore2005 = ((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() { public TimeAwareElement filter(TimeAwareVertex element) { return element.getEdges(OUT, "smokingStatus").iterator().hasNext() ? element : null; } }).iterator().hasNext();
  50. 50. use case: longitudinal patient data➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());
  51. 51. use case: longitudinal patient data➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());➡ extract the patients that have an edge to the cancer node
  52. 52. gephi plugin for fluxgraph 2010
  53. 53. gephi plugin for fluxgraph 2001
  54. 54. gephi plugin for blueprints! 1 ➡ available on github http://github.com/datablend/gephi-blueprints-plugin ➡ Support for neo4j, orientdb, dex, rexter, ...1. Kudos to Timmy Storms (@timmystorms)
  55. 55. Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×