Budapest Neo4j Meetup, 13 February 2018 @ Liferay Hungary

1. Learning Timed Automata with Cypher
Gábor Szárnyas, Rebeka Farkas, Márton Elekes, Anna Gujgiczer
Budapest Neo4j Meetup 13 February 2018

2. Sicco Verwer, Mathijs de Weerdt, Cees Witteveen:
An algorithm for learning real-time automata.
Benelearn 2007

3. Automaton learning

4. STATE MACHINE
Off
Stop
Continue
Prepare
Go
switchPhase
switchPhase
switch
Phase
onOff
onOff
onOff onOff
onOff
switch
Phase
switch
Phase

5. STATE MACHINE
Off
Stop
Continue
Prepare
Go
switchPhase
switchPhase
switch
Phase
onOff
onOff
onOff onOff
onOff
switch
Phase
switch
Phase

6. AUTOMATON
 States: accepting/rejecting
 Run: (finite) event sequence
 Accepted run:
ends in an accepting state
 Modelled behaviour:
accepted runsOff
Stop
Continue
Prepare
Go
switchPhase
switchPhase
switch
Phase
onOff
onOff
onOff onOff
onOff
switch
Phase
switch
Phase

7. AUTOMATON
 States: accepting/rejecting
 Run: (finite) event sequence
 Accepted run:
ends in an accepting state
 Modelled behaviour:
accepted runsOff
Stop
Continue
Prepare
Go
switchPhase
switchPhase
switch
Phase
onOff
onOff
onOff onOff
onOff
switch
Phase
+
- -
- -
switch
Phase

8. TIMING
 Time spent in a state
 Guard condition: an interval
Off
Stop
Continue
Prepare
Go
switchPhase
switchPhase
switch
Phase
onOff
onOff
onOff onOff
onOff
switch
Phase
+
- -
- -
switch
Phase

9. TIMING
 Time spent in a state
 Guard condition: an interval
Off
Stop
Continue
Prepare
Go
switchPhase
switchPhase
switch
Phase
onOff
onOff
onOff onOff
onOff
switch
Phase
+
- -
- -
[3,5]
[30,35]
[1,2]
switch
Phase
[30,35]
[0,∞]
[0,∞]
[0,∞]
[0,∞]
[0,∞]
[0,∞]

10. AUTOMATON LEARNING
 System: unknown internal implementation (black box)
 Goal: determine the internal operation of the system
(~reverse engineering)
 Approach: observation  runs  automaton
 Application: monitor synthesis, testing

11. AUTOMATON LEARNING: THE BASICS
+
-
-
+
a,1
a,5
+
a,3
a,5 a,7
a,2
+ -
a,[1,2]
+
a,[1,5]
+
a,[3,5]
-
a,[1,5]
+ -
a, [1,2]
a,[1,2]
a, [3,5] a, [3,5]

12. AUTOMATON LEARNING: AN ALGORITHM
 Repeating three operations
o Split inconsistent states
+ -
a,[1,2]
+
a,[1,5]
+
a,[3,5]
-
a,[1,5]
+ +/-
a,[1,5]
+/-
a,[1,5]
+
-
-
+
a,1
a,5
+
a,3
a,5 a,7
a,2

13. AUTOMATON LEARNING: AN ALGORITHM
 Repeating three operations
o Split inconsistent states
o Merge similar states
-
-
b
+
+
a
a
-b
+
+
a
a
-b +
a

14. AUTOMATON LEARNING: AN ALGORITHM
 Repeating three operations
o Split inconsistent states
o Merge similar states
o Color to finalise a state
 Selecting an operation
o Using metrics based on the result
of the operation
 Operations have to be executed
Select operation
Split Merge Color

15. Implementation

16. INTERESTING QUERIES
 How to select a “longest path”
o i.e. a maximal path that cannot be extended further
o Use WHERE NOT
 Merge
o Transitive closure for the states to-be-merged
 Split: categorisation/copying subtrees
o Merge might result in multiple origNext edges
o Which one to use for categorisation?

17. WHERE NOT «PATTERN»
 Finding the end of a path
WHERE NOT (v)-[:Next]->()
:Next :Next
… …v :Next

18. WHERE NOT «PATTERN»
 Finding the end of a path
WHERE NOT (v)-[:Next]->()
:Next :Next
… …v :Next

19. WHERE NOT «PATTERN»
 Finding the end of a path
 Finding the beginning of a path
WHERE NOT (v)-[:Next]->()
WHERE NOT ()-[:Next]->(root)
:Next :Next
… …:Next
root

20. WHERE NOT «PATTERN»
 Finding the end of a path
 Finding the beginning of a path
WHERE NOT (v)-[:Next]->()
WHERE NOT ()-[:Next]->(root)
:Next :Next
… …:Next
root

21. WHERE NOT «PATTERN»
 Finding the end of a path
 Finding the beginning of a path
 These two can be used to select the complete path
WHERE NOT (v)-[:Next]->()
WHERE NOT ()-[:Next]->(root)
:Next :Next
… …:Next
root

22. MERGE
:indexPair :indexPair
-
-:Next
:Next
+
+
:Next
:Next
…
…
:indexPair
-:Next
+:Next
…
 Select states to merge
 Calculate metrics
 Filter inconsistencies
 Return result

23. MERGE +
-
:indexPair
:indexPair
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
:indexPair
:indexPair
+
-:Next
:Next

24. MERGE +
-
:indexPair
:indexPair
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
:indexPair
:indexPair
+
-:Next
:Next
while (driver.run(createIndexPairs, mergeMap))

25. MERGE +
-
:indexPair
:indexPair
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
:indexPair
:indexPair
+
-:Next
:Next
MATCH (s1:IndexedMerge)-[:IndexPair*..]-(s2:IndexedMerge),
s1p = (s1)-[:Next*0..]->(s12:State), s2p = (s2)-[:Next*0..]->(s22:State)
WHERE id(s1) > id(s2) AND length(s1p) = length(s2p)
AND NOT (s12)-[:IndexPair*0..]-(s22)
WITH s12, s22, s1p, s2p,[s1r IN rels(s1p) | s1r.symbol] AS s1ss, [s2r IN rels(s2p) | s2r.symbol] AS s2ss,
[s1r IN rels(s1p) | s1r.Tmin] AS s1mins, [s2r IN rels(s2p) | s2r.Tmin ] AS s2mins,
[s1r IN rels(s1p) | s1r.Tmax] AS s1maxs, [s2r IN rels(s2p) | s2r.Tmax ] AS s2maxs
WHERE s1ss = s2ss AND s1mins = s2mins AND s1maxs = s2maxs
WITH collect([s12, s22]) as pairs
MATCH ()-[ip:IndexPair]->()
WITH max(ip.index) + 1 as nextIndex, pairs
UNWIND range(0, length(pairs)-1) as idx
WITH pairs[idx][0] as s12, pairs[idx][1] as s22, idx + nextIndex as edgeIndex
CREATE (s12)-[:IndexPair {index: edgeIndex}]->(s22)
SET s12:IndexedMerge, s22:IndexedMerge
while (driver.run(createIndexPairs, mergeMap))

26. MERGE +
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Next
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path

27. MERGE
MATCH (s1:IndexedMerge)-[:IndexPair*]-(s2:IndexedMerge),
s1p = (s1)-[:Next*0..]->(s12:State),
s2p = (s2)-[:Next*0..]->(s22:State)
WHERE id(s1) > id(s2) AND length(s1p) = length(s2p)
AND NOT (s12)-[:IndexPair*0..]-(s22)
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Next
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path

28. MERGE
MATCH (s1:IndexedMerge)-[:IndexPair*]-(s2:IndexedMerge),
s1p = (s1)-[:Next*0..]->(s12:State),
s2p = (s2)-[:Next*0..]->(s22:State)
WHERE id(s1) > id(s2) AND length(s1p) = length(s2p)
AND NOT (s12)-[:IndexPair*0..]-(s22)
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Next
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
Find critical state pairs

29. MERGE
MATCH (s1:IndexedMerge)-[:IndexPair*]-(s2:IndexedMerge),
s1p = (s1)-[:Next*0..]->(s12:State),
s2p = (s2)-[:Next*0..]->(s22:State)
WHERE id(s1) > id(s2) AND length(s1p) = length(s2p)
AND NOT (s12)-[:IndexPair*0..]-(s22)
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Next
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
s1 s1
s2 s2
Find critical state pairs

30. MERGE
MATCH (s1:IndexedMerge)-[:IndexPair*]-(s2:IndexedMerge),
s1p = (s1)-[:Next*0..]->(s12:State),
s2p = (s2)-[:Next*0..]->(s22:State)
WHERE id(s1) > id(s2) AND length(s1p) = length(s2p)
AND NOT (s12)-[:IndexPair*0..]-(s22)
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Next
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
s1 s1
s2 s2
WITH s12, s22, s1p, s2p,
[s1r IN rels(s1p) | s1r.symbol] AS s1ss, [s2r IN rels(s2p) | s2r.symbol] AS s2ss,
[s1r IN rels(s1p) | s1r.Tmin] AS s1mins, [s2r IN rels(s2p) | s2r.Tmin ] AS s2mins,
[s1r IN rels(s1p) | s1r.Tmax] AS s1maxs, [s2r IN rels(s2p) | s2r.Tmax ] AS s2maxs
WHERE s1ss = s2ss AND s1mins = s2mins AND s1maxs = s2maxs
...
Find critical state pairs

31. MERGE
MATCH (s1:IndexedMerge)-[:IndexPair*]-(s2:IndexedMerge),
s1p = (s1)-[:Next*0..]->(s12:State),
s2p = (s2)-[:Next*0..]->(s22:State)
WHERE id(s1) > id(s2) AND length(s1p) = length(s2p)
AND NOT (s12)-[:IndexPair*0..]-(s22)
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Next
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
s1 s1
s2 s2
WITH s12, s22, s1p, s2p,
[s1r IN rels(s1p) | s1r.symbol] AS s1ss, [s2r IN rels(s2p) | s2r.symbol] AS s2ss,
[s1r IN rels(s1p) | s1r.Tmin] AS s1mins, [s2r IN rels(s2p) | s2r.Tmin ] AS s2mins,
[s1r IN rels(s1p) | s1r.Tmax] AS s1maxs, [s2r IN rels(s2p) | s2r.Tmax ] AS s2maxs
WHERE s1ss = s2ss AND s1mins = s2mins AND s1maxs = s2maxs
...
• Paths of same length,
• Event sequences of same
lengths, etc.
Find critical state pairs

32. MERGE
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Nexts1 s1
s2 s2

33. MERGE
CREATE (s12)-[:IndexPair {index: edgeIndex}]->(s22)
SET s12:IndexedMerge, s22:IndexedMerge
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Nexts1 s1
s2 s2

34. MERGE
CREATE (s12)-[:IndexPair {index: edgeIndex}]->(s22)
SET s12:IndexedMerge, s22:IndexedMerge
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Nexts1 s1
s2 s2
Mark newly found
index pairs

35. MERGE
CREATE (s12)-[:IndexPair {index: edgeIndex}]->(s22)
SET s12:IndexedMerge, s22:IndexedMerge
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Nexts1 s1
s2 s2
:indexPair :indexPair
Mark newly found
index pairs

36. MERGE
CREATE (s12)-[:IndexPair {index: edgeIndex}]->(s22)
SET s12:IndexedMerge, s22:IndexedMerge
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Nexts1 s1
s2 s2
:indexPair :indexPair
Mark newly found
index pairs
while (driver.run(createIndexPairs, mergeMap))

37. MERGE
CREATE (s12)-[:IndexPair {index: edgeIndex}]->(s22)
SET s12:IndexedMerge, s22:IndexedMerge
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Nexts1 s1
s2 s2
:indexPair :indexPair
Mark newly found
index pairs
while (driver.run(createIndexPairs, mergeMap))
Repeatedly called from
Java code until fixpoint

38. MERGE
CREATE (s12)-[:IndexPair {index: edgeIndex}]->(s22)
SET s12:IndexedMerge, s22:IndexedMerge
 Problem:
o Inconsistencies can be transitive
 Solution:
o Variable length path
+
-
:indexPair
:indexPair
:indexPair
:indexPair
+
-:Next
:Nexts1 s1
s2 s2
:indexPair :indexPair
Mark newly found
index pairs
while (driver.run(createIndexPairs, mergeMap))
Repeatedly called from
Java code until fixpoint

39. SPLIT
:Origin
:Origin
:Origin
:Origin
[1,6]
1
6
 Splits an edge
 Based on timestamp: t
o Original transition
 The two endpoints…
o Smaller: < t
o Larger: ≥ t
2
:Origin :Origin
+
+
-
+/-

40. SPLIT
:Origin
:Origin
:Origin
:Origin
[1,6]
1
6
2
:Origin :Origin
 Splits an edge
 Based on timestamp: 5
o Original transition
 The two endpoints…
o Smaller: < t
o Larger: ≥ t
+
+
-
+/-

41. SPLIT
 Splits an edge
 Based on timestamp: 5
o Original transition
 The two endpoints…
o Smaller: < t
o Larger: ≥ t
 Results
[1,4]
[5,6]
+
-

42. SPLIT
 Problem
o Node a is a result of an earlier
merge operation
o The subtree regenerated from
node b should belong to the…
• Larger subtree because of 6
• Smaller subtree because of 1
:Origin
:Origin :Origin
[1,6]
1
6
b
6
:Origin :Origin
…
a

43.  Solution
o “Shortest” path to node a
o Based on the last number: 6
SPLIT
:Origin
:Origin :Origin
[1,6]
1
6
b
6
:Origin :Origin
…
a

44. SPLIT
MATCH (r:Red)-[n:Next]->(b:Blue)
WITH r, b, n.Tmax as Tmax, n.Tmin as Tmin, n.symbol as symbol
UNWIND range(Tmin, Tmax-1) AS t
MATCH (b)-[:Next*0..]->(s1:State)-[:Origin]->(so1:OrigState), trace1=(so1)<-[OrigNext*0..]-(redOrigNext:OrigState), (redOrigNext)<-[no1:OrigNext]-(redOrig:OrigState)<-[:Origin]-(r)
WHERE none(x IN nodes(trace1) WHERE (x)<-[:Origin]-(r))
WITH r, b, t, Tmin, Tmax, symbol, s1, no1, so1
MATCH (b)-[:Next*0..]->(s2:State)-[:Origin]->(so2:OrigState), trace2=(so2)<-[OrigNext*0..]-(redOrigNext:OrigState), (redOrigNext)<-[no2:OrigNext]-(redOrig:OrigState)<-[:Origin]-(r)
WHERE none(x IN nodes(trace2) WHERE (x)<-[:Origin]-(r))
WITH t, r, b, s1, s2, toInteger(no1.time)>t as cat1, toInteger(no2.time)>t as cat2, Tmin, Tmax, symbol, so1, so2
WHERE s1 = s2
AND cat1 <> cat2
AND id(so1) < id(so2)
WITH r, b, t, Tmin, Tmax, symbol,
[coalesce(so1.accepting, false), coalesce(so1.rejecting, false)] AS so1ar,
[coalesce(so2.accepting, false), coalesce(so2.rejecting, false)] AS so2ar
WITH
r, b, t, Tmin, Tmax, symbol,
CASE
WHEN so1ar[0] <> so2ar[0] AND so1ar[1] <> so2ar[1] THEN 1
WHEN so1ar = so2ar AND (so1ar = [false, true] OR so1ar = [true, false]) THEN -1 // there is at least one true
ELSE 0
END AS score
WITH r, b, t, sum(score) AS metric, Tmin, Tmax, symbol
ORDER BY metric DESC
LIMIT 1
WITH r, b, t, metric, Tmin, Tmax, symbol
MATCH (b)-[:Next*0..]->(s1:State)-[:Origin]->(so1:OrigState), trace1=(so1)<-[OrigNext*0..]-(redOrigNext:OrigState), (redOrigNext)<-[no1:OrigNext]-(redOrig:OrigState)<-[:Origin]-(r)
WHERE none(x IN nodes(trace1) WHERE (x)<-[:Origin]-(r)) AND toInteger(no1.time)>t
WITH r, b, t, metric, Tmin, Tmax, symbol, collect(so1) as so1s
MATCH (b)-[:Next*0..]->(s2:State)-[:Origin]->(so2:OrigState), trace2=(so2)<-[OrigNext*0..]-(redOrigNext:OrigState), (redOrigNext)<-[no2:OrigNext]-(redOrig:OrigState)<-[:Origin]-(r)
WHERE none(x IN nodes(trace2) WHERE (x)<-[:Origin]-(r)) AND toInteger(no2.time)<=t
RETURN r, b, t, metric, Tmin, Tmax, symbol, so1s, collect(so2) as so2s

45. SPLIT
:Origin
:Origin :Origin
[1,6]
1
6
6
:Origin :Origin
…
a s2
ao
an
b
 Solution
o “Shortest” path to node a
o Based on the last number: 6
MATCH
(a)-[:Next*0..]->(s2:State)-[:Origin]->(b:OrigState),
trace=(b)<-[OrigNext*0..]-(an:OrigState),
(an)<-[no:OrigNext]-(ao:OrigState)<-[:Origin]-(a)
The split candidate in
the original runs

46. SPLIT
The split candidate in
the original runs
:Origin
:Origin :Origin
[1,6]
1
6
6
:Origin :Origin
…ao
a
an
s2
b
MATCH
(a)-[:Next*0..]->(s2:State)-[:Origin]->(b:OrigState),
trace=(b)<-[OrigNext*0..]-(an:OrigState),
(an)<-[no:OrigNext]-(ao:OrigState)<-[:Origin]-(a)
 Solution
o “Shortest” path to node a
o Based on the last number: 6

47. SPLIT
WHERE none(x IN nodes(trace) WHERE (x)<-[:Origin]-(a))
The shortest possible trail, i.e.
a node, from which node a can
only be reached directly.
 Solution
o “Shortest” path to node a
o Based on the last number: 6
:Origin
:Origin :Origin
[1,6]
1
6
6
:Origin :Origin
…
a s2
ao
b
an
MATCH
(a)-[:Next*0..]->(s2:State)-[:Origin]->(b:OrigState),
trace=(b)<-[OrigNext*0..]-(an:OrigState),
(an)<-[no:OrigNext]-(ao:OrigState)<-[:Origin]-(a)

48. Technical details

49.  Visualisation
ADVANTAGES OF NEO4J

50.  Visualisation
 Flexible data model
o New labels, properties
o No need to define the schema upfront
ADVANTAGES OF NEO4J

51.  Visualisation
 Flexible data model
o New labels, properties
o No need to define the schema upfront
 Cypher: high-level declarative graph query language
ADVANTAGES OF NEO4J

52. WORKFLOW
 Neo4j web browser UI

53. WORKFLOW
MATCH (n)
RETURN n
 Neo4j web browser UI

54. WORKFLOW
MATCH (n)
RETURN n
 Neo4j web browser UI

55. WORKFLOW
 Neo4j web browser UI

56. WORKFLOW
 Neo4j web browser UI

57. WORKFLOW
 Neo4j web browser UI

58. WORKFLOW
 Neo4j web browser UI
We should combine
transformation and
visualisation

59. WORKFLOW
 Neo4j web browser UI
We should combine
transformation and
visualisation

60. WORKFLOW
 Neo4j web browser UI
We should combine
transformation and
visualisation

61. WORKFLOW
 Neo4j web browser UI
We should combine
transformation and
visualisation

62. WORKFLOW
 Neo4j web browser UI
We should combine
transformation and
visualisation

63. CHAINING QUERIES
 Chaining queries to write a complex step of the algorithm
with a single Cypher query
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
RETURN *
node
:Red {prop1: "val1"}
:Red {prop1: "val2"}

64. CHAINING QUERIES
 Chaining queries to write a complex step of the algorithm
with a single Cypher query
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
RETURN *
node
:Red {prop1: "val1"}
:Red {prop1: "val2"}
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
WITH 1 AS dummy
RETURN *
dummy
1
1

65. CHAINING QUERIES
 Chaining queries to write a complex step of the algorithm
with a single Cypher query
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
RETURN *
node
:Red {prop1: "val1"}
:Red {prop1: "val2"}
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
WITH 1 AS dummy
RETURN *
dummy
1
1
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
WITH 1 AS dummy
MATCH (n)
RETURN *
node dummy
:Red {prop1: "val1"} 1
:Red {prop1: "val2"} 1
:Green {value: 13} 1
:Red {prop1: "val1"} 1
:Red {prop1: "val2"} 1
:Green {value: 13} 1

66. CHAINING QUERIES
 Chaining queries to write a complex step of the algorithm
with a single Cypher query
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
RETURN *
node
:Red {prop1: "val1"}
:Red {prop1: "val2"}
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
WITH 1 AS dummy
RETURN *
dummy
1
1
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
WITH 1 AS dummy
MATCH (n)
RETURN *
node dummy
:Red {prop1: "val1"} 1
:Red {prop1: "val2"} 1
:Green {value: 13} 1
:Red {prop1: "val1"} 1
:Red {prop1: "val2"} 1
:Green {value: 13} 1
Cartesian product
 all rows are
displayed twice

67. CHAINING QUERIES
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
RETURN *
node
:Red {prop1: "val1"}
:Red {prop1: "val2"}
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
WITH count(*) as dummy
RETURN *
dummy
2
 Chaining queries to write a complex step of the algorithm
with a single Cypher query

68. CHAINING QUERIES
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
RETURN *
node
:Red {prop1: "val1"}
:Red {prop1: "val2"}
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
WITH count(*) as dummy
RETURN *
dummy
2
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
WITH count(*) as dummy
MATCH (n)
RETURN *
node dummy
:Red {prop1: "val1"} 2
:Red {prop1: "val2"} 2
:Green {value: 13} 2
 Chaining queries to write a complex step of the algorithm
with a single Cypher query

69. CHAINING QUERIES
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
RETURN *
node
:Red {prop1: "val1"}
:Red {prop1: "val2"}
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
WITH count(*) as dummy
RETURN *
dummy
2
MATCH (node:Blue)
REMOVE node:Blue
SET node:Red
WITH count(*) as dummy
MATCH (n)
RETURN *
node dummy
:Red {prop1: "val1"} 2
:Red {prop1: "val2"} 2
:Green {value: 13} 2
Exactly 1 result
 The second query will not
produce unnecessary
duplicates
 Chaining queries to write a complex step of the algorithm
with a single Cypher query

70. RUNNING AND VISUALISING THE ALGORITHM

71. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

72. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

73. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

74. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

75. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

76. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

77. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

78. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

79. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

80. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

81. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

82. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

83. RUNNING AND VISUALISING THE ALGORITHM
...
WITH count(*) as dummy
MATCH (n)
RETURN n

84. EXECUTING QUERIES
Passing information between queries:

85. EXECUTING QUERIES
Passing information between queries:
 Node labels

86. EXECUTING QUERIES
Passing information between queries:
 Node labels
 Nodes with temporary information

87. EXECUTING QUERIES
Passing information between queries:
 Node labels
 Nodes with temporary information
 Passing nodes/edges as parameters
(only supported in embedded mode)

88. EXECUTING QUERIES
Passing information between queries:
 Node labels
 Nodes with temporary information
 Passing nodes/edges as parameters
(only supported in embedded mode)
redNode num
:Red {prop1: "val1"} 1

89. EXECUTING QUERIES
Passing information between queries:
 Node labels
 Nodes with temporary information
 Passing nodes/edges as parameters
(only supported in embedded mode)
redNode num
:Red {prop1: "val1"} 1
Exactly
1 result

90. EXECUTING QUERIES
Passing information between queries:
 Node labels
 Nodes with temporary information
 Passing nodes/edges as parameters
(only supported in embedded mode)
redNode num
:Red {prop1: "val1"} 1
WITH $redNode AS redNode
MATCH (g:Green {num: $num})
CREATE (redNode)-[:Edge]->(g)
Exactly
1 result

91. LOOPS
 Many loop-like problems can be solved using lists:
o List comprehensions: [x IN xs WHERE condition | f(x)]
o Reduce: reduce(acc = "", x IN list | acc + x.prop)

92. LOOPS
 Many loop-like problems can be solved using lists:
o List comprehensions: [x IN xs WHERE condition | f(x)]
o Reduce: reduce(acc = "", x IN list | acc + x.prop)
 However, loops still might be necessary:
o Run queries from Java code
gds.execute(step1Query);
gds.execute(step2Query);

93. LOOPS
 Many loop-like problems can be solved using lists:
o List comprehensions: [x IN xs WHERE condition | f(x)]
o Reduce: reduce(acc = "", x IN list | acc + x.prop)
 However, loops still might be necessary:
o Run queries from Java code
o The loop condition checks the number of rows in the result
while (gds.execute(conditionQuery).hasNext()) {
gds.execute(step1Query);
gds.execute(step2Query);
}

94. IMPORT-EXPORT
 Save current state of the algorithm

95. IMPORT-EXPORT
 Save current state of the algorithm
 APOC* GraphML import-export
*Awesome Procedures on Cypher

96. IMPORT-EXPORT
 Save current state of the algorithm
 APOC* GraphML import-export
*Awesome Procedures on Cypher
// save
CALL apoc.export.graphml.all('my.graphml',
{storeNodeIds:true, readLabels:true, useTypes:true})

97. IMPORT-EXPORT
 Save current state of the algorithm
 APOC* GraphML import-export
*Awesome Procedures on Cypher
// save
CALL apoc.export.graphml.all('my.graphml',
{storeNodeIds:true, readLabels:true, useTypes:true})
// load
MATCH (n) // delete all
DETACH DELETE n
WITH count(*) AS dummy //-----------------
CALL apoc.import.graphml('my.graphml',
{storeNodeIds:true, readLabels:true, useTypes:true})
YIELD nodes, relationships
WITH count(*) AS dummy //-----------------
MATCH (n) RETURN n // show all

98. BACKTRACKING
 In many cases, we need to restore a previous
state and continue from that one
o Next step is based on metrics in the algorithm ?

99. BACKTRACKING
 In many cases, we need to restore a previous
state and continue from that one
o Next step is based on metrics in the algorithm ?
Metrics:

100. BACKTRACKING
 In many cases, we need to restore a previous
state and continue from that one
o Next step is based on metrics in the algorithm ?
Metrics: 5

101. BACKTRACKING
 In many cases, we need to restore a previous
state and continue from that one
o Next step is based on metrics in the algorithm ?
Metrics: 5 12

102. BACKTRACKING
 In many cases, we need to restore a previous
state and continue from that one
o Next step is based on metrics in the algorithm ?
Metrics: 5 12 8

103. BACKTRACKING
 In many cases, we need to restore a previous
state and continue from that one
o Next step is based on metrics in the algorithm ?
Metrics: 5 12 8

104. BACKTRACKING
 In many cases, we need to restore a previous
state and continue from that one
o Next step is based on metrics in the algorithm
 Solutions:
o Export -> Re-import
o Use transactions:
?
Metrics: 5 12 8

105. BACKTRACKING
 In many cases, we need to restore a previous
state and continue from that one
o Next step is based on metrics in the algorithm
 Solutions:
o Export -> Re-import
o Use transactions:
• Abort transaction
?
Metrics: 5 12 8

106. BACKTRACKING
 In many cases, we need to restore a previous
state and continue from that one
o Next step is based on metrics in the algorithm
 Solutions:
o Export -> Re-import
o Use transactions:
• Abort transaction
• Only applicable to one level of backtracking
?
Metrics: 5 12 8

107. DEBUGGING
 Save and play back the states that occurred during execution

108. DEBUGGING
 Save and play back the states that occurred during execution
o A counter for the states is stored in a node

109. DEBUGGING
 Save and play back the states that occurred during execution
o A counter for the states is stored in a node
o Saved query to step back

110. DEBUGGING
 Save and play back the states that occurred during execution
o A counter for the states is stored in a node
o Saved query to step back

111. DEBUGGING
 Save and play back the states that occurred during execution
o A counter for the states is stored in a node
o Saved query to step back

112. DEBUGGING
 For complex errors

113. DEBUGGING
 For complex errors
o Formulate an error pattern in Cypher
MATCH (node:Faulty)
RETURN node

114. DEBUGGING
 For complex errors
o Formulate an error pattern in Cypher
o Use this to define a breakpoint in the code
MATCH (node:Faulty)
RETURN node
if (gds.execute(query).hasNext())
// breakpoint

115. DEBUGGING
 For complex errors
o Formulate an error pattern in Cypher
o Use this to define a breakpoint in the code
o Load the state where
the error occurred
o Debug step by step
MATCH (node:Faulty)
RETURN node
if (gds.execute(query).hasNext())
// breakpoint

116. Lessons learnt

117. Sicco Verwer:
Efficient identification of timed automata.
PhD disszertáció
PSEUDOCODE

118. Sicco Verwer:
Efficient identification of timed automata.
PhD disszertáció
PSEUDOCODE

119. Sicco Verwer:
Efficient identification of timed automata.
PhD disszertáció
PSEUDOCODE

120. Sicco Verwer:
Efficient identification of timed automata.
PhD disszertáció
PSEUDOCODE

121. Sicco Verwer:
Efficient identification of timed automata.
PhD disszertáció
PSEUDOCODE

123. EXPRESSIVE POWER OF CYPHER
 Most graph queries can be decided in polynomial time (P)
 Checking whether two disjoint paths exist is NP-complete
 Cypher is Turing-complete
o List comprehensions, reduce
o The graph can be used to store the tape of the Turing machine
 Cypher 9 (current version)
 Cypher 10 (multiple graph support)

124. SUMMARY
 Neo4j prototype
o Rapid development
o Lots of APOC procedures for specific problems (e.g. mergeNodes)
 Issues
o APOC bug (hangs after)
o Visualisation glitches
o Lack of fixpoint algorithms
 For graph algorithms, we’d recommend to
o create a prototype in Neo4j,
o once the algorithm is understood, rewrite it in imperative code.

125. OUR TEAM
Thanks to: Oszkár Semeráth, Tamás Tóth, András Vörös

126. ACKNOWLEDGEMENTS
 This work partially supported by the ÚNKP-17-1-I. grant by
the Ministry of Human Capacities.
 The advisors were partially supported by the MTA-BME
Lendület Cyber-Physical Systems Research Group.

127. Ω