My presentation at IEEE VisWeek 2012 in Seattle, WA //// Abstract: Event sequence data is common in many domains, ranging from electronic medical records (EMRs) to sports events. Moreover, such sequences often result in measurable outcomes (e.g., life or death, win or loss). Collections of event sequences can be aggregated together to form event progression pathways. These pathways can then be connected with outcomes to model how alternative chains of events may lead to different results. This paper describes the Outflow visualization technique, designed to (1) aggregate multiple event sequences, (2) display the aggregate pathways through different event states with timing and cardinality, (3) summarize the pathways’ corresponding outcomes, and (4) allow users to explore external factors that correlate with specific pathway state transitions. Results from a user study with twelve participants show that users were able to learn how to use Outflow easily with limited training and perform a range of tasks both accurately and rapidly.

1. InfoVis 2012
Seattle, WA
Outflow
Exploring Flow, Factors and Outcomes
of Temporal Event Sequences
Krist Wongsuphasawat
HCIL, University of Maryland
David Gotz
IBM Research
m

2. m
Events

3. m
Event | 12:15 p.m. Lunch

4. m
Event Sequences
Event Event Event

5. Daily Activity
7:30 a.m. 7:45 a.m. 8:15 a.m.
Wake Up Exercise Go to work
m

6. Soccer Game
10th minute 25th minute 90th minute
Team A scores Team B scores Team A scores
m

7. Soccer Game
Time
Game #1
10th minute 25th minute 90th minute
Goal Concede Goal
m

8. Many games
Time
Game #1
Goal Concede Goal
Game #2
Goal Goal Concede
Game #3
Goal Concede Concede
Game #n
Concede Goal Goal Goal
m

9. with outcome
Time
Game #1 Win (1)
Goal Concede Goal
Game #2 Win (1)
Goal Goal Concede
Game #3 Lose (0)
Goal Concede Concede
Game #n Win (1)
Concede Goal Goal Goal
m

10. 7 events per entity
7 event types
823543 co mbinations
m

11. Enjoy!
m

12. consumable
m

13. Overview / Summary
Event Sequences
with Outcome
m

14. m
7
Steps

15. m
Step 1 | Aggregation

16. Event Sequences
Entity #1
Entity #2
Outflow
Graph
Entity #3
Entity #4
Entity #5
Entity #6
Entity #7
…
Entity #n
m

17. Assumption
• Events are persistent.
Entity #1
e1 e2 e3
Entity #1
m

18. Assumption
• Events are persistent.
Entity #1
e1 e2 e3
Entity #1
e1 e1 e1
m

19. Assumption
• Events are persistent.
Entity #1
e1 e2 e3
Entity #1
e1 e1 e1
e2 e2
m

20. Assumption
• Events are persistent.
Entity #1
e1 e2 e3
Entity #1
e1 e1 e1
e2 e2
e3
m

21. Assumption
• Events are persistent.
Entity #1
e1 e2 e3
Entity #1
e1 e1 e1
[e1] e2 e2
e3
States [e1, e2]
[e1, e2, e3]
m

22. Select alignment point
Pick a state
What are the paths What are the paths
that led to ? after ?
Example
Soccer: Goal, Concede, Goal
m

23. Select alignment point
Pick a state
What are the paths What are the paths
that led to ? after ?
or just an empty state []
m

24. Outflow Graph
Alignment Point
[e1, e2, e3]
m

25. 1 entity
Outflow Graph
Alignment Point
[e1]
[e1, e2]
[ ]
[e1, e2, e3]
[e1, e2, e3, e5]
m

26. 2 entities
Outflow Graph
Alignment Point
[e1]
[e1, e2]
[ ]
[e1, e3]
[e1, e2, e3]
[e1, e2, e3, e5]
m

27. 3 entities
Outflow Graph
Alignment Point
[e1]
[e1, e2]
[e1, e2, e3, e4]
[ ]
[e1, e3]
[e1, e2, e3]
[e1, e2, e3, e5]
[e3]
m

28. n entities
Outflow Graph
Alignment Point
[e1]
[e1, e2]
[e1, e2, e3, e4]
[ ]
[e2]
[e1, e3]
[e1, e2, e3]
[e1, e2, e3, e5]
[e3]
[e2, e3]
m

29. n entities
Outflow Graph
Alignment Point
[e1]
[e1, e2]
[e1, e2, e3, e4]
[ ]
[e2]
[e1, e3]
[e1, e2, e3]
[e1, e2, e3, e5]
[e3]
[e2, e3]
Average outcome = 0.4
Average time = 10 days
layer Number of entities = 10
m

30. Soccer Results
Alignment Point
1-0
2-0
2-2
0-0
1-1
2-1
3-1
0-1
0-2
m

31. m
Step 2 | Visual Encoding

32. Past Future
Alignment
Node’s horizontal position
shows sequence of states.
e1!
e2!
e3!
End of path
e1!
e1!
e2!
time link e1!
Node’s height is
edge edge e2!
number of entities.
e4!
e2!
Color is outcome Time edge’s width is
measure. duration of transition. m

33. m
Step 3 | Graph Drawing

34. m

35. m

36. 3.1 Sugiyama’s heuristics
• Directed Acyclic Graph (DAG) layout
– Sugiyama, K., Tagawa, S. & Toda, M., 1981.
Methods for Visual Understanding of Hierarchical System Structures.
IEEE Transactions on Systems, Man, and Cybernetics, 11(2), p.109-125.
• Reduce edge crossing
m

37. 41 crossings
m

38. 12 crossings
m

39. m

40. 3.2 Force-directed layout
• Spring simulation
Each node is particle.
x
Total force = Force from edges - Repulsion between nodes
m

41. m

42. m

43. 3.3 Edge Routing
• Avoid unnecessary crossings
Reroute
m

44. 3.3 Edge Routing
• After routing
m

45. m

46. m

47. m
Step 4 | Interactions

48. Interactions
• Panning
• Zooming
• Brushing
• Pinning
• Tooltip
• Event type selection
m

49. m
Demo

50. m
Step 5 | Simplification

51. Node Clustering
• Cluster nodes in each layer
• Similarity measure: Outcome, etc.
• Threshold (0-1)
m

52. m

53. m

54. m
Step 6 | Factors

55. Factors
Time
Entity #1
[e1] [e1, e2] [e1, e2, e3]
Factor 1 Factor 2 Factor 3 Factor 4
m

56. Factors
Time
Patient #1
[e1] [e1, e2] [e1, e2, e3]
Yellow Injury Red Substitution
Which factors are correlated to each state?
m

57. Information Retrieval
Which keywords are correlated to each document?
State 1 State 2 State 3
… … …
Factor xxx … …
… … …
Doc#1 Doc#2 Doc#3
Which factors are correlated to each state?
m

58. Present factors
Alignment Point
Factor 1 [e1] [e1,e2]
[e1,e2,e3,e4]
[] [e2] [e1,e3]
[e1,e2,e3]
[e1,e2,e3,e5]
[e3] [e2,e3]
m

59. Absent factors
Alignment Point
[e1] [e1,e2]
[e1,e2,e3,e4]
Factor 2
[] [e2] [e1,e3]
Factor 2
[e1,e2,e3]
[e1,e2,e3,e5]
[e3] [e2,e3]
m

60. tf-idf
• Term frequency
tf =
Number of times a term t appear in the document
Number of terms in the document
• Inverse document frequency
idf = log ( Number of documents
Number of documents that has the term t + 1
)
m

61. Score based on tf-idf
• Ratio (presence)
Rp = Number of entities with factor f before state
Number or entities in the state
• Inverse state ratio (presence)
R-1
sp = log ( Number of states
Number of states preceded by factor f + 1
)
m

62. m

63. m
Step 7 | User Study

64. User Study
• Goal:
Evaluate Outflow’s ability
to support event sequence analysis tasks
• 12 participants
• 60 minutes each
• 9 tasks + 7 training tasks
• Questionnaire
m

65. Results
• Accurate:
3 mistakes from 108 tasks
• Fast:
Average 5-60 seconds
• Findings:
– From video
– Different outcomes for each incoming paths
– Etc.
m

66. Future Work
• Integration with prediction algorithm
• Additional layout techniques
• Advanced factor analysis
• Deeper evaluations with domain experts
m

67. Conclusions
• Event sequences with outcome
• Outflow
– Interactive visual summary
– Explore flow & outcome
– Factors
– Multi-step layout process
• Not specific to sports
Contact: kristw@twitter.com dgotz@us.ibm.com
@kristwongz
m

68. Heart failure (CHF) patient
Time
Patient #1 Die (0)
Aug 1998 Oct 1998 Jan 1999
Ankle Edema Cardiomegaly Weight Loss
m

69. Event Sequences
Medical Transportation
Sports Education
Web logs Logistics
and more…
m

70. Acknowledgement
• Charalambos (Harry) Stavropoulos
• Robert Sorrentino
• Jimeng Sun
• Comments from HCIL colleagues
m

71. Conclusions
• Event sequences with outcome
• Outflow
– Interactive visual summary
– Explore flow & outcome
– Factors
– Multi-step layout process
• Not specific to medical or sports
Contact: kristw@twitter.com dgotz@us.ibm.com
@kristwongz
m

72. m
THANK YOU
ขอบคุณครับ