Multi-scale visual analysis of time-varying electrocorticography data via clustering of brain regions Motivation There exists a need for effective and easy-to-use software tools supporting the analysis of complex Electrocorticography (ECoG) data. Understanding how epileptic seizures develop or identifying diagnostic indicators for neurological diseases require the in-depth analysis of neural activity data from ECoG. Such data is multi-scale and is of high spatio-temporal resolution. Comprehensive analysis of this data should be supported by interactive visual analysis methods that allow a scientist to understand functional patterns at varying levels of granularity and comprehend its time-varying behavior. Results We introduce a novel multi-scale visual analysis system, ECoG ClusterFlow, for the detailed exploration of ECoG data. Our system detects and visualizes dynamic high-level structures, such as communities, derived from the time-varying connectivity network. The system supports two major views: 1) an overview summarizing the evolution of clusters over time and 2) an electrode view using hierarchical glyph-based design to visualize the propagation of clusters in their spatial, anatomical context. We present case studies that were performed in collaboration with neuroscientists and neurosurgeons using simulated and recorded epileptic seizure data to demonstrate our system's effectiveness. Conclusion ECoG ClusterFlow supports the comparison of spatio-temporal patterns for specific time intervals and allows a user to utilize various clustering algorithms. Neuroscientists can identify the site of seizure genesis and its spatial progression during various the stages of a seizure. Our system serves as a fast and powerful means for the generation of preliminary hypotheses that can be used as a basis for subsequent application of rigorous statistical methods, with the ultimate goal being the clinical treatment of epileptogenic zones.

1. Hierarchical Spatio-Temporal Visual
Analysis of
Electrocorticography (ECoG) Data
Sugeerth Murugesan1,2, Kristofer Bouchard2, Edward
Chang3, Max Dougherty2, Bernd Hamann1, Gunther Weber1,2
1
1UC Davis,2Lawrence Berkeley National Lab,3UC San Francisco

2. Visual Analysis of Dynamic Functional
Brain Networks
On-going Work
2

3. Introduction
• Human brain
• Massively connected
• Dynamically
reconfiguring
• Complex communication
patterns
3
Communication patterns between brain regions[1]
[1] J. Bottger et al., Conexel visualization: a software implementation of glyphs and
edge-bundling for dense connectivity data using BrainGL, J. Neuroscience

4. Electrocorticography
● ECoG
○ High temporal
and spatial
resolution
○ High signal-to-
noise ratio Human ECoG Data, 4mm
spatial resolution, [1]
Edward Chang, UCSF
Micro-electrode array
200μm spatial resolution,
[2] Kristofer Bouchard, LBNL
4

5. Challenges
• Matching of temporal scales
• Appropriate scales not obvious a priori
5

6. System Overview
6

7. Choosing K, Consensus Clustering
7
Maximum value, K =2
Final
Result
For one
time step
Time-series
1 2
[1] Monti et al., Consensus clustering: a resampling-based method for class discovery and visualization of gene expression microarray data

8. Cluster Evolution View
8
[1] Vehlow, Corinna, et al. "Visualizing the evolution of communities in dynamic graphs." Computer Graphics Forum. Vol. 34. No. 1. 2015.

9. Electrode View
9
Time Step
Time
Step
Start
Point
Time Steps
Varying granularity
Chosen Design
Time
Step
Start
Point

10. Synthetic Data
10

11. Identifying Temporal States
11

12. Seizure Dataset
12

13. Functional Patterns
13

14. Conclusion
• Fast and effective scientific exploration of brain networks
• Better comprehension of multi-scale changes
• Better characterization of genesis and dynamic propagation of
epileptic seizures
14

15. Future Work
• Semi-supervised consensus clustering
• Scalable visual analysis
• Quantitative evaluation of flexibility and stability of clusters
15

16. Acknowledgement
This work is supported by the U.S. Department of Energy under Contract No. DE-
AC02-05CH11231 through
•LBNL Laboratory Directed Research and Development (LDRD) grant
“Towards Exascale: High Performance Visualization and Analytics Program”,
program manager Dr. Lucy Nowell.
We thank the members of the
•LBNL Vis Group
•LBNL Data Science & Technology Department
•LBNL Bouchard Group
16

17. Thank
you
17

18. Backup Slides
18

19. ECoG Cluster Flow Overview
19
● Cluster evolution view: summarizes evolution of clusters over time
● Electrode view: depicts propagation of clusters in their spatial domain

20. Community Changes over Space and
Time
20

21. ECoG Functional Networks
Derivation of Functional Networks
21
Graph
Quantification
Graph-
theoretical
Analysis
Colors represent
modules

22. Analysis of connectivity data
Graph theoretic methods
Modular organization
Hierarchical modularity
Challenges
Time-varying 22[2] Complex network measures of brain connectivity: uses and interpretations
Functional connectivity of brain
networks

23. Major Visual Analysis Tasks in Brain
Connectivity Analysis
• How can we start to identify brain states?
• How does brain transition into different states ?
• How can we easily compare patterns associated with
different brain states?
23

24. Brain Networks
Properties
Modular organization
Hierarchical organization on
multiple scales
Small- worldness
24
Spatial Scales Temporal Scales

25. Challenges
Networks
Change over time
Shifting structural properties
Shifting topological attributes
Additional data attributes
Scalability
Visual Scalability
25

26. Interactivity
26

27. Cluster Matching
L1,1 = 11
L1,2 = 11
L2,1 = 10
L2,2 = 2
• All possible
matching
• Maximum
cost value for
each time
27

28. Lessons Learnt
• Static timeline based representations
• Careful with Visual Designs
• Bottom-up vs Top-down
28

29. Graph-Theoretical Analysis
• Network structure
– Hierarchical structure
– Multiple Scales
– Modular
• Visual Analysis
– Neurological Disease
progression
– Normal Functioning 29
Community Pattern
Detection
Source: Stanford School of
Medicine

30. Graph-Theoretical Analysis
• Network structure
– Hierarchical structure
– Multiple Scales
– Modular
• Visual Analysis
– Neurological Disease
progression
– Normal Functioning 30
Community Pattern
Detection
Source: Stanford School of
Medicine

31. Changes in Graphs
31

32. Electrode Spatial Views
• Multiple Temporal
Scales
– Glyphs
– User-defined
aggregation
– Hierarhical
32

33. Hierarchical Exploration
33

34. Multi-scale Spatio-Temporal Visual
Patterns
34
Spatial Scales Temporal Scales

35. Spatial Evolution Patterns
35

36. Electrocorticography
36
● ECoG:
○ High temporal
and spatial
resolution
○ High signal-to-
noise ratio Human ECoG Data, 4mm
spatial resolution, [1]
Edward Chang, UCSF
Micro-electrode array
200μm spatial resolution,
[2] Kristofer Bouchard, LBNL

37. Notes
37