This document presents research on using EEG signals to perform interactive object segmentation in images. The researchers propose a system where an image is partitioned into windows that are rapidly displayed to a user wearing an EEG headset. The user's EEG data is analyzed to identify their response to "target" windows containing objects versus other windows. This brain response data is then used to guide an image segmentation algorithm to segment the objects from the image. Initial results show the approach can achieve reasonable segmentation accuracy, though further improvements are still needed before it would be practical compared to using a mouse. Future work may explore changing parameters and combining the BCI with eye tracking.
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
Object Segmentation Using EEG Signals
1. Object Segmentation in Images
using EEG Signals
Eva Mohedano, Graham Healy, Kevin McGuinness, !
Xavier Giró-i-Nieto, Noel E. O’Connor and Alan F. Smeaton!
!!
November 6, 2014
2. Outline
•Interactive Object Segmentation!
•ACM MultiMedia High Risk High Reward 2014!
•Related Work!
•System Proposal!
•Results!
•Conclusions
E. Mohedano
2
5. Interactive Object Segmentation
5
1) P. Arbelaez and L. Cohen. Constrained image segmentation from hierarchical boundaries. In
CVPR'08, 2008.!
2) McGuinness, K., & O’Connor, N. E. (2010). A comparative evaluation of interactive segmentation
algorithms. Pattern Recognition, 43(2), 434-444.
E. Mohedano
6. Outline
•Interactive Object Segmentation!
•ACM MultiMedia High Risk High Reward 2014!
•Related Work!
•System Proposal!
•Results!
•Conclusions
E. Mohedano
02 May 2014
6
10. Brain-Computer Interface (BCI)
• Non invasive!
• Well established tool within
clinical practice
10
Strengths
E. Mohedano
11. Brain-Computer Interface (BCI)
• Mostly BCI applications
remain prototypes not
used outside laboratories!
• Users need to be trained!
• Poor BCI performances!
• Low signal-to-noise ratio!
• High dimensional data
11
Challenges HIGH RISK
E. Mohedano
12. Potentially High Reward
• Medical applications!
• Locked in Syndrome (LIS)!
• Prosthetics control, wheelchairs,
spellers
12
•Healthy Users!
• BCI with Virtual
Reality technologies!
• Augmenting gaze
control
E. Mohedano
13. Outline
•Interactive Object Segmentation!
•ACM MultiMedia High Risk High Reward 2014!
•Related Work!
•System Proposal!
•Results!
•Conclusions
02 May 2014
13
E. Mohedano
14. Related Work: RSVP
14
!
•A positive waveform occurring
approximately 300-550ms after
an infrequent task-relevant
stimulus
E. Mohedano
19. Related Work
•BCI speller
19
Ref: D. Fernández-Cañellas, “Modeling temporal dependency of brain responses to rapidly
stimuli in ERP based BCIs” (2013)
E. Mohedano
20. Index
•Interactive Object Segmentation!
•ACM MultiMedia High Risk High Reward 2014!
•Related Work!
•System Proposal!
•Results!
•Conclusions
02 May 2014
20
E. Mohedano
24. System Proposal
Data Acquisition!
Set of 22 images with an associated ground truth mask
24
E. Mohedano
25. System Proposal
Data Acquisition!
Images were partitioned into 192 non overlapped windows!
! ! ! ! !
25
• 15% Target windows!
• RSVP windows at 5Hz!
• User asked to count the
target windows
visualised
E. Mohedano
27. 27
!
1) Down sample from 1000Hz to 250Hz!
2) Bandpass filter 0.1-70 Hz!
3) Cut EEG activity related to each visual event!
4) Down sample from 250Hz to 20Hz!
5) Concatene 31 channels (434D)
!
Support Vector Machine Model (SVM)
System Proposal
EEG processing
!
EEG feature vectors
E. Mohedano
34. Index
•Interactive Object Segmentation!
•ACM MultiMedia High Risk High Reward 2014!
•Related Work!
•System Proposal!
•Results!
•Conclusions
02 May 2014
34
E. Mohedano
35. 35
Conclusions
The approach is feasible: it is possible to use BCI as an interactive
segmentation method based on simple EEG processing.
E. Mohedano
36. Conclusions
36
BCI Interaction for segmentation Mouse Interaction for segmentation
E. Mohedano
BCI is time consuming
Mouse interaction provides better results
37. Future work
37
• Improvements in EEG processing!
• Change resolution of windows!
• Use object candidates instead of a grid!
• Active search!
• Combine local EEG with eye tracker
E. Mohedano
38. Thank you!
38
Questions?
!
This publication resulted from research conducted with the financial
support of Science Foundation Ireland (SFI) under grant number SFI/12/
RC/2289 and partially funded by the Project TEC2013-43935-R BigGraph
of the Spanish Government.
E. Mohedano