Computational social neuroscience of interpersonal coordination dynamics
1. Computational Social Neuroscience of
Interpersonal Coordination Dynamics
Alignment in body & mind, Ludwig Maximilians University — May 7th, 2019
Guillaume Dumas, Ph.D ( @introspection)
Human Genetics & Cognitive Functions laboratory, Institut Pasteur
2. The social cognition paradox
Dumas, Kelso & Nadel, Front Psych 2014
Mentalizing
Meta-cognition
Representation
Symbols
…
Mirroring
Sensorimotor
Interaction
Meaning
…
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/20192
3. Multiscale Neuroscience: Systems Biology beyond individuals
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/20193
Institute for Systems Biology
4. Multiscale Neuroscience: Systems Biology beyond individuals
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/20193
Institute for Systems Biology
5. Naturalising Social Interaction
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/20194
6. Naturalising Social Interaction
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/20195
7. 13 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
Dual-Video in Developmental psychology
Nadel (1999)
8. 14 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
Combining Dual-Video & Dual-EEG
Dumas et al., PLoS ONE 2010
9. 18 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
Social Interaction ≠ Social Perception
Guionnet et al. 2011; Dumas et al. 2012; Nadel & Dumas, J Cog Edu & Psych 2010
10. 18 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
Social Interaction ≠ Social Perception
Guionnet et al. 2011; Dumas et al. 2012; Nadel & Dumas, J Cog Edu & Psych 2010
11. 21 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
Inter-Brain Synchronization during Social Interaction
Dumas et al., PLoS ONE 2010
Brain-to-brain synchronization correlates with the social roles
13. 33 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
If you want to setup an hyperscanning-EEG system …
Barraza, Dumas, Liu, Blanco-Gomez, van den Heuvel, Baart, & Pérez (MethodX 2019)
14.
15. Hope and hypes of Hyperscanning
Year
Publications in Pubmed
16. Hope and hypes of Hyperscanning
Year
Publications in Pubmed
1965?
17. Hope and hypes of Hyperscanning
Year
Publications in Pubmed
1965?
18. Hope and hypes of Hyperscanning
Year
Publications in Pubmed
1965?
21. Beyond social perception and unidirectional coupling
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201915
Dumas & Fairhurst (PsyArxiv)
Similarity
Behavioural
Neurobiological
Cultural
Communication
Unidirectional
Reciprocal
22. Towards a computational Social Interactive Neuroscience
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201916
Kelso, Dumas, & Tognoli, Neural Networks 2013
Experimental
23. Towards a computational Social Interactive Neuroscience
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201916
Kelso, Dumas, & Tognoli, Neural Networks 2013
ExperimentalComputational
24. Coordination Dynamics within~between brains
Kelso, JAS, Dumas, G, & Tognoli, E (2013) Outline of general theory of
behavior and brain coordination. Neural Networks, 37,120-131
25. Coordination Dynamics: a common language between scales?
Kelso, JAS, Dumas, G, & Tognoli, E (2013) Outline of general theory of
behavior and brain coordination. Neural Networks, 37,120-131
26. Coordination between Models
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201919
31. Cross-frequency coupling
Subject 1
Low frequency (Delta/Theta)
Subject 2
Low frequency (Delta/Theta)
Subject 1
High frequency (Beta/Gamma)
Subject 2
High frequency (Beta/Gamma)
32. Computational Toy-Model
No modulation Low modulation High modulation
Lowcoupling
Lowfrequency
Highcoupling
Lowfrequency
d=1.7 (p<0.001)d=0.6 (p<0.01)n.s.
N_simulations=100; Kruskal-Wallis; code available on Github
33. 25 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
More questions than answers …
How does it relate to anatomical and physiological similarities?
At the intra-individual level?
At the inter-individual level?
Dynamique+Structure
34. 26 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
Social Computational Neuroscience
Dumas et al., PLoS ONE 2012
+ x 2 =
Dual-brain model (Kuramoto oscillators,
realistic brain areas, realistic connectivity)
and their interbrain sensorimotor coupling
35. 27 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
From Cortical Sources to Scalp EEG Traces
Dumas et al., PLoS ONE 2012
FFTBrainAreaSignals
Cortical Level Scalp Level
Cintra
CintraFFTEEGSignals
36. 28 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
Probing the Impact of the Anatomical Topology
Dumas et al., PLoS ONE 2012
Normal Shuffle
37. 29 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
At the Intra-Individual Level
γ-PLV at ROI level
Order parameter
(instantaneous synchronization index)
Dumas et al., PLoS ONE 2012
38. 29 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
At the Intra-Individual Level
γ-PLV at ROI level
Order parameter
(instantaneous synchronization index)
Same real anatomy
Dumas et al., PLoS ONE 2012
39. 29 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
At the Intra-Individual Level
γ-PLV at ROI level
Order parameter
(instantaneous synchronization index)
Same real anatomy Same shuffled anatomy
Dumas et al., PLoS ONE 2012
40. 29 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
At the Intra-Individual Level
γ-PLV at ROI level
Order parameter
(instantaneous synchronization index)
Same real anatomy Same shuffled anatomy Different shuffled anatomy
Dumas et al., PLoS ONE 2012
41. 30 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
Scalp PLV in Low- & High-Frequencies
Dumas et al., PLoS ONE 2012
γ-PLV at ROI level
Order parameter
(instantaneous synchronization index)
α-PLV at scalp level
γ-PLV at scalp level
42. 30 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
Scalp PLV in Low- & High-Frequencies
Dumas et al., PLoS ONE 2012
γ-PLV at ROI level
Order parameter
(instantaneous synchronization index)
α-PLV at scalp level
γ-PLV at scalp level
43. 31 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
The Many Faces of Inter-Brain Synchronization
Dumas et al., PLoS ONE 2012
44. 31 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
The Many Faces of Inter-Brain Synchronization
Dumas et al., PLoS ONE 2012
Real connectivity facilitates inter-brain synchronization
45. 31 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
The Many Faces of Inter-Brain Synchronization
Dumas et al., PLoS ONE 2012
Real connectivity facilitates inter-brain synchronization
Residual synchronization
46. 31 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
The Many Faces of Inter-Brain Synchronization
Dumas et al., PLoS ONE 2012
Real connectivity facilitates inter-brain synchronization
Residual synchronization
Anatomical structure facilitates:
47. 32 • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/2019
Diversity (in biology) seems problematic
The ASD ”idiosyncratic brain”
Hahamy et al., Nature Neuroscience 2015
48. Autism as a Family of Multi-scale Disorders
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201933
Comorbidities
intellectual disability
epilepsy
motor control problems
ADHD
Tics
Anxiety
sleep disorders
gastrointestinal problems.
Epidemiology
0.7–1.1% of the population
4/1 male/female ratio
Huguet, Ey & Bourgeron, Ann Rev Genom & Hum Gen 2013
49. Autism as a Family of Multi-scale Disorders
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201933
Comorbidities
intellectual disability
epilepsy
motor control problems
ADHD
Tics
Anxiety
sleep disorders
gastrointestinal problems.
Epidemiology
0.7–1.1% of the population
4/1 male/female ratio
Huguet, Ey & Bourgeron, Ann Rev Genom & Hum Gen 2013
50. Autism as a Family of Multi-scale Disorders
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201933
Comorbidities
intellectual disability
epilepsy
motor control problems
ADHD
Tics
Anxiety
sleep disorders
gastrointestinal problems.
Epidemiology
0.7–1.1% of the population
4/1 male/female ratio
Huguet, Ey & Bourgeron, Ann Rev Genom & Hum Gen 2013
CONDITIONSCONDITIONS
51. Socio-cultural
Cognitive-behavioral
Biological
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201934
Autism as a Dialectical Misattunement
Beyond Autism: Introducing the Dialectical Misattunement
Hypothesis and a Bayesian Account of Intersubjectivity. Bolis et al. 2017
52. Modeling Dialectical Misattunement
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201935
Dumas, Zillekens et al., (in preps)
Imme
ZILLEKENS
Leonard
SCHILBACH
Juha
LAHNAKOSKI
TYP-TYP
TYP-ASD
ASD-ASD
53. Modeling Dialectical Misattunement
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201936
Dumas, Zillekens et al., (in preps)
Imme
ZILLEKENS
Leonard
SCHILBACH
Juha
LAHNAKOSKI
Coupling at Inter-individual levelCouplingatIntra-individuallevel
54. Modeling Dialectical Misattunement
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201937
Dumas, Zillekens et al., (in preps)
Imme
ZILLEKENS
Leonard
SCHILBACH
Juha
LAHNAKOSKI
55. From Models with Models to Models with Humans
Stable coordination maintained when baseline is shifted
Stable coordination maintained when movement amplitude decreased
Kelso JAS, de Guzman GC, Reveley C, Tognoli E (2009) Virtual Partner Interaction:
Exploring Novel Behaviors via Coordination Dynamics. PLoS ONE 4(6)
56. Principle-based social machines
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201939
57. Virtual Partner Interaction
Kelso JAS, de Guzman GC, Reveley C, Tognoli E (2009) Virtual Partner Interaction:
Exploring Novel Behaviors via Coordination Dynamics. PLoS ONE 4(6)
Dumas, G., de Guzman, G. C., Tognoli, E., & Kelso, J. S. (2014). The Human Dynamic
Clamp as a Paradigm for Social Interaction. PNAS, 111(35), E3726–E3734.
58. Virtual Partner Interaction
Kelso JAS, de Guzman GC, Reveley C, Tognoli E (2009) Virtual Partner Interaction:
Exploring Novel Behaviors via Coordination Dynamics. PLoS ONE 4(6)
Dumas, G., de Guzman, G. C., Tognoli, E., & Kelso, J. S. (2014). The Human Dynamic
Clamp as a Paradigm for Social Interaction. PNAS, 111(35), E3726–E3734.
59. Virtual Partner Interaction
Kelso JAS, de Guzman GC, Reveley C, Tognoli E (2009) Virtual Partner Interaction:
Exploring Novel Behaviors via Coordination Dynamics. PLoS ONE 4(6)
Dumas, G., de Guzman, G. C., Tognoli, E., & Kelso, J. S. (2014). The Human Dynamic
Clamp as a Paradigm for Social Interaction. PNAS, 111(35), E3726–E3734.
60. The Human Dynamic Clamp: computational models
Dumas, de Guzman, Tognoli & Kelso (PNAS, 2014).
The Human Dynamic Clamp as a Paradigm for Social Interaction
61. The Human Dynamic Clamp: computational models
Dumas, de Guzman, Tognoli & Kelso (PNAS, 2014).
The Human Dynamic Clamp as a Paradigm for Social Interaction
62. The Human Dynamic Clamp: computational models
Adaptive Excitator for Movement Frequency (metaparameter)
Dumas, de Guzman, Tognoli & Kelso (PNAS, 2014).
The Human Dynamic Clamp as a Paradigm for Social Interaction
63. The Human Dynamic Clamp: computational models
Adaptive Excitator for Movement Frequency (metaparameter)
Intentional Coordination: VP becomes VT (Virtual Teacher)
Dumas, de Guzman, Tognoli & Kelso (PNAS, 2014).
The Human Dynamic Clamp as a Paradigm for Social Interaction
64. The Human Dynamic Clamp: behavioural data
Dumas, Moreau, Tognoli & Kelso (in preps) Transient coupling of neural networks
for self and other underlies attribution of intention and humanness
65. The Human Dynamic Clamp: neurophysiological data
Dumas, Moreau, Tognoli & Kelso (in preps) Transient coupling of neural networks
for self and other underlies attribution of intention and humanness
66. The Human Dynamic Clamp: neurophysiological data
Dumas, Moreau, Tognoli & Kelso (in preps) Transient coupling of neural networks
for self and other underlies attribution of intention and humanness
67. The Human Dynamic Clamp: neurophysiological data
Dumas, Moreau, Tognoli & Kelso (in preps) Transient coupling of neural networks
for self and other underlies attribution of intention and humanness
68. The Human Dynamic Clamp: neurophysiological data
Dumas, Moreau, Tognoli & Kelso (in preps) Transient coupling of neural networks
for self and other underlies attribution of intention and humanness
69. The Human Dynamic Clamp: neurophysiological data
Dumas, Moreau, Tognoli & Kelso (in preps) Transient coupling of neural networks
for self and other underlies attribution of intention and humanness
72. • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201952
Social Neuroscience for Therapeutic Approaches in Autism
V0 V9
UMR
73. • Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201953
Social Neuroscience for Therapeutic Approaches in Autism
74. From laboratory to hospital: HDC-EEG in autism
Lefebvre et al. (INSAR 2019)
75. “What am I doing with this wiggling finger?”
???
76. From hospital to home: mixed reality video games
Illustration of Pop’Balloons (Actimage, 2018)
77. From hospital to home: mixed reality video games
Video of Pop’Balloons (Actimage, 2018)
78. Computational Psychiatry at Scale
Dumas et al. (in preps.)
Heartbeat
+ LSTM
Movement
+ ConvNet
First person perspective
Profiles
Code
AutoEncoder (AE)
Latent
space
90. Old Paris Team
Jacqueline Nadel
Jacques Martinerie
Robert Soussignan
Mario Chavez
Boca Raton TeamCurrent Paris Team
Collaborators
Merle Fairhurst
Quentin Moreau
Anna Banki
J.-A. Micoulaud-Franchi
Leonhard Schilbach
Imme Zillekens
Juha Lahnakoski
Dimitris Bolis
Pavel Goldstein
Thomas Bourgeron
Richard Delorme
Jean-Pierre Changeux
Yang-Min Kim
Aline Lefebvre
Florence Baillin
Amandine Pedoux
Frederique Amsellem
Thomas Rolland
Freddy Cliquet
Alexandre Mathieu
J.A. Scott Kelso
Emmanuelle Tognoli
Armin Fuchs
Mengsen Zhang
Eva Loth
Jumana Ahmad
Danilo Bzdok
Denis Engemann
Alexandre Lehmann
Julien Laroche
Suzanne Dikker
Rebecca Jones
Viviane Kostrubiec
Interdisciplinarity
Open Science
Art~Science
Alius Team
David Dupuis
Matthieu Koroma
Raphaël Millière
Martin Fortier
Baptiste Gilles
… and all the
associated
researchers!
Fundings
• Guillaume Dumas • Computational Social Neuroscience of Interpersonal Coordination Dynamics • 07/05/201963
91. Thanks for your attention
Contact: guillaume.dumas@pasteur.fr
Web: Extrospection.eu • Twitter: @introspection
Phi
92. Thanks for your attention
Contact: guillaume.dumas@pasteur.fr
Web: Extrospection.eu • Twitter: @introspection
PhiPhi