SlideShare ist ein Scribd-Unternehmen logo
1 von 72
Downloaden Sie, um offline zu lesen
Towards psychoinformatics
with machine learning and brain imaging
Gaël Varoquaux
Towards psychoinformatics
with machine learning and brain imaging
Gaël Varoquaux
Propose and illustrate a research program
Accumulation
of functional brain data
G Varoquaux 2
Theories of the mind
Laws of the mind for
perception, decision,
action, emotion...
G Varoquaux 3
Studying mental processes
1 Craft an experimental condition that recruits it
G Varoquaux 4
Studying mental processes
1 Craft an experimental condition that recruits it
2 Do an elementary psychological manipulation
G Varoquaux 4
Studying mental processes
- -Results in
a contrast
Study by
oppositions
G Varoquaux 4
Studying mental processes
- -Results in
a contrast
Study by
oppositions
Results tied to a simple psychological manipulation
bound to a paradigm
G Varoquaux 4
Proposing
Generalization
to build broader theories
bridging paradigms
Paradigm 1: Seen
G Varoquaux 5
Proposing
Generalization
to build broader theories
bridging paradigms
Paradigm 1: Seen Paradigm 2: Imagined
G Varoquaux 5
Generalization
for broader theories
bridging paradigms
Seen objects
G Varoquaux 6
Brain imaging
⇒ brain-mind associations
DiïŹƒculty: modeling
behavior & cognition
G Varoquaux 7
Predictive models can broaden theories by
generalizing brain-mind associations to arbitrary new
tasks and stimuli [Varoquaux and Poldrack 2018]
1 Beyond oppositions: encoding
2 Generalizing across tasks
3 Universal cognitive representations
4 Across subjects: biomarkers
G Varoquaux 8
1 Beyond oppositions: encoding
[Eickenberg... 2017]
G Varoquaux 9
1 Decomposing psychological process
To study vision: Breaking down stimuli
G Varoquaux 10
1 Decomposing psychological process
To study vision: Breaking down stimuli
[Hubel and Wiesel 1962]
Neurons receptive to
Gabors (edges)
G Varoquaux 10
1 Decomposing psychological process
To study vision: Breaking down stimuli
[Hubel and Wiesel 1962]
Neurons receptive to
Gabors (edges)
[Logothetis... 1995]
Shapes in inferior
temporal cortex
G Varoquaux 10
1 Decomposing psychological process
To study vision: Breaking down stimuli
Image
V1
cortex
V2
cortex
Inferior
temporal
cortex
Fusiform
face area
Jack?
Is there a “face” region? A “foot” region? A “left big toe” region?
G Varoquaux 10
1 Crafting stimuli for cognitive oppositions
Is there a “face” region? A “foot” region? A “left big toe” region?
vs
G Varoquaux 11
1 Crafting stimuli for cognitive oppositions
Is there a “face” region? A “foot” region? A “left big toe” region?
vs
G Varoquaux 11
1 Crafting stimuli for cognitive oppositions
Is there a “face” region? A “foot” region? A “left big toe” region?
vs
-
G Varoquaux 11
1 Crafting stimuli for cognitive oppositions
Is there a “face” region? A “foot” region? A “left big toe” region?
vs
-
Representing only one aspect of the stimuli:
too much reductionism
G Varoquaux 11
1 Not crafting stimuli for cognitive oppositions
Is there a “face” region? A “foot” region? A “left big toe” region?
vs
-
Representing only one aspect of the stimuli:
too much reductionism
Encoding
Building complex representations of stimuli
Predicting brain response from them
G Varoquaux 11
1 Decomposing visual stimuli
Low-level visual cortex is tuned
to natural image statistics
[Olshausen et al. 1996]
What drives high-level representations?
The concept of “left big toe”?
G Varoquaux 12
1 Decomposing visual stimuli
Low-level visual cortex is tuned
to natural image statistics
[Olshausen et al. 1996]
What drives high-level representations?
Convolutional Net
G Varoquaux 12
1 Brain mapping: encoding with conv nets
StimulirepresentationBrainactivity
Input Layer 1 Layer 5
Linear predictive models
Convolutional Net
G Varoquaux 13
1 Brain mapping: encoding with conv nets
StimulirepresentationBrainactivity
Input Layer 1 Layer 5
Linear predictive models
Convolutional Net
Explains the brain activity from the stimuli
much better than hand-crafted features
On data from [Kay... 2008] natural images
[Huth... 2012] movies
[Eickenberg... 2017]
G Varoquaux 13
1 Brain mapping: encoding with conv nets
[Eickenberg... 2017]
High-level conv-net layer map to high-level visual areas
G Varoquaux 14
1 Brain mapping with conv nets: retinotopy
Mapping response to exentricity: artiïŹcal retinotopy
G Varoquaux 15
1 Brain mapping with conv nets: high-level concepts
Opposing face versus place in the [Haxby... 2001] stimuli
G Varoquaux 16
[Eickenberg... 2017]
Beyond oppositions: encoding
Recovers the
hierarchy of
visual modules
Generalizes
from natural
images
to retinotopy or
face vs place
G Varoquaux 17
2 Generalizing across tasks
Characterizing the function
of brain structures
How much is observed activity
a consequence of speciïŹcities
of the paradigm?
[Schwartz... 2013, Varoquaux... 2018]
G Varoquaux 18
2 Large-scale decoding for reverse inference
Cat
Dog
Ouaf
Miaou
[Poldrack... 2009]
G Varoquaux 19
2 Large-scale decoding for reverse inference
Cat
Dog
Ouaf
Miaou
[Poldrack... 2009]
Variability: an opportunity and a challenge
Technical heterogeneity (scanner, stimulus modality...)
Paradigmatic isolation (cognition) [Newell 1973]
G Varoquaux 19
2 Generalizing to arbitrary paradigms
[Schwartz... 2013, Varoquaux... 2018]
What is this
brain doing?
Describe tasks by their cognitive components
Multi-label prediction: presence or absence of each label
Visual
Auditory
Foot
Hand
Calculation
Reading
Checkboard
Face
Place
Object
Digit
Saccade
...
...
...
...
...
...
G Varoquaux 20
2 Generalizing to arbitrary paradigms
[Schwartz... 2013, Varoquaux... 2018]
What is this
brain doing?
Describe tasks by their cognitive components
Multi-label prediction: presence or absence of each label
Visual
Auditory
Checkboard
Face
...
...
...
...
Prediction across studies
Describe a task never seen
from the brain activity it evokes
G Varoquaux 20
2 Generalizing to arbitrary paradigms
[Schwartz... 2013, Varoquaux... 2018]
What is this
brain doing?
+
+
Regions speciïŹc to that predict
facets of cognition
G Varoquaux 20
2 Decoding mega-analysis
[Schwartz... 2013, Varoquaux... 2018]
30 studies 837 subjects 196
experimental
conditions
6919 activation maps
DiïŹ€erent labs
Dehaene
Poldrack
Wager
...
Various cognitive domains
Language
Vision
Decision making
Mathematics
...
All manually preprocessed, labeled, and curated
G Varoquaux 21
2 Decoding mega-analysis
[Schwartz... 2013, Varoquaux... 2018]
30 studies 837 subjects 196
experimental
conditions
6919 activation maps
DiïŹ€erent labs
Dehaene
Poldrack
Wager
...
Various cognitive domains
Language
Vision
Decision making
Mathematics
...
All manually preprocessed, labeled, and curated
Decoding arbitrary new paradigms
visualauditoryVcheckerboard
Hcheckerboardobjectsscramblefacesplacesdigits
wordsrighthandlefthandrightfootleftfootlanguagehumansound
non-humansound
0.2
0.4
0.6
0.8
1.0
Predictionscore
(ROCAUC)
ontologydecoder
logisticregression
naiveBayes
neurosynth
­0.3
­0.2
­0.1
0.0
+0.1
+0.2
+0.3
Relative score
G Varoquaux 21
2 Mapping: regions speciïŹc to facets of cognition
[Schwartz... 2013, Varoquaux... 2018]
Contrasts: good rejection of confounds
Decoding: reverse inference
⇒ Consensus to deïŹne regions
The “place” concept
Forward
term eïŹ€ect
Brain response to tasks with
the “place” concept
Forward
ontology contrast
Contrasting responses to
“place” and related concepts
Reverse
decoder map
Regions that predict a place-
recognition task
Consensus
forward & reverse
Regions selected by forward
and reverse inference
Replaces the careful crafting of control conditions
G Varoquaux 22
2 A functional atlas: central gyrus, motor networks
[Schwartz... 2013, Varoquaux... 2018]
Neurosynth
Reverse inference
(Logistic regression)
Decoding in an
ontology
z = -15
Reverse inference
-15
nference DecodDecoding in an
visual faces places objects scrambled words digits horizontal - vertical
checkerboard
Visual regions
Contrasts
-based
analysis
Neurosynth
reverse
inference
Decoding
and
ontology
+ Language and auditory mapping in temporal cortex
+ Calculation and spatial attenation in IPS
+ Mapping the motor system
G Varoquaux 23
[Varoquaux... 2018]
Generalizing across tasks
Cat
Dog
Ouaf
Miaou
Multi-label
prediction
Prediction
(discriminative
models) more robust
to heterogeneity
Decoding
⇒ evidence beyond
⇒ a paradigm
Atlasing cognition
G Varoquaux 24
Need an ontology of cognition
Episodic memory
working memory
lexical memory
Syntax comprehension
Reading
Mental calculation
Shape recognition
Face recognition
Memory
Language
Vision
G Varoquaux 25
3 Universal cognitive
representations [Mensch... 2017]
Face Shape
Reading Shape
Language Music
Saccades Calculation
G Varoquaux 26
3 Universal cognitive
representations [Mensch... 2017]
Face Shape
Reading Shape
Language Music
Saccades CalculationMulti-task learning
Solve many related but diïŹ€erent problems
Learn commonalities
intermediate representations
G Varoquaux 26
3 Deep architecture
[Bengio 2009]
...
Great for multiple output (tasks)
G Varoquaux 27
3 Deep architecture
[Bengio 2009]
...
Great for multiple output (tasks)
Millions of parameters, thousands of data points
G Varoquaux 27
3 Deep architecture
2
2
Great for multiple output (tasks)
Millions of parameters, thousands of data points
Simplify
G Varoquaux 27
3 Shallow architecture
2
linear model
Great for multiple output (tasks)
Millions of parameters, thousands of data points
Simplify simplify more
G Varoquaux 27
3 Shallow architecture
2
2
Great for multiple output (tasks)
Millions of parameters, thousands of data points
Simplify simplify more
G Varoquaux 27
3 Deep linear model – multi-task – Bayesian
4TB resting-state data
HCP900
OpenfMRI
HCP
Camcan
Brainomics
42000 task
fMRI contrast maps
for one model
...
Many
rest & task
fMRI studies
...
x
Information ïŹ‚ow
x Learned network
combination
1st layer
assignment
Non-negative
matrix factorization
SUCCESSFUL
GO
HOUSE
HORIZONTAL
CHECKERBOARD
Study-wise decoding
REWARD
BALLOON
EXPLODE
............
Deep linear model
Joint training
Bayesian deep learning
Resting-state
functional loadings
512 resting-state
networks
Task functional
loadings
128 task-optimized
networks
Contrast
maps
200,000
voxels
Contrast
name
Probability
vector
FACE BODY
HOUSE TOOL
512 components
[Mensch... 2018]
G Varoquaux 28
3 Deep linear model – multi-task – Bayesian
OpenfMRI
HCP
Camcan
Brainomics
...
...
Information ïŹ‚ow
Non-negative
matrix factorization
SUCCESSFUL
GO
HOUSE
HORIZONTAL
CHECKERBOARD
Study-wise decoding
REWARD
BALLOON
EXPLODE
............
Deep linear model
Task functional
loadings
128 task-optimized
networks
[Mensch... 2018]
Intermediate representation:
Task-optimized networks
G Varoquaux 28
3 Deep linear model – multi-task – Bayesian
OpenfMRI
HCP
Camcan
Brainomics
...
...
Information ïŹ‚ow
Non-negative
matrix factorization
SUCCESSFUL
GO
HOUSE
HORIZONTAL
CHECKERBOARD
Study-wise decoding
REWARD
BALLOON
EXPLODE
............
Deep linear model
Task functional
loadings
128 task-optimized
networks
[Mensch... 2018]
Intermediate representation:
Task-optimized networks
Improves decoding in each study
0% 20% 40% 60% 80%
Decoding accuracy on test set
Stop-signal w/ spoken & manual resp.
The Human Connectome Project
UCLA LA5C
Simon task
Plain or mirror-reversed text
Stop-signal
CamCan audio-visual
Localizer
Cross-language repetition priming
Twin localizer
High-level math
Spatio-temporal judgement (retake)
Constit. struct. of sent. & music
Sentence/music complexity
Word & object processing
Emotion regulation
Arithmetic & saccades
Brainomics localizer
Incidental encoding
Spatio-temporal judgement
False belief
Visual object recognition
Face recognition
Stop-signal & classification
Classification learning
Compression
Motor task & word/verb generation
BART, stop-signal, emotion
Rhyme judgment
Auditory & Visual Oddball
Weather prediction
Stop-signal & classification (retake)
Balloon Analog Risk-taking
Mixed-gambles
Classif. learning & reversal
Task fMRI study
Decoding from
voxels
Decoding from
multi-study
networks
Chance level
-5% 0% 5%10%15%
Multi-study acc. gain
G Varoquaux 28
3 Deep linear model – multi-task – Bayesian
OpenfMRI
HCP
Camcan
Brainomics
...
...
Information ïŹ‚ow
Non-negative
matrix factorization
SUCCESSFUL
GO
HOUSE
HORIZONTAL
CHECKERBOARD
Study-wise decoding
REWARD
BALLOON
EXPLODE
............
Deep linear model
Task functional
loadings
128 task-optimized
networks
[Mensch... 2018]
Intermediate representation:
Task-optimized networks
Improves decoding in each study
4 16 32 100 400
Number of train subjects
0%
5%
10%
15%
20%
Multistudyacc.gain
Study
Small studies beneïŹt from large ones
G Varoquaux 28
3 Deep linear model – multi-task – Bayesian
OpenfMRI
HCP
Camcan
Brainomics
...
...
Information ïŹ‚ow
Non-negative
matrix factorization
SUCCESSFUL
GO
HOUSE
HORIZONTAL
CHECKERBOARD
Study-wise decoding
REWARD
BALLOON
EXPLODE
............
Deep linear model
Task functional
loadings
128 task-optimized
networks
[Mensch... 2018]
Intermediate representation:
Task-optimized networks
Enables smaller sample sizes
7 15 23 31 39
# training subjects in target study
70%
75%
80%
85%
Testaccuracy
Pinel et al. '07
9 18 28 37 47
70%
80%
90%
Papadopoulos-
Orfanos '12
60 12
60%
65%
C
(Sh
37 47
oulos-
'12
60 121 181 242 302
60%
65%
CamCan AV
(Shafto et al. '14)
Standard decoding Multi-study decoder
2 4 7 9 12
20%
40%
60%
Cohen '09
G Varoquaux 28
3 Deep linear model – multi-task – Bayesian
OpenfMRI
HCP
Camcan
Brainomics
...
...
Information ïŹ‚ow
Non-negative
matrix factorization
SUCCESSFUL
GO
HOUSE
HORIZONTAL
CHECKERBOARD
Study-wise decoding
REWARD
BALLOON
EXPLODE
............
Deep linear model
Task functional
loadings
128 task-optimized
networks
[Mensch... 2018]
Intermediate representation:
Task-optimized networks
G Varoquaux 28
3 Deep linear model – multi-task – Bayesian
OpenfMRI
HCP
Camcan
Brainomics
...
...
Information ïŹ‚ow
Non-negative
matrix factorization
SUCCESSFUL
GO
HOUSE
HORIZONTAL
CHECKERBOARD
Study-wise decoding
REWARD
BALLOON
EXPLODE
............
Deep linear model
Task functional
loadings
128 task-optimized
networks
[Mensch... 2018]
Intermediate representation:
Task-optimized networks
G Varoquaux 28
3 Deep linear model – multi-task – Bayesian
OpenfMRI
HCP
Camcan
Brainomics
...
...
Information ïŹ‚ow
Non-negative
matrix factorization
SUCCESSFUL
GO
HOUSE
HORIZONTAL
CHECKERBOARD
Study-wise decoding
REWARD
BALLOON
EXPLODE
............
Deep linear model
Task functional
loadings
128 task-optimized
networks
[Mensch... 2018]
Intermediate representation:
Task-optimized networks
G Varoquaux 28
[Mensch... 2017]
Universal cognitive representations
Multi-task decoding:
- decoding the original task labels
Deep linear models:
- intermediate representations across studies
- statistical power from big to small studies
Face Shape
Reading Shape
Language Music
Saccades Calculation
G Varoquaux 29
[Mensch... 2017]
Universal cognitive representations
Multi-task decoding:
- decoding the original task labels
Deep linear models:
- intermediate representations across studies
- statistical power from big to small studies
Face Shape
Reading Shape
Language Music
Saccades Calculation
G Varoquaux 29
4 Across subjects: biomarkers
G Varoquaux 30
4 Beyond heterogeneity: predicting autism across sites
Accuracy
Fraction of subjects used
More data is better (up to 1000 subjects)
[Abraham... 2017]
G Varoquaux 31
4 Brain aging: a surrogate biomarker
[Liem... 2017]
Predicting brain aging = chronological age
Multi-modal: brain connectivity and morphology
Age prediction: mean absolute error of 4.3 years
G Varoquaux 32
4 Brain aging: a surrogate biomarker
[Liem... 2017]
Predicting brain aging = chronological age
Multi-modal: brain connectivity and morphology
Age prediction: mean absolute error of 4.3 years
Discrepency with chronological age
correlates with cognitive impairment
0 2 4
Brain aging discrepancy
(years)
-0.38
0.74
1.72
Objective Cognitive
Impairment group
Normal
Mild
Major
G Varoquaux 32
@GaelVaroquaux
Psychoinformatics with machine learning
Prediction for broader theories
AI to model stimuli / the world
Explicit generalization across paradigms
Cat
Dog
Ouaf
Miaou
@GaelVaroquaux
Psychoinformatics with machine learning
Prediction for broader theories
AI to model stimuli / the world
Explicit generalization across paradigms
Beyond oppositions
Encoding complete descriptions of tasks
Decoding multiple facets of cognitions
@GaelVaroquaux
Psychoinformatics with machine learning
Prediction for broader theories
AI to model stimuli / the world
Explicit generalization across paradigms
Beyond oppositions
Encoding complete descriptions of tasks
Decoding multiple facets of cognitions
Useful with imperfect labels
Extracting common representations
Surrogate biomarkers
@GaelVaroquaux
Psychoinformatics with machine learning
Prediction for broader theories
AI to model stimuli / the world
Explicit generalization across paradigms
Beyond oppositions
Encoding complete descriptions of tasks
Decoding multiple facets of cognitions
Useful with imperfect labels
Extracting common representations
Surrogate biomarkers
Software: nilearn
http://nilearn.github.io ni
References I
A. Abraham, M. P. Milham, A. Di Martino, R. C. Craddock,
D. Samaras, B. Thirion, and G. Varoquaux. Deriving
reproducible biomarkers from multi-site resting-state data: An
autism-based example. NeuroImage, 147:736–745, 2017.
Y. Bengio. Learning deep architectures for ai. Foundations and
trends in Machine Learning, 2:1–127, 2009.
M. Eickenberg, A. Gramfort, G. Varoquaux, and B. Thirion. Seeing
it all: Convolutional network layers map the function of the
human visual system. NeuroImage, 152:184–194, 2017.
J. V. Haxby, I. M. Gobbini, M. L. Furey, ... Distributed and
overlapping representations of faces and objects in ventral
temporal cortex. Science, 293:2425, 2001.
D. H. Hubel and T. N. Wiesel. Receptive ïŹelds, binocular
interaction and functional architecture in the cat’s visual cortex.
The Journal of physiology, 160:106, 1962.
References II
A. G. Huth, S. Nishimoto, A. T. Vu, and J. L. Gallant. A
continuous semantic space describes the representation of
thousands of object and action categories across the human
brain. Neuron, 76:1210, 2012.
K. N. Kay, T. Naselaris, R. J. Prenger, and J. L. Gallant.
Identifying natural images from human brain activity. Nature,
452:352, 2008.
F. Liem, G. Varoquaux, J. Kynast, F. Beyer, S. K. Masouleh, J. M.
Huntenburg, L. Lampe, M. Rahim, A. Abraham, R. C.
Craddock, ... Predicting brain-age from multimodal imaging
data captures cognitive impairment. NeuroImage, 2017.
N. K. Logothetis, J. Pauls, and T. Poggio. Shape representation in
the inferior temporal cortex of monkeys. Current Biology, 5:552,
1995.
References III
A. Mensch, J. Mairal, B. Thirion, and G. Varoquaux. Learning
neural representations of human cognition across many fMRI
studies. In NIPS, 2017.
A. Mensch, J. Mairal, B. Thirion, and G. Varoquaux. Extracting
universal representations of cognition across brain-imaging
studies. In prep, 2018.
A. Newell. You can’t play 20 questions with nature and win:
Projective comments on the papers of this symposium. 1973.
B. Olshausen ... Emergence of simple-cell remainsceptive ïŹeld
properties by learning a sparse code for natural images. Nature,
381:607, 1996.
R. A. Poldrack, Y. O. Halchenko, and S. J. Hanson. Decoding the
large-scale structure of brain function by classifying mental
states across individuals. Psychological Science, 20:1364, 2009.
Y. Schwartz, B. Thirion, and G. Varoquaux. Mapping cognitive
ontologies to and from the brain. In NIPS, 2013.
References IV
G. Varoquaux and R. A. Poldrack. Predictive models can overcome
reductionism in cognitive neuroimaging. In rev, 2018.
G. Varoquaux and B. Thirion. How machine learning is shaping
cognitive neuroimaging. GigaScience, 3:28, 2014.
G. Varoquaux, Y. Schwartz, R. A. Poldrack, B. Gauthier,
D. Bzdok, J. Poline, and B. Thirion. Atlases of cognition with
large-scale brain mapping. In rev, 2018.

Weitere Àhnliche Inhalte

Ähnlich wie Towards psychoinformatics with machine learning and brain imaging

Machine learning and cognitive neuroimaging: new tools can answer new questions
Machine learning and cognitive neuroimaging: new tools can answer new questionsMachine learning and cognitive neuroimaging: new tools can answer new questions
Machine learning and cognitive neuroimaging: new tools can answer new questionsGael Varoquaux
 
Atlases of cognition with large-scale human brain mapping
Atlases of cognition with large-scale human brain mappingAtlases of cognition with large-scale human brain mapping
Atlases of cognition with large-scale human brain mappingGael Varoquaux
 
Brain maps from machine learning? Spatial regularizations
Brain maps from machine learning? Spatial regularizationsBrain maps from machine learning? Spatial regularizations
Brain maps from machine learning? Spatial regularizationsGael Varoquaux
 
Brain reading, compressive sensing, fMRI and statistical learning in Python
Brain reading, compressive sensing, fMRI and statistical learning in PythonBrain reading, compressive sensing, fMRI and statistical learning in Python
Brain reading, compressive sensing, fMRI and statistical learning in PythonGael Varoquaux
 
Python for brain mining: (neuro)science with state of the art machine learnin...
Python for brain mining: (neuro)science with state of the art machine learnin...Python for brain mining: (neuro)science with state of the art machine learnin...
Python for brain mining: (neuro)science with state of the art machine learnin...Gael Varoquaux
 
Can we recover meaning full spatial information from multivariate pattern ana...
Can we recover meaning full spatial information from multivariate pattern ana...Can we recover meaning full spatial information from multivariate pattern ana...
Can we recover meaning full spatial information from multivariate pattern ana...Gael Varoquaux
 
Coding for science and innovation
Coding for science and innovationCoding for science and innovation
Coding for science and innovationGael Varoquaux
 
Succeeding in academia despite doing good_software
Succeeding in academia despite doing good_softwareSucceeding in academia despite doing good_software
Succeeding in academia despite doing good_softwareGael Varoquaux
 
Social-sparsity brain decoders: faster spatial sparsity
Social-sparsity brain decoders: faster spatial sparsitySocial-sparsity brain decoders: faster spatial sparsity
Social-sparsity brain decoders: faster spatial sparsityGael Varoquaux
 
Individual Brain Charting, a high-resolution fMRI dataset for cognitive mappi...
Individual Brain Charting, a high-resolution fMRI dataset for cognitive mappi...Individual Brain Charting, a high-resolution fMRI dataset for cognitive mappi...
Individual Brain Charting, a high-resolution fMRI dataset for cognitive mappi...Ana LuĂ­sa Pinho
 
Simple big data, in Python
Simple big data, in PythonSimple big data, in Python
Simple big data, in PythonGael Varoquaux
 

Ähnlich wie Towards psychoinformatics with machine learning and brain imaging (11)

Machine learning and cognitive neuroimaging: new tools can answer new questions
Machine learning and cognitive neuroimaging: new tools can answer new questionsMachine learning and cognitive neuroimaging: new tools can answer new questions
Machine learning and cognitive neuroimaging: new tools can answer new questions
 
Atlases of cognition with large-scale human brain mapping
Atlases of cognition with large-scale human brain mappingAtlases of cognition with large-scale human brain mapping
Atlases of cognition with large-scale human brain mapping
 
Brain maps from machine learning? Spatial regularizations
Brain maps from machine learning? Spatial regularizationsBrain maps from machine learning? Spatial regularizations
Brain maps from machine learning? Spatial regularizations
 
Brain reading, compressive sensing, fMRI and statistical learning in Python
Brain reading, compressive sensing, fMRI and statistical learning in PythonBrain reading, compressive sensing, fMRI and statistical learning in Python
Brain reading, compressive sensing, fMRI and statistical learning in Python
 
Python for brain mining: (neuro)science with state of the art machine learnin...
Python for brain mining: (neuro)science with state of the art machine learnin...Python for brain mining: (neuro)science with state of the art machine learnin...
Python for brain mining: (neuro)science with state of the art machine learnin...
 
Can we recover meaning full spatial information from multivariate pattern ana...
Can we recover meaning full spatial information from multivariate pattern ana...Can we recover meaning full spatial information from multivariate pattern ana...
Can we recover meaning full spatial information from multivariate pattern ana...
 
Coding for science and innovation
Coding for science and innovationCoding for science and innovation
Coding for science and innovation
 
Succeeding in academia despite doing good_software
Succeeding in academia despite doing good_softwareSucceeding in academia despite doing good_software
Succeeding in academia despite doing good_software
 
Social-sparsity brain decoders: faster spatial sparsity
Social-sparsity brain decoders: faster spatial sparsitySocial-sparsity brain decoders: faster spatial sparsity
Social-sparsity brain decoders: faster spatial sparsity
 
Individual Brain Charting, a high-resolution fMRI dataset for cognitive mappi...
Individual Brain Charting, a high-resolution fMRI dataset for cognitive mappi...Individual Brain Charting, a high-resolution fMRI dataset for cognitive mappi...
Individual Brain Charting, a high-resolution fMRI dataset for cognitive mappi...
 
Simple big data, in Python
Simple big data, in PythonSimple big data, in Python
Simple big data, in Python
 

Mehr von Gael Varoquaux

Evaluating machine learning models and their diagnostic value
Evaluating machine learning models and their diagnostic valueEvaluating machine learning models and their diagnostic value
Evaluating machine learning models and their diagnostic valueGael Varoquaux
 
Measuring mental health with machine learning and brain imaging
Measuring mental health with machine learning and brain imagingMeasuring mental health with machine learning and brain imaging
Measuring mental health with machine learning and brain imagingGael Varoquaux
 
Machine learning with missing values
Machine learning with missing valuesMachine learning with missing values
Machine learning with missing valuesGael Varoquaux
 
Dirty data science machine learning on non-curated data
Dirty data science machine learning on non-curated dataDirty data science machine learning on non-curated data
Dirty data science machine learning on non-curated dataGael Varoquaux
 
Representation learning in limited-data settings
Representation learning in limited-data settingsRepresentation learning in limited-data settings
Representation learning in limited-data settingsGael Varoquaux
 
Better neuroimaging data processing: driven by evidence, open communities, an...
Better neuroimaging data processing: driven by evidence, open communities, an...Better neuroimaging data processing: driven by evidence, open communities, an...
Better neuroimaging data processing: driven by evidence, open communities, an...Gael Varoquaux
 
Similarity encoding for learning on dirty categorical variables
Similarity encoding for learning on dirty categorical variablesSimilarity encoding for learning on dirty categorical variables
Similarity encoding for learning on dirty categorical variablesGael Varoquaux
 
Simple representations for learning: factorizations and similarities
Simple representations for learning: factorizations and similarities Simple representations for learning: factorizations and similarities
Simple representations for learning: factorizations and similarities Gael Varoquaux
 
A tutorial on Machine Learning, with illustrations for MR imaging
A tutorial on Machine Learning, with illustrations for MR imagingA tutorial on Machine Learning, with illustrations for MR imaging
A tutorial on Machine Learning, with illustrations for MR imagingGael Varoquaux
 
Scikit-learn and nilearn: Democratisation of machine learning for brain imaging
Scikit-learn and nilearn: Democratisation of machine learning for brain imagingScikit-learn and nilearn: Democratisation of machine learning for brain imaging
Scikit-learn and nilearn: Democratisation of machine learning for brain imagingGael Varoquaux
 
Computational practices for reproducible science
Computational practices for reproducible scienceComputational practices for reproducible science
Computational practices for reproducible scienceGael Varoquaux
 
Estimating Functional Connectomes: Sparsity’s Strength and Limitations
Estimating Functional Connectomes: Sparsity’s Strength and LimitationsEstimating Functional Connectomes: Sparsity’s Strength and Limitations
Estimating Functional Connectomes: Sparsity’s Strength and LimitationsGael Varoquaux
 
On the code of data science
On the code of data scienceOn the code of data science
On the code of data scienceGael Varoquaux
 
Scientist meets web dev: how Python became the language of data
Scientist meets web dev: how Python became the language of dataScientist meets web dev: how Python became the language of data
Scientist meets web dev: how Python became the language of dataGael Varoquaux
 
Scikit-learn: the state of the union 2016
Scikit-learn: the state of the union 2016Scikit-learn: the state of the union 2016
Scikit-learn: the state of the union 2016Gael Varoquaux
 
Inter-site autism biomarkers from resting state fMRI
Inter-site autism biomarkers from resting state fMRIInter-site autism biomarkers from resting state fMRI
Inter-site autism biomarkers from resting state fMRIGael Varoquaux
 
Scikit-learn for easy machine learning: the vision, the tool, and the project
Scikit-learn for easy machine learning: the vision, the tool, and the projectScikit-learn for easy machine learning: the vision, the tool, and the project
Scikit-learn for easy machine learning: the vision, the tool, and the projectGael Varoquaux
 
Building a cutting-edge data processing environment on a budget
Building a cutting-edge data processing environment on a budgetBuilding a cutting-edge data processing environment on a budget
Building a cutting-edge data processing environment on a budgetGael Varoquaux
 
Scikit-learn: apprentissage statistique en Python. Créer des machines intelli...
Scikit-learn: apprentissage statistique en Python. Créer des machines intelli...Scikit-learn: apprentissage statistique en Python. Créer des machines intelli...
Scikit-learn: apprentissage statistique en Python. Créer des machines intelli...Gael Varoquaux
 
Open Source Scientific Software
Open Source Scientific SoftwareOpen Source Scientific Software
Open Source Scientific SoftwareGael Varoquaux
 

Mehr von Gael Varoquaux (20)

Evaluating machine learning models and their diagnostic value
Evaluating machine learning models and their diagnostic valueEvaluating machine learning models and their diagnostic value
Evaluating machine learning models and their diagnostic value
 
Measuring mental health with machine learning and brain imaging
Measuring mental health with machine learning and brain imagingMeasuring mental health with machine learning and brain imaging
Measuring mental health with machine learning and brain imaging
 
Machine learning with missing values
Machine learning with missing valuesMachine learning with missing values
Machine learning with missing values
 
Dirty data science machine learning on non-curated data
Dirty data science machine learning on non-curated dataDirty data science machine learning on non-curated data
Dirty data science machine learning on non-curated data
 
Representation learning in limited-data settings
Representation learning in limited-data settingsRepresentation learning in limited-data settings
Representation learning in limited-data settings
 
Better neuroimaging data processing: driven by evidence, open communities, an...
Better neuroimaging data processing: driven by evidence, open communities, an...Better neuroimaging data processing: driven by evidence, open communities, an...
Better neuroimaging data processing: driven by evidence, open communities, an...
 
Similarity encoding for learning on dirty categorical variables
Similarity encoding for learning on dirty categorical variablesSimilarity encoding for learning on dirty categorical variables
Similarity encoding for learning on dirty categorical variables
 
Simple representations for learning: factorizations and similarities
Simple representations for learning: factorizations and similarities Simple representations for learning: factorizations and similarities
Simple representations for learning: factorizations and similarities
 
A tutorial on Machine Learning, with illustrations for MR imaging
A tutorial on Machine Learning, with illustrations for MR imagingA tutorial on Machine Learning, with illustrations for MR imaging
A tutorial on Machine Learning, with illustrations for MR imaging
 
Scikit-learn and nilearn: Democratisation of machine learning for brain imaging
Scikit-learn and nilearn: Democratisation of machine learning for brain imagingScikit-learn and nilearn: Democratisation of machine learning for brain imaging
Scikit-learn and nilearn: Democratisation of machine learning for brain imaging
 
Computational practices for reproducible science
Computational practices for reproducible scienceComputational practices for reproducible science
Computational practices for reproducible science
 
Estimating Functional Connectomes: Sparsity’s Strength and Limitations
Estimating Functional Connectomes: Sparsity’s Strength and LimitationsEstimating Functional Connectomes: Sparsity’s Strength and Limitations
Estimating Functional Connectomes: Sparsity’s Strength and Limitations
 
On the code of data science
On the code of data scienceOn the code of data science
On the code of data science
 
Scientist meets web dev: how Python became the language of data
Scientist meets web dev: how Python became the language of dataScientist meets web dev: how Python became the language of data
Scientist meets web dev: how Python became the language of data
 
Scikit-learn: the state of the union 2016
Scikit-learn: the state of the union 2016Scikit-learn: the state of the union 2016
Scikit-learn: the state of the union 2016
 
Inter-site autism biomarkers from resting state fMRI
Inter-site autism biomarkers from resting state fMRIInter-site autism biomarkers from resting state fMRI
Inter-site autism biomarkers from resting state fMRI
 
Scikit-learn for easy machine learning: the vision, the tool, and the project
Scikit-learn for easy machine learning: the vision, the tool, and the projectScikit-learn for easy machine learning: the vision, the tool, and the project
Scikit-learn for easy machine learning: the vision, the tool, and the project
 
Building a cutting-edge data processing environment on a budget
Building a cutting-edge data processing environment on a budgetBuilding a cutting-edge data processing environment on a budget
Building a cutting-edge data processing environment on a budget
 
Scikit-learn: apprentissage statistique en Python. Créer des machines intelli...
Scikit-learn: apprentissage statistique en Python. Créer des machines intelli...Scikit-learn: apprentissage statistique en Python. Créer des machines intelli...
Scikit-learn: apprentissage statistique en Python. Créer des machines intelli...
 
Open Source Scientific Software
Open Source Scientific SoftwareOpen Source Scientific Software
Open Source Scientific Software
 

KĂŒrzlich hochgeladen

Pests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdfPests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdfPirithiRaju
 
❀Jammu Kashmir Call Girls 8617697112 Personal Whatsapp Number 💩✅.
❀Jammu Kashmir Call Girls 8617697112 Personal Whatsapp Number 💩✅.❀Jammu Kashmir Call Girls 8617697112 Personal Whatsapp Number 💩✅.
❀Jammu Kashmir Call Girls 8617697112 Personal Whatsapp Number 💩✅.Nitya salvi
 
GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)Areesha Ahmad
 
Forensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdfForensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdfrohankumarsinghrore1
 
Botany 4th semester file By Sumit Kumar yadav.pdf
Botany 4th semester file By Sumit Kumar yadav.pdfBotany 4th semester file By Sumit Kumar yadav.pdf
Botany 4th semester file By Sumit Kumar yadav.pdfSumit Kumar yadav
 
Chemical Tests; flame test, positive and negative ions test Edexcel Internati...
Chemical Tests; flame test, positive and negative ions test Edexcel Internati...Chemical Tests; flame test, positive and negative ions test Edexcel Internati...
Chemical Tests; flame test, positive and negative ions test Edexcel Internati...ssuser79fe74
 
Kochi ❀CALL GIRL 84099*07087 ❀CALL GIRLS IN Kochi ESCORT SERVICE❀CALL GIRL
Kochi ❀CALL GIRL 84099*07087 ❀CALL GIRLS IN Kochi ESCORT SERVICE❀CALL GIRLKochi ❀CALL GIRL 84099*07087 ❀CALL GIRLS IN Kochi ESCORT SERVICE❀CALL GIRL
Kochi ❀CALL GIRL 84099*07087 ❀CALL GIRLS IN Kochi ESCORT SERVICE❀CALL GIRLkantirani197
 
Chemistry 4th semester series (krishna).pdf
Chemistry 4th semester series (krishna).pdfChemistry 4th semester series (krishna).pdf
Chemistry 4th semester series (krishna).pdfSumit Kumar yadav
 
SAMASTIPUR CALL GIRL 7857803690 LOW PRICE ESCORT SERVICE
SAMASTIPUR CALL GIRL 7857803690  LOW PRICE  ESCORT SERVICESAMASTIPUR CALL GIRL 7857803690  LOW PRICE  ESCORT SERVICE
SAMASTIPUR CALL GIRL 7857803690 LOW PRICE ESCORT SERVICEayushi9330
 
Site Acceptance Test .
Site Acceptance Test                    .Site Acceptance Test                    .
Site Acceptance Test .Poonam Aher Patil
 
COST ESTIMATION FOR A RESEARCH PROJECT.pptx
COST ESTIMATION FOR A RESEARCH PROJECT.pptxCOST ESTIMATION FOR A RESEARCH PROJECT.pptx
COST ESTIMATION FOR A RESEARCH PROJECT.pptxFarihaAbdulRasheed
 
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...ssifa0344
 
GBSN - Microbiology (Unit 2)
GBSN - Microbiology (Unit 2)GBSN - Microbiology (Unit 2)
GBSN - Microbiology (Unit 2)Areesha Ahmad
 
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticsPulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticssakshisoni2385
 
Zoology 4th semester series (krishna).pdf
Zoology 4th semester series (krishna).pdfZoology 4th semester series (krishna).pdf
Zoology 4th semester series (krishna).pdfSumit Kumar yadav
 
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Servicenishacall1
 
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...Monika Rani
 
High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑
High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑
High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑Damini Dixit
 
Formation of low mass protostars and their circumstellar disks
Formation of low mass protostars and their circumstellar disksFormation of low mass protostars and their circumstellar disks
Formation of low mass protostars and their circumstellar disksSĂ©rgio Sacani
 
Presentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptxPresentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptxgindu3009
 

KĂŒrzlich hochgeladen (20)

Pests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdfPests of cotton_Sucking_Pests_Dr.UPR.pdf
Pests of cotton_Sucking_Pests_Dr.UPR.pdf
 
❀Jammu Kashmir Call Girls 8617697112 Personal Whatsapp Number 💩✅.
❀Jammu Kashmir Call Girls 8617697112 Personal Whatsapp Number 💩✅.❀Jammu Kashmir Call Girls 8617697112 Personal Whatsapp Number 💩✅.
❀Jammu Kashmir Call Girls 8617697112 Personal Whatsapp Number 💩✅.
 
GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)GBSN - Biochemistry (Unit 1)
GBSN - Biochemistry (Unit 1)
 
Forensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdfForensic Biology & Its biological significance.pdf
Forensic Biology & Its biological significance.pdf
 
Botany 4th semester file By Sumit Kumar yadav.pdf
Botany 4th semester file By Sumit Kumar yadav.pdfBotany 4th semester file By Sumit Kumar yadav.pdf
Botany 4th semester file By Sumit Kumar yadav.pdf
 
Chemical Tests; flame test, positive and negative ions test Edexcel Internati...
Chemical Tests; flame test, positive and negative ions test Edexcel Internati...Chemical Tests; flame test, positive and negative ions test Edexcel Internati...
Chemical Tests; flame test, positive and negative ions test Edexcel Internati...
 
Kochi ❀CALL GIRL 84099*07087 ❀CALL GIRLS IN Kochi ESCORT SERVICE❀CALL GIRL
Kochi ❀CALL GIRL 84099*07087 ❀CALL GIRLS IN Kochi ESCORT SERVICE❀CALL GIRLKochi ❀CALL GIRL 84099*07087 ❀CALL GIRLS IN Kochi ESCORT SERVICE❀CALL GIRL
Kochi ❀CALL GIRL 84099*07087 ❀CALL GIRLS IN Kochi ESCORT SERVICE❀CALL GIRL
 
Chemistry 4th semester series (krishna).pdf
Chemistry 4th semester series (krishna).pdfChemistry 4th semester series (krishna).pdf
Chemistry 4th semester series (krishna).pdf
 
SAMASTIPUR CALL GIRL 7857803690 LOW PRICE ESCORT SERVICE
SAMASTIPUR CALL GIRL 7857803690  LOW PRICE  ESCORT SERVICESAMASTIPUR CALL GIRL 7857803690  LOW PRICE  ESCORT SERVICE
SAMASTIPUR CALL GIRL 7857803690 LOW PRICE ESCORT SERVICE
 
Site Acceptance Test .
Site Acceptance Test                    .Site Acceptance Test                    .
Site Acceptance Test .
 
COST ESTIMATION FOR A RESEARCH PROJECT.pptx
COST ESTIMATION FOR A RESEARCH PROJECT.pptxCOST ESTIMATION FOR A RESEARCH PROJECT.pptx
COST ESTIMATION FOR A RESEARCH PROJECT.pptx
 
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...
TEST BANK For Radiologic Science for Technologists, 12th Edition by Stewart C...
 
GBSN - Microbiology (Unit 2)
GBSN - Microbiology (Unit 2)GBSN - Microbiology (Unit 2)
GBSN - Microbiology (Unit 2)
 
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceuticsPulmonary drug delivery system M.pharm -2nd sem P'ceutics
Pulmonary drug delivery system M.pharm -2nd sem P'ceutics
 
Zoology 4th semester series (krishna).pdf
Zoology 4th semester series (krishna).pdfZoology 4th semester series (krishna).pdf
Zoology 4th semester series (krishna).pdf
 
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
9999266834 Call Girls In Noida Sector 22 (Delhi) Call Girl Service
 
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
Vip profile Call Girls In Lonavala 9748763073 For Genuine Sex Service At Just...
 
High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑
High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑
High Profile 🔝 8250077686 📞 Call Girls Service in GTB Nagar🍑
 
Formation of low mass protostars and their circumstellar disks
Formation of low mass protostars and their circumstellar disksFormation of low mass protostars and their circumstellar disks
Formation of low mass protostars and their circumstellar disks
 
Presentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptxPresentation Vikram Lander by Vedansh Gupta.pptx
Presentation Vikram Lander by Vedansh Gupta.pptx
 

Towards psychoinformatics with machine learning and brain imaging

  • 1. Towards psychoinformatics with machine learning and brain imaging GaĂ«l Varoquaux
  • 2. Towards psychoinformatics with machine learning and brain imaging GaĂ«l Varoquaux Propose and illustrate a research program
  • 3. Accumulation of functional brain data G Varoquaux 2
  • 4. Theories of the mind Laws of the mind for perception, decision, action, emotion... G Varoquaux 3
  • 5. Studying mental processes 1 Craft an experimental condition that recruits it G Varoquaux 4
  • 6. Studying mental processes 1 Craft an experimental condition that recruits it 2 Do an elementary psychological manipulation G Varoquaux 4
  • 7. Studying mental processes - -Results in a contrast Study by oppositions G Varoquaux 4
  • 8. Studying mental processes - -Results in a contrast Study by oppositions Results tied to a simple psychological manipulation bound to a paradigm G Varoquaux 4
  • 9. Proposing Generalization to build broader theories bridging paradigms Paradigm 1: Seen G Varoquaux 5
  • 10. Proposing Generalization to build broader theories bridging paradigms Paradigm 1: Seen Paradigm 2: Imagined G Varoquaux 5
  • 11. Generalization for broader theories bridging paradigms Seen objects G Varoquaux 6
  • 12. Brain imaging ⇒ brain-mind associations DiïŹƒculty: modeling behavior & cognition G Varoquaux 7
  • 13. Predictive models can broaden theories by generalizing brain-mind associations to arbitrary new tasks and stimuli [Varoquaux and Poldrack 2018] 1 Beyond oppositions: encoding 2 Generalizing across tasks 3 Universal cognitive representations 4 Across subjects: biomarkers G Varoquaux 8
  • 14. 1 Beyond oppositions: encoding [Eickenberg... 2017] G Varoquaux 9
  • 15. 1 Decomposing psychological process To study vision: Breaking down stimuli G Varoquaux 10
  • 16. 1 Decomposing psychological process To study vision: Breaking down stimuli [Hubel and Wiesel 1962] Neurons receptive to Gabors (edges) G Varoquaux 10
  • 17. 1 Decomposing psychological process To study vision: Breaking down stimuli [Hubel and Wiesel 1962] Neurons receptive to Gabors (edges) [Logothetis... 1995] Shapes in inferior temporal cortex G Varoquaux 10
  • 18. 1 Decomposing psychological process To study vision: Breaking down stimuli Image V1 cortex V2 cortex Inferior temporal cortex Fusiform face area Jack? Is there a “face” region? A “foot” region? A “left big toe” region? G Varoquaux 10
  • 19. 1 Crafting stimuli for cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs G Varoquaux 11
  • 20. 1 Crafting stimuli for cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs G Varoquaux 11
  • 21. 1 Crafting stimuli for cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs - G Varoquaux 11
  • 22. 1 Crafting stimuli for cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs - Representing only one aspect of the stimuli: too much reductionism G Varoquaux 11
  • 23. 1 Not crafting stimuli for cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs - Representing only one aspect of the stimuli: too much reductionism Encoding Building complex representations of stimuli Predicting brain response from them G Varoquaux 11
  • 24. 1 Decomposing visual stimuli Low-level visual cortex is tuned to natural image statistics [Olshausen et al. 1996] What drives high-level representations? The concept of “left big toe”? G Varoquaux 12
  • 25. 1 Decomposing visual stimuli Low-level visual cortex is tuned to natural image statistics [Olshausen et al. 1996] What drives high-level representations? Convolutional Net G Varoquaux 12
  • 26. 1 Brain mapping: encoding with conv nets StimulirepresentationBrainactivity Input Layer 1 Layer 5 Linear predictive models Convolutional Net G Varoquaux 13
  • 27. 1 Brain mapping: encoding with conv nets StimulirepresentationBrainactivity Input Layer 1 Layer 5 Linear predictive models Convolutional Net Explains the brain activity from the stimuli much better than hand-crafted features On data from [Kay... 2008] natural images [Huth... 2012] movies [Eickenberg... 2017] G Varoquaux 13
  • 28. 1 Brain mapping: encoding with conv nets [Eickenberg... 2017] High-level conv-net layer map to high-level visual areas G Varoquaux 14
  • 29. 1 Brain mapping with conv nets: retinotopy Mapping response to exentricity: artiïŹcal retinotopy G Varoquaux 15
  • 30. 1 Brain mapping with conv nets: high-level concepts Opposing face versus place in the [Haxby... 2001] stimuli G Varoquaux 16
  • 31. [Eickenberg... 2017] Beyond oppositions: encoding Recovers the hierarchy of visual modules Generalizes from natural images to retinotopy or face vs place G Varoquaux 17
  • 32. 2 Generalizing across tasks Characterizing the function of brain structures How much is observed activity a consequence of speciïŹcities of the paradigm? [Schwartz... 2013, Varoquaux... 2018] G Varoquaux 18
  • 33. 2 Large-scale decoding for reverse inference Cat Dog Ouaf Miaou [Poldrack... 2009] G Varoquaux 19
  • 34. 2 Large-scale decoding for reverse inference Cat Dog Ouaf Miaou [Poldrack... 2009] Variability: an opportunity and a challenge Technical heterogeneity (scanner, stimulus modality...) Paradigmatic isolation (cognition) [Newell 1973] G Varoquaux 19
  • 35. 2 Generalizing to arbitrary paradigms [Schwartz... 2013, Varoquaux... 2018] What is this brain doing? Describe tasks by their cognitive components Multi-label prediction: presence or absence of each label Visual Auditory Foot Hand Calculation Reading Checkboard Face Place Object Digit Saccade ... ... ... ... ... ... G Varoquaux 20
  • 36. 2 Generalizing to arbitrary paradigms [Schwartz... 2013, Varoquaux... 2018] What is this brain doing? Describe tasks by their cognitive components Multi-label prediction: presence or absence of each label Visual Auditory Checkboard Face ... ... ... ... Prediction across studies Describe a task never seen from the brain activity it evokes G Varoquaux 20
  • 37. 2 Generalizing to arbitrary paradigms [Schwartz... 2013, Varoquaux... 2018] What is this brain doing? + + Regions speciïŹc to that predict facets of cognition G Varoquaux 20
  • 38. 2 Decoding mega-analysis [Schwartz... 2013, Varoquaux... 2018] 30 studies 837 subjects 196 experimental conditions 6919 activation maps DiïŹ€erent labs Dehaene Poldrack Wager ... Various cognitive domains Language Vision Decision making Mathematics ... All manually preprocessed, labeled, and curated G Varoquaux 21
  • 39. 2 Decoding mega-analysis [Schwartz... 2013, Varoquaux... 2018] 30 studies 837 subjects 196 experimental conditions 6919 activation maps DiïŹ€erent labs Dehaene Poldrack Wager ... Various cognitive domains Language Vision Decision making Mathematics ... All manually preprocessed, labeled, and curated Decoding arbitrary new paradigms visualauditoryVcheckerboard Hcheckerboardobjectsscramblefacesplacesdigits wordsrighthandlefthandrightfootleftfootlanguagehumansound non-humansound 0.2 0.4 0.6 0.8 1.0 Predictionscore (ROCAUC) ontologydecoder logisticregression naiveBayes neurosynth ­0.3 ­0.2 ­0.1 0.0 +0.1 +0.2 +0.3 Relative score G Varoquaux 21
  • 40. 2 Mapping: regions speciïŹc to facets of cognition [Schwartz... 2013, Varoquaux... 2018] Contrasts: good rejection of confounds Decoding: reverse inference ⇒ Consensus to deïŹne regions The “place” concept Forward term eïŹ€ect Brain response to tasks with the “place” concept Forward ontology contrast Contrasting responses to “place” and related concepts Reverse decoder map Regions that predict a place- recognition task Consensus forward & reverse Regions selected by forward and reverse inference Replaces the careful crafting of control conditions G Varoquaux 22
  • 41. 2 A functional atlas: central gyrus, motor networks [Schwartz... 2013, Varoquaux... 2018] Neurosynth Reverse inference (Logistic regression) Decoding in an ontology z = -15 Reverse inference -15 nference DecodDecoding in an visual faces places objects scrambled words digits horizontal - vertical checkerboard Visual regions Contrasts -based analysis Neurosynth reverse inference Decoding and ontology + Language and auditory mapping in temporal cortex + Calculation and spatial attenation in IPS + Mapping the motor system G Varoquaux 23
  • 42. [Varoquaux... 2018] Generalizing across tasks Cat Dog Ouaf Miaou Multi-label prediction Prediction (discriminative models) more robust to heterogeneity Decoding ⇒ evidence beyond ⇒ a paradigm Atlasing cognition G Varoquaux 24
  • 43. Need an ontology of cognition Episodic memory working memory lexical memory Syntax comprehension Reading Mental calculation Shape recognition Face recognition Memory Language Vision G Varoquaux 25
  • 44. 3 Universal cognitive representations [Mensch... 2017] Face Shape Reading Shape Language Music Saccades Calculation G Varoquaux 26
  • 45. 3 Universal cognitive representations [Mensch... 2017] Face Shape Reading Shape Language Music Saccades CalculationMulti-task learning Solve many related but diïŹ€erent problems Learn commonalities intermediate representations G Varoquaux 26
  • 46. 3 Deep architecture [Bengio 2009] ... Great for multiple output (tasks) G Varoquaux 27
  • 47. 3 Deep architecture [Bengio 2009] ... Great for multiple output (tasks) Millions of parameters, thousands of data points G Varoquaux 27
  • 48. 3 Deep architecture 2 2 Great for multiple output (tasks) Millions of parameters, thousands of data points Simplify G Varoquaux 27
  • 49. 3 Shallow architecture 2 linear model Great for multiple output (tasks) Millions of parameters, thousands of data points Simplify simplify more G Varoquaux 27
  • 50. 3 Shallow architecture 2 2 Great for multiple output (tasks) Millions of parameters, thousands of data points Simplify simplify more G Varoquaux 27
  • 51. 3 Deep linear model – multi-task – Bayesian 4TB resting-state data HCP900 OpenfMRI HCP Camcan Brainomics 42000 task fMRI contrast maps for one model ... Many rest & task fMRI studies ... x Information ïŹ‚ow x Learned network combination 1st layer assignment Non-negative matrix factorization SUCCESSFUL GO HOUSE HORIZONTAL CHECKERBOARD Study-wise decoding REWARD BALLOON EXPLODE ............ Deep linear model Joint training Bayesian deep learning Resting-state functional loadings 512 resting-state networks Task functional loadings 128 task-optimized networks Contrast maps 200,000 voxels Contrast name Probability vector FACE BODY HOUSE TOOL 512 components [Mensch... 2018] G Varoquaux 28
  • 52. 3 Deep linear model – multi-task – Bayesian OpenfMRI HCP Camcan Brainomics ... ... Information ïŹ‚ow Non-negative matrix factorization SUCCESSFUL GO HOUSE HORIZONTAL CHECKERBOARD Study-wise decoding REWARD BALLOON EXPLODE ............ Deep linear model Task functional loadings 128 task-optimized networks [Mensch... 2018] Intermediate representation: Task-optimized networks G Varoquaux 28
  • 53. 3 Deep linear model – multi-task – Bayesian OpenfMRI HCP Camcan Brainomics ... ... Information ïŹ‚ow Non-negative matrix factorization SUCCESSFUL GO HOUSE HORIZONTAL CHECKERBOARD Study-wise decoding REWARD BALLOON EXPLODE ............ Deep linear model Task functional loadings 128 task-optimized networks [Mensch... 2018] Intermediate representation: Task-optimized networks Improves decoding in each study 0% 20% 40% 60% 80% Decoding accuracy on test set Stop-signal w/ spoken & manual resp. The Human Connectome Project UCLA LA5C Simon task Plain or mirror-reversed text Stop-signal CamCan audio-visual Localizer Cross-language repetition priming Twin localizer High-level math Spatio-temporal judgement (retake) Constit. struct. of sent. & music Sentence/music complexity Word & object processing Emotion regulation Arithmetic & saccades Brainomics localizer Incidental encoding Spatio-temporal judgement False belief Visual object recognition Face recognition Stop-signal & classification Classification learning Compression Motor task & word/verb generation BART, stop-signal, emotion Rhyme judgment Auditory & Visual Oddball Weather prediction Stop-signal & classification (retake) Balloon Analog Risk-taking Mixed-gambles Classif. learning & reversal Task fMRI study Decoding from voxels Decoding from multi-study networks Chance level -5% 0% 5%10%15% Multi-study acc. gain G Varoquaux 28
  • 54. 3 Deep linear model – multi-task – Bayesian OpenfMRI HCP Camcan Brainomics ... ... Information ïŹ‚ow Non-negative matrix factorization SUCCESSFUL GO HOUSE HORIZONTAL CHECKERBOARD Study-wise decoding REWARD BALLOON EXPLODE ............ Deep linear model Task functional loadings 128 task-optimized networks [Mensch... 2018] Intermediate representation: Task-optimized networks Improves decoding in each study 4 16 32 100 400 Number of train subjects 0% 5% 10% 15% 20% Multistudyacc.gain Study Small studies beneïŹt from large ones G Varoquaux 28
  • 55. 3 Deep linear model – multi-task – Bayesian OpenfMRI HCP Camcan Brainomics ... ... Information ïŹ‚ow Non-negative matrix factorization SUCCESSFUL GO HOUSE HORIZONTAL CHECKERBOARD Study-wise decoding REWARD BALLOON EXPLODE ............ Deep linear model Task functional loadings 128 task-optimized networks [Mensch... 2018] Intermediate representation: Task-optimized networks Enables smaller sample sizes 7 15 23 31 39 # training subjects in target study 70% 75% 80% 85% Testaccuracy Pinel et al. '07 9 18 28 37 47 70% 80% 90% Papadopoulos- Orfanos '12 60 12 60% 65% C (Sh 37 47 oulos- '12 60 121 181 242 302 60% 65% CamCan AV (Shafto et al. '14) Standard decoding Multi-study decoder 2 4 7 9 12 20% 40% 60% Cohen '09 G Varoquaux 28
  • 56. 3 Deep linear model – multi-task – Bayesian OpenfMRI HCP Camcan Brainomics ... ... Information ïŹ‚ow Non-negative matrix factorization SUCCESSFUL GO HOUSE HORIZONTAL CHECKERBOARD Study-wise decoding REWARD BALLOON EXPLODE ............ Deep linear model Task functional loadings 128 task-optimized networks [Mensch... 2018] Intermediate representation: Task-optimized networks G Varoquaux 28
  • 57. 3 Deep linear model – multi-task – Bayesian OpenfMRI HCP Camcan Brainomics ... ... Information ïŹ‚ow Non-negative matrix factorization SUCCESSFUL GO HOUSE HORIZONTAL CHECKERBOARD Study-wise decoding REWARD BALLOON EXPLODE ............ Deep linear model Task functional loadings 128 task-optimized networks [Mensch... 2018] Intermediate representation: Task-optimized networks G Varoquaux 28
  • 58. 3 Deep linear model – multi-task – Bayesian OpenfMRI HCP Camcan Brainomics ... ... Information ïŹ‚ow Non-negative matrix factorization SUCCESSFUL GO HOUSE HORIZONTAL CHECKERBOARD Study-wise decoding REWARD BALLOON EXPLODE ............ Deep linear model Task functional loadings 128 task-optimized networks [Mensch... 2018] Intermediate representation: Task-optimized networks G Varoquaux 28
  • 59. [Mensch... 2017] Universal cognitive representations Multi-task decoding: - decoding the original task labels Deep linear models: - intermediate representations across studies - statistical power from big to small studies Face Shape Reading Shape Language Music Saccades Calculation G Varoquaux 29
  • 60. [Mensch... 2017] Universal cognitive representations Multi-task decoding: - decoding the original task labels Deep linear models: - intermediate representations across studies - statistical power from big to small studies Face Shape Reading Shape Language Music Saccades Calculation G Varoquaux 29
  • 61. 4 Across subjects: biomarkers G Varoquaux 30
  • 62. 4 Beyond heterogeneity: predicting autism across sites Accuracy Fraction of subjects used More data is better (up to 1000 subjects) [Abraham... 2017] G Varoquaux 31
  • 63. 4 Brain aging: a surrogate biomarker [Liem... 2017] Predicting brain aging = chronological age Multi-modal: brain connectivity and morphology Age prediction: mean absolute error of 4.3 years G Varoquaux 32
  • 64. 4 Brain aging: a surrogate biomarker [Liem... 2017] Predicting brain aging = chronological age Multi-modal: brain connectivity and morphology Age prediction: mean absolute error of 4.3 years Discrepency with chronological age correlates with cognitive impairment 0 2 4 Brain aging discrepancy (years) -0.38 0.74 1.72 Objective Cognitive Impairment group Normal Mild Major G Varoquaux 32
  • 65. @GaelVaroquaux Psychoinformatics with machine learning Prediction for broader theories AI to model stimuli / the world Explicit generalization across paradigms Cat Dog Ouaf Miaou
  • 66. @GaelVaroquaux Psychoinformatics with machine learning Prediction for broader theories AI to model stimuli / the world Explicit generalization across paradigms Beyond oppositions Encoding complete descriptions of tasks Decoding multiple facets of cognitions
  • 67. @GaelVaroquaux Psychoinformatics with machine learning Prediction for broader theories AI to model stimuli / the world Explicit generalization across paradigms Beyond oppositions Encoding complete descriptions of tasks Decoding multiple facets of cognitions Useful with imperfect labels Extracting common representations Surrogate biomarkers
  • 68. @GaelVaroquaux Psychoinformatics with machine learning Prediction for broader theories AI to model stimuli / the world Explicit generalization across paradigms Beyond oppositions Encoding complete descriptions of tasks Decoding multiple facets of cognitions Useful with imperfect labels Extracting common representations Surrogate biomarkers Software: nilearn http://nilearn.github.io ni
  • 69. References I A. Abraham, M. P. Milham, A. Di Martino, R. C. Craddock, D. Samaras, B. Thirion, and G. Varoquaux. Deriving reproducible biomarkers from multi-site resting-state data: An autism-based example. NeuroImage, 147:736–745, 2017. Y. Bengio. Learning deep architectures for ai. Foundations and trends in Machine Learning, 2:1–127, 2009. M. Eickenberg, A. Gramfort, G. Varoquaux, and B. Thirion. Seeing it all: Convolutional network layers map the function of the human visual system. NeuroImage, 152:184–194, 2017. J. V. Haxby, I. M. Gobbini, M. L. Furey, ... Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293:2425, 2001. D. H. Hubel and T. N. Wiesel. Receptive ïŹelds, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of physiology, 160:106, 1962.
  • 70. References II A. G. Huth, S. Nishimoto, A. T. Vu, and J. L. Gallant. A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron, 76:1210, 2012. K. N. Kay, T. Naselaris, R. J. Prenger, and J. L. Gallant. Identifying natural images from human brain activity. Nature, 452:352, 2008. F. Liem, G. Varoquaux, J. Kynast, F. Beyer, S. K. Masouleh, J. M. Huntenburg, L. Lampe, M. Rahim, A. Abraham, R. C. Craddock, ... Predicting brain-age from multimodal imaging data captures cognitive impairment. NeuroImage, 2017. N. K. Logothetis, J. Pauls, and T. Poggio. Shape representation in the inferior temporal cortex of monkeys. Current Biology, 5:552, 1995.
  • 71. References III A. Mensch, J. Mairal, B. Thirion, and G. Varoquaux. Learning neural representations of human cognition across many fMRI studies. In NIPS, 2017. A. Mensch, J. Mairal, B. Thirion, and G. Varoquaux. Extracting universal representations of cognition across brain-imaging studies. In prep, 2018. A. Newell. You can’t play 20 questions with nature and win: Projective comments on the papers of this symposium. 1973. B. Olshausen ... Emergence of simple-cell remainsceptive ïŹeld properties by learning a sparse code for natural images. Nature, 381:607, 1996. R. A. Poldrack, Y. O. Halchenko, and S. J. Hanson. Decoding the large-scale structure of brain function by classifying mental states across individuals. Psychological Science, 20:1364, 2009. Y. Schwartz, B. Thirion, and G. Varoquaux. Mapping cognitive ontologies to and from the brain. In NIPS, 2013.
  • 72. References IV G. Varoquaux and R. A. Poldrack. Predictive models can overcome reductionism in cognitive neuroimaging. In rev, 2018. G. Varoquaux and B. Thirion. How machine learning is shaping cognitive neuroimaging. GigaScience, 3:28, 2014. G. Varoquaux, Y. Schwartz, R. A. Poldrack, B. Gauthier, D. Bzdok, J. Poline, and B. Thirion. Atlases of cognition with large-scale brain mapping. In rev, 2018.