The talk I held on the international multisensory research forum 2016, Suzhou, China.
In a couple of sentences:
Are there (at least) two domains of short-term memory? Spatial and verbal or are there only one? That is, do we hold verbal information (e.g., numbers) and spatial (spaces in our environment) differently?
If there are two domains a spatial change should only affect spatial memory.
However, if there are one domain both verbal and spatial short-term memory should be affected.
Using a distraction paradigm we examined this by at some times change the location of a vibration and sound (from one side of the body to the other).
It did affect both tasks and our results suggest that there are one domain.
Digitized Continuous Magnetic Recordings for the August/September 1859 Storms...
Spatial Change in Multisensory Distractors Impact on Spatial and Verbal Short-term memroy
1. Spatial Change in Multisensory
Distractors Impact on Spatial
and Verbal Short-term Memory
Erik Marsja1
, John Everett Marsh2
, Gregory Neely1
, Patrik
Hanson1
, Jessica K-Ljungberg1,3
1
Department of Psychology, Umeå University, Sweden
2
School of Psychology, University of Central Lancashire, UK
3
School of Psychology, Cardiff University, UK
Contact: erik.marsja at umu.se, erik at marsja.se, http://www.marsja.se
2. Domains of short-term memory
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 2
3. Distraction & STM
• Irrelevant sounds makes us forget
• Changing-state vs Steady-state (see Dalton & Hughes,
2014):
–”A-B-A-B-A-B-A-B-A” vs. ”A-A-A-A-A-A-A-A”
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 3
4. Distraction & STM
• Sudden and unexpected changes (i.e.,
deviants; e.g., ”a-a-a-b-a-a-a”)
• Temporal deviants (Hughes et al., 2005)
• Change in voice (i.e., from male to female; Hughes et al., 2007)
• Change of sound (e.g., Lange, 2005)
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 4
5. Domain-general or domain-
specific?
• Domain-general:
– E.g., Cowan’s Model of STM (1988, 1995)
• Domain-specific:
– Primacy model (e.g., the phonological loop; e.g., Page & Norris,
2003)
– Baddeley’s Model (e.g., Baddeley, 2015)
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 5
6. Multisensory perspective
• Predictive coding framework (e.g., Quak, 2015):
– Unexpected events -> mismatch –> model
update
– STM is amodal –> multisensory
• Representations held in memory regardless of domain
or modality
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 6
7. Spatial vs. Verbal STM & Distraction
• Deviants do not affect spatial (Lange, 2005)
– Only affected verbal STM
– Visual spatial deviants
• Rectangles changing color
• Distractors moving prior to target -> small effect
STM consists of at least spatial and verbal domains (Lange,
2005)
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 7
8. Spatial vs. Verbal STM
• Visual deviants less potent (Leiva et al.,
2015)
• Spatial change not examined
• Tactile deviants
• Multisensory
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 8
9. The current study
• Domain general;
– A spatial change (i.e., from one side of the body
to the other) would disrupt performance in both
spatial and verbal STM tasks
• Domain specific;
– A spatial change should disrupt performance in
the spatial task only
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 9
10. Method - sample
• N = 29
• Age = 18 – 42
• Reported:
– Normal hearing and seing
– No somatosensory deficits
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 10
11. Method – STM Tasks
• Serial-recall (i.e., recall the items in the order they were
presented)
– Verbal task
– Spatial task
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 11
12. Method – STM tasks
• 7 targets presented one at a time
- Spatial: dots on a 5x5 matrix
- Verbal: 7 digits from the set of 1-9
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 12
14. Method – distractors
• Standard trials:
– Sounds and vibrations on one side of the body
(e.g., right ear & right upper arm)
– 80 % of trials
• Deviant trials:
– Change from one side of the body to the other
(e.g., from right to left)
– 20 % of trials
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 14
15. Method – durations
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 15
16. Results
• Proportion of recalled items
• 2 (Task: Spatial, Verbal) x 2 (trial type:
Standard, Deviant) ANOVA for repeated
measures
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 16
19. Conclusions
• Predictive coding framework:
– MMN (Winkler, et al.m 2007)
– Violations of expectation
• Forced-choice paradigms (e.g., Parmentier et al.,, 2014)
• Short-term memory (e.g., Vachon et al., 2012)
– Model update -> lesser activation of the
memory traces? (cf., Cowan, 1995)
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 19
20. Next steps
• Spatial change did affect both tasks:
– We do not know:
• Auditory change?
• Tactile change?
• Both?
• Will another type of change affect both tasks?
June 16, 2016
IMRF2016 - Marsja - Spatial Change in
Multisensory Distractors 20
21. Thank you for your attention!
June 16, 2016
IMRF2016 - Marsja - Spatial Change
in Multisensory Distractors 21
Dr. Marsh Dr. Neely Dr. Hansson
Dr. K-Ljungberg
Hinweis der Redaktion
I am going to present the first study in an ongoing project.
Our findings suggest that there spatial and verbal short-term memory are not so distinctive at all.
When it comes to attention short-term memory is typically divided into two domains; the visuo-spatial and the verbal.
In the visuo-spatial information on spatial locations are held whereas in the verbal it is more to auditory but also visual. For instance, rehearsing visually presented digits would be verbal memory.
As with many of our cognitive abilities the short-term memory have been found to be prone to distractions. In one research paradigm the effect of irrelevant sounds have been found to affect encoding of memory.
Changing-sounds like the sounds of a changed to b back and forth has been found to be more distracting compared to a steady stream of just the sound of the letter A.
Of particular interest for the current study sudden and unexpected events, called deviants, have also been found to disrupt short-term memory performance;
Changing the isi between two sounds in the auditory sequence
Change in voice from male to female
Changing the sound from one to another
Have all been found disrupt short-term memory performance
Lets return to domains of stm;
It has been a debate whether the stm is domain-general or domain-specific
Domain-general stm would be that all types of information are held in same storage
Domain-specific is what I outlined earlier; there are at least 2 domains, in our case spatial and verbal
It can also be a matter of sensory modality
Spatial distractors have been far less researched when it comes to short-term memory but one study found that deviants do not affect spatial memory
Spatial changes defined as change in color of boxes flanking the items-to-be-recalled did not affect memory
If the distractor moved prior to target it had a small effect
Lange concluded that short-term memory is built on spatial and verbal domains
That leads us up to the current study;
If short-term memroy is domain-general a spatial change would disrupt performance in both spatial and verbal STM tasks
However,
If STM is domain-specific a spatial shange should disrupt performance in the spatial task only.
In both STM tasks we used 7 targets.
In the spatial the subjects were presented with dots presented on the screen in a 5by5 matrix
And in the verbal task 7 digits were presented
I will present a schematic overview of the task later
During encoding of the items-to-be-recalled subjects were exposed to a multisensory stream consisting of 10 distractors.
The distractors were presented on one side of the body in the standard trials
Whereas on the deviant trials the stimulation were changed to the other side of the body
Here is an overview of the tasks
The items to be recalled were resented for 400ms with 450ms intervalls
The distractor stream consisted of sounds and vibrations with a duration of 250 ms with an intervall of 350ms.
In botht tasks subjects had to click on the order of the items were presented
Analysis were carried out on the proportion of recalled items in each trial type and these proprtions were analysed with a 2 by 2 ANOVA for repeated measures
A spatial change in a multisensory stream affect both spatial and verbal STM;
STM may not consist of spatial and verbal domains
If we return to the predictive coding framework,
In other tasks settings using event-related potentials it has been suggested that the mismatch negativity that is elicited by deviant stimuli is due to prediction errors,
Behavioral data, seen in prolongation of response times to visual targets, have also been suggested to be due to that the deviant sound, for instance, is a violation of the cognitive systems prediction of upcoming events.
Perhaps that the model have to updated leads to lesser activation of the memory traces of the items to be remembered?
Although that spatial change did affect both tasks we don’t know whether this was due to the change in the sound sequence or the tactile sequence. It could be both that adds it.
Next step would therefore be to manipulate one modality at a time.
Another question is whether another change, maybe in the ISI between distractors, would affect both tasks also?