Talk at the Conference of the Society for Cognitive Studies of the Moving Image, Berlin 2013

1. TRUMPETS AND ORANGE-COLOURED DAYS
Inferences, associations and synaesthesia in filmic text
Janina Wildfeuer
Faculty of Linguistics and Literary Sciences
Bremen University, Germany

2. 2
Starting point
• how is meaning constructed crossmodally?
• how do we interpret/infer the intersemiotic interplay of
the modalities?

3. 3
Starting point
„Cinema can give us much more than Rimbaudian
correspondences […], it can create a veritable intersensory
reciprocity.“
Chion 1994: 134
„In der Bewältigung der polyphonie informationelle – wie
Barthes dieses Zusammenspiel sehr anschaulich genannt hat –
bringt der Zuschauer sich selbst ins Spiel, sein Gefühl
dafür, wie die verschiedenartigen Reize
zusammenhängen, die über Auge und Ohr in ihn einziehen.“
Flückiger 2001: 137

4. 4
Intersensory perception – synchretism – synaesthesia
„The perception of films, like natural perception, is both
synesthetic and synoptic […].“
Plantinga 2009: 141
„Alternatively, one might argue that all cinema is
synaesthetic, because in lived experience the senses have a
way of skirting the hierarchies and divisions we use to
define and explain them.“
Barker 2008: 236f.

5. 5
Synaesthesia and audiovisual perception
experiencing sensory
data
=synaesthesia
sensory interpretation
attributing semantic
content
cognitive interpretation
Fahlenbrach 2003/2010semantic networking?

6. 6
Analysing Filmic Textuality:
Segmented Film Discourse Representation Structures
THEORIE DES FILMVERSTEHENSLOGIC OF FILM DISCOURSE INTERPRETATION
1. level of identification and arrangement of the
meaning-making entities
2. level of coherence and structure
1. logic
2. logic
constructing logical forms of discourse
relating the logical forms via discourse relations
Wildfeuer 2013

7. 7
Amélie (Le fabuleux destin d‘Amélie Poulain)
(Jeunet, 2001)

8. 8
Constructing logical forms of discourse

9. 9
"Dear Mrs. Wells, we recently recovered a mailbag that was lost in a plane crash on October
12th 1969 on Mont Blanc. We are forwarding the enclosed letter to your address. Please
accept our apologies for the uncustomary delay. Jacques Grosjean, Customer Services.”
Constructing logical forms of discourse

10. 10
2. logic: film discourse relations

11. 11
Relating the logical forms via discourse
relations

12. 12
Relating the logical forms via discourse relations

13. 13
Relating the logical forms via discourse relations
"Darling Mado, I'm in exile. I can't sleep, can't eat. I think of you endlessly. I know I've made
the biggest mistake of my life. I turned down that woman's money. If all goes well, I'll soon
be able to afford a house. I dream of better times ahead when you'll forgive me and join me
here one orange-coloured day. Your ever-loving Adrien.

14. 14
Relating the logical forms via discourse relations

15. 15
Relating the logical
forms via discourse
relations

16. 16
Synaesthesia and audiovisual perception
experiencing sensory
data
=synaesthesia
sensory interpretation
attributing semantic
content
cognitive interpretation
semantic networking!

17. 17
Synaesthesia and audiovisual perception
„Object recognition and identification are goal directed
tasks in which perceptual systems parse sensory inputs into
salient cues, group these cues together into perceptual
objects, and assign them to categories.“
Seeley 2012: 501f.
„The results of this study suggests that sound affects
processing at the level of the „visual“ cortex.“
Shams et al. 2004: 30

18. 18
Where to go from here…?
constructing logical forms of discourse
relating the logical forms via discourse relations
neuroscientific research ?
filmic (sensory)
data
narrative patterns
symbolic
meanings, etc.

19. THANK YOU!
Janina Wildfeuer
http://www.fb10.uni-bremen.de/bitt
wildfeuer@uni-bremen.de