This document discusses how video games use multimodal metaphors in game character design to integrate sensory experiences with meaning. It presents conceptual metaphor theory and argues that games employ structural and orientational metaphors through audiovisual design. Examples are given of how games metaphorically represent concepts like power, virtue and control through a character's motion, abilities and interactions within the game world.

1. M U LT I M O D A L M E TA P H O R S
A N D G A M E C H A R A C T E R D E S I G N
F E L I X S C H R Ö T E R , U N I V E R S I T Y O F H A M B U R G
MEDIALITY AND MULTIMODALITY ACROSS MEDIA
WINTER SCHOOL AT THE GRADUATE ACADEMY OF THE UNIVERSITY OF TUEBINGEN, GERMANY
JANUARY 29, 2014
felixschroeter@uni-hamburg.de @felixjs www.felixschroeter.de

2. multimodality
linguistics
film studies
gamestudies
O’HALLORAN 2004
narratology
KRESS/VAN LEUWEN 2001
semiotics
PAGE 2010
BATEMAN/SCHMIDT 2010
DENA 2010

3. ‣ address players through
multiple sensory channels,
‣ rely on players’ input through
different multimodal control
schemes (Oviatt 2002),
‣ combine ‘narrative’ and
‘game’ modes (Dena 2010).
V I D E O G A M E S …

4. – C H A R L E S F O R C E V I L L E ( 2 0 0 9 : 2 4 )
„[M]ultimodal metaphors are metaphors whose
target and source are each represented exclusively
or predominantly in different modes.“

5. C L A I M
Audiovisual metaphors are used in game character
design to integrate multi-sensory experiences with
higher cognitive meaning and ‘merge’ ludic and
narrative dimensions of characters.

6. M U LT I M O D A L I T Y A N D
C O N C E P T U A L M E TA P H O R T H E O RY
Conceptual Metaphor Theory (CMT)
• Human beings make sense of (abstract) phenomena by
understanding them metaphorically in terms of
physical, concrete phenomena (Lakoff/Johnson 1980).
• In conceptual metaphors (e.g. LIFE IS A JOURNEY)
elements of a source domain (JOURNEY) are mapped
onto elements of a target domain (LIFE).

7. M U LT I M O D A L I T Y A N D
C O N C E P T U A L M E TA P H O R T H E O RY
• pictorial / visual metaphors (e.g. Kennedy 1982)
• audiovisual metaphors (e.g. Forceville 2006,
Fahlenbrach 2010)
• multimodal metaphors (e.g. Forceville/Urios-Aparisi
2009)

8. M U LT I M O D A L I T Y A N D
C O N C E P T U A L M E TA P H O R T H E O RY
LIFE IS A JOURNEY
The Straight Story (USA 1999)

9. – s e e K AT H R I N FA H L E N B R A C H ( 2 0 1 0 : 9 3 – 9 4 )
audiovisual metaphors = intentionally designed
symbolical forms that refer to conceptual metaphors in
their specific use of pictures, sounds, and movement

10. M U LT I M O D A L I T Y A N D
C O N C E P T U A L M E TA P H O R T H E O RY
ANGER IS LOSS OF BALANCE
Crank (USA 2006)

11. M U LT I M O D A L I T Y A N D
C O N C E P T U A L M E TA P H O R T H E O RY
MADNESS IS LOSS OF BALANCE
Batman: Arkham Asylum (Rocksteady/Eidos Interactive 2009)

12. V I D E O G A M E S A N D M E TA P H O R S
• interface as metaphor for the game as an information
system (Järvinen 2007, Schrape 2012)
• simulation as metaphor for real-world processes
(Aarseth 2001, Juul 2005, or Bogost 2011)
• conceptual metaphors in game rhetoric (Bogost 2005),
aesthetics (Rusch 2009, Begy 2010), and story structure
(Kromhout 2010, Kromhout/Forceville 2013)

13. M U LT I M O D A L M E TA P H O R S
A N D G A M E C H A R A C T E R D E S I G N
Possible target domains: POWER, HEROISM, VIRTUE

14. M U LT I M O D A L M E TA P H O R S
A N D G A M E C H A R A C T E R D E S I G N
• structural metaphor: source domain provides
rich knowledge structure for the target concept
(Kövecses 2010)
‣ LIFE IS A JOURNEY
‣ TIME IS MOTION

15. M U LT I M O D A L M E TA P H O R S
A N D G A M E C H A R A C T E R D E S I G N
PURPOSIVE ACTIVITY IS MOVING FORWARD TOWARD A DESTINATION
Batman: Arkham City (Rocksteady/WB Interactive 2011)

16. M U LT I M O D A L M E TA P H O R S
A N D G A M E C H A R A C T E R D E S I G N
POWER IS SELF-PROPELLED MOTION
InFamous (Sucker Punch/Sony 2009)

17. M U LT I M O D A L M E TA P H O R S
A N D G A M E C H A R A C T E R D E S I G N
• orientational metaphor: maps basic mental schemas
of spatial orientation on (typically abstract) target
concepts (Kövecses 2010)
‣ MORE IS UP | LESS IS DOWN
‣ VIRTUE IS UP | LACK OF VIRTUE IS DOWN
‣ CONTROL IS UP | LACK OF CONTROL IS DOWN

18. M U LT I M O D A L M E TA P H O R S
A N D G A M E C H A R A C T E R D E S I G N
VIRTUE IS UP | CONTROL IS UP
InFamousBatman: Arkham City

19. M U LT I M O D A L M E TA P H O R S
A N D G A M E C H A R A C T E R D E S I G N
• ontological metaphor: gives (mostly abstract) target
concepts a new ontological status as objects, substances,
or containers (Kövecses 2010)
• allows for further structuring through structural metaphors
‣ STATE AS CONTAINER („being in love“)
‣ ACTION AS OBJECT („giving someone a call“)

20. M U LT I M O D A L M E TA P H O R S
A N D G A M E C H A R A C T E R D E S I G N
POWER IS A SUBSTANCE | HUMAN IS A CONTAINER
InFamous

21. M U LT I M O D A L M E TA P H O R S
A N D G A M E C H A R A C T E R D E S I G N
POWER IS A SUBSTANCE | HUMAN IS A CONTAINER
The Elder Scrolls V: Skyrim (Bethesda 2011)

22. C O N C L U S I O N
Audiovisual metaphors are used in game character
design to integrate multi-sensory experiences with
higher cognitive meaning and ‘merge’ ludic and
narrative dimensions of characters.

23. T H A N K Y O U !
felixschroeter@uni-hamburg.de
@felixjs
www.felixschroeter.de

24. • Aarseth, Espen. 2001. “Allegories of
Space. The Question of Spatiality in
Computer Games.” In Cybertext Yearbook
2000, edited by Markku Eskelinen and
Raine Koskimaa, pp. 152–171. Jyväskylä:
University of Jyväskylä.
• Bateman, John A., and Karl-Heinrich
Schmidt. 2012. Multimodal Film Analysis.
How Film Mean. New York/London:
Routledge.
• Begy, Jason. 2010. “Interpreting Abstract
Games: The Metaphorical Potential of
Formal Game Elements.“ MA thesis,
Cambridge, MA: MIT. Accessed December
12, 2014.
• Bogost, Ian. 2005. “Frame and Metaphor
in Political Games.” In Proceedings of the
DiGRA 2005 Conference. Vancouver:
DiGRA. Accessed December 12, 2014.
• Bogost, Ian. 2011. How to Do Things with
Videogames. Minneapolis: University of
Minnesota Press.
• Dena, Christy. 2010. “Beyond Multimedia,
Narrative, and Game: The Contributions of
Multimodality and Polymorphic Fictions.”
In New Perspectives on Narrative and
Multimodality, edited by Ruth Page, pp.
183–201. New York/London: Routledge.
• Fahlenbrach, Kathrin. 2010. Audiovisuelle
Metaphern. Zur Körper- und Affektästhetik
in Film und Fernsehen. Marburg: Schüren.
• Fahlenbrach, Kathrin. In prep. “Affective
Spaces and Metaphors in Video Games.”
In Games, Cognition, and Emotion. Essays
in Cognitive Video Game Studies, edited
by Bernard Perron and Felix Schröter.
Jefferson, NC: McFarland.
• Forceville, Charles. 2006. “The Source-Path-
Goal Schema in the Autobiographical Journey
Documentary: McElwee, Van der Keuken,
Cole.” In New Review of Film and Television
Studies 4: 241–261.
• Forceville, Charles. 2009. “Non-Verbal and
Multimodal Metaphor in a Cognitivist
Framework: Agendas for Research.” In
Multimodal Metaphor, edited by Charles
Forceville and Eduardo Urios-Aparisi, pp. 19–
42. Berlin/New York: de Gruyter.
• Forceville, Charles, and Eduardo Urios-
Aparisi. 2009. “Introduction.” In Multimodal
Metaphor, edited by Charles Forceville and
Eduardo Urios-Aparisi, pp. 3–17. Berlin/New
York: de Gruyter.
• Järvinen, Aki. 2008. Games Without Frontiers.
Theories and Methods for Game Studies and
Design. PhD thesis, University of Tampere.
• Juul, Jesper. 2005. Half-Real. Video Games
between Real Rules and Fictional Worlds.
Cambridge, MA: MIT Press.
• Kennedy, John M. 1982. “Metaphor in
Pictures”. In Perception 11: 589–605.
• Kövecses, Zoltán. 2010. Metaphor. A Practical
Introduction. Second Edition. Oxford/New
York: Oxford University Press.
• Kress, Gunter, and Theo van Leeuwen. 2001.
Multimodal Discourse: The Modes and Media
of Contemporary Communication. London:
Arnold.
• Kromhout, Roelf. 2010. Source-Path-Goal
Structure in Multimodal and Interactive
Videogames: Half-Life 2, Grim Fandango and
Heavy Rain. MA thesis, University of
Amsterdam.
• Kromhout, Roelf, and Charles
Forceville. 2013. “LIFE IS A
JOURNEY. The Source-Path-Goal
Schema in the Videogames Half-
Life, Heavy Rain, and Grim
Fandango.” Metaphor and the
Social World 3(1): 100–116.
• Lakoff, George, and Mark Johnson.
1980. Metaphors We Live By.
Chicago: Chicago University Press.
• O’Halloran, Kay L. (ed.). 2004.
Multimodal Discourse Analysis.
Systemic-Functional Perspectives.
London/New York: continuum.
• Oviatt, Sharon. 2002. “Multimodal
interfaces.” In The Human-
Computer Interaction Handbook:
Fundamentals, Evolving
Technologies and Emerging
Applications, edited by Julie A.
Jacko and Andrew Sears, pp. 286–
304. Hilssdale, NJ: Erlbaum.
• Page, Ruth (ed.). 2010. New
Perspectives on Narrative and
Multimodality. New York/London:
Routledge.
• Rusch, Doris. 2009. “Mechanisms of
the Soul. Tackling the Human
Condition in Videogames.” In
Breaking New Ground: Innovation
in Games, Play, Practice, and
Theory. Proceedings of the DiGRA
2009 Conference. Brunel: DiGRA.
• Schrape, Niklas. 2012. Die Rhetorik
von Computerspielen. Wie
politische Spiele überzeugen.
Frankfurt/New York, NY: Campus
Verlag.