1.
The Role Of Visual Grammar and
Player Perception In An Online Game
Banphot Nobaew
Faculty of Humanities
E-learning Lab - Center for User-driven Innovation,
Learning and Design
Aalborg University, 2015
1

2.
Agenda
• What is World of Warcraft?
• Why did I choose this topic?
• Research Objectives and Research Questions
• Theoretical Inspiration
– Defining ‘Grammar’ and ‘Visual Grammar’
– Theoretical Foci
• Developing a Visual Grammar
– Defining Units of Analysis
– Collecting Research Data
– Analysis
• Site of the Game
• Site of the Player
• Breaking the Rules
• Conclusion
2

3.
What is World of Warcraft?
3

4.
What is World of Warcraft?
• What is World of Warcraft?
– It is a ‘massively multiplayer online role-playing game’, or MMORPG
which was introduced to the public in November 2004.
– In 2010, the Annual Report of Blizzard Entertainment reported that there
were twelve million subscribers.
– The stories and visual design draw on mythology, ancient history,
science fiction, eastern cultures and literary works such as Lord of the
Rings.
• What happen in World of Warcraft?
– Some couples got married because of the game.
– Many schools used this game to teach students about the literacy.
– Many people get addicted to this game.
– People sell and buy the game objects aim to complete the quests.
– People spent a lot of money and times for this game.
4

5.
Why did I choose this topic?
• Positive feedback from the digital documentary film project which was
tested on school students. The film was made by integrating real moving
images and computer graphics.
• My previous research in rural northern Thailand investigated learning and
communication processes; it revealed a significant correlation between
narrative structures and multimedia structures.
• Meanings are conveyed most effectively through well-designed integration
of storyline and media.
• Games have been adapted for education, medicine, training, creative arts,
design, business, etc. These games are a new digital medium; games have
been adapted in different ways, for example: for learning and online
socializing.
• Games adopt styles, techniques and devices from other media; they also
borrow visual grammar from all other visual arts. All visual forms and
functions have been imitated in games. 5

6.
Research Objectives and
Research Questions
6

7.
Research Objectives
The aim of my research is to provide designers of digital games
and digital educational materials with a heuristic approach to
visual grammar for the creation of a visual language.
Research Outcome
The end result of this study will be to propose a fundamental
visual grammar for analyzing three-dimensional online games.
The visual grammar can be adapted to investigate all online
games and other interactive media and, additionally, be tool for
game designers and in educational applications.
7

8.
Research Questions
In this study, I focus on visual perceptions and visual
experiences of players rather than on benefits that they
may derive. We need to understand how games operate.
• How does the player make sense of visual objects and
gameplay?
• How are the meanings of visual objects produced and
consumed?
8

9.
A Further Research Question
• What is the grammar of a three-dimensional online
multiplayer role-play game and how does the game
create players’ visual perceptions? This question leads
to three sub-questions:
– How do the visual elements of game design express meaning
and how are meanings conveyed to players?
– How do players perceive and conceive meanings in elements of
visual design and interpret them during gameplay?
– What is the visual grammar of game?
9

10.
Theoretical Foundations
10

11.
Defining ‘Grammar’
• ‘Visual’- originally referred to a beam imagined to proceed from the eye
and make vision possible (Oxford Dictionary).
• ‘Grammar’ has several meanings:
– The whole system and structure of a language or of languages in
general, usually taken as consisting of syntax and morphology
(including inflexions) and sometimes also phonology and
semantics.
– A particular analysis of the system and structure of language or of
a specific language.
– A set of actual or presumed prescriptive notions about correct use
of a language.
– The basic elements of an area of knowledge or skill.
– Computing a set of rules governing what strings are valid or
allowable in a language or text. (oxforddictionaries.com)
11

12.
Reading Images: the Grammar of Visual Design by Kress and Leeuwen (2006) provides a
framework for analysis of social interaction between audience, artwork and advertising; this
is adapted here to study and record interaction between player and online game.
David Buckingham (2010) mentions visual language of games and establishes a framework
for game literacy with four components:
• Representation is realism through graphics, sounds and verbal language; it incorporates
characters, virtual identities of social groups and game genre.
• Production comprises auteur, technology and software, game designers and
production team and the relationship of games with other media.
• Audience the experience and pleasure of playing as derived from rules and
structures.
• Language includes:
 The functions of verbal text (audio and written), and non-verbal text
such as still and moving images, sounds and music.
 The distinctive properties of game genres.
 The characteristics and organization of space and time in different
game genres, and the role of player in the game landscape.
 Game properties such as rules, rewards, objectives, obstacles and
other (Buckingham 2010:66-67).
Defining ‘Visual Grammar’
12

13.
‘Visual Grammar’ in this study
‘Visual language’ has been defined by various scholars; definitions
relevant to this research are:
A ‘system where basic elements are interrelated according to certain
laws or regularities’ (Saint-Martin 1990)
‘It is based on the tight integration of words and visual elements’
(Horn 1998)
My definition is ‘the principles and rules of a visual game
landscape (the game territory and environment) for composing the
forms of visual elements and structuring the meaning of
perceptual experience. It creates a system of visual
communication in a particular context’.
13

14.
Theoretical Foci
The distinctive sites and theoretical practices applied in this research are:
(a)Site of the Game – the analytical framework applied in this research
was a combination of the work of these scholars: Saint Martin’s Visual
Semiotics (1990); Arnheim’s Theory of Visual Perception (1974); and
gestalt theory from many writers.
(b)Site of Player – the analytical framework drawn on Halliday’s Social
Semiotics (1985); Multimodal Discourse Analysis of Norris & Jones
(2005), Kress & van Leeuwen (2001) and Rose (2001); van Leeuwen’s
modes of analysis of speech, music and sound (1999); Martinec’s
gestures (2000); and Norris’s interactions (2004).
(c)Site of production – not of primary concern in this research except
insofar as the sites of game and player interact with it.
(d)Interaction between the sites of game and player.
The two main fields of study share a common origin: Hjelmslev’s (1943) ‘Plane
of Language’ has two components: Content Plane and Expression Plane. The
content and expression sides of language in general are referred to as planes
(ibid. 59). 14

15.
Developing a Visual Grammar
15

16.
Defining Units of Analysis
Rose (2001), and Gillian and Rose (2007) propose site modalities and
methodology for interpretation of visual material; the site modalities are the site
of production; the site of image; and site of audiences.
Rose’s ideas can be adapted for this study to define sites and modalities in
World of Warcraft.
(i) The site of the game - interpretation of visual signs and their composition
for meanings
(ii) The site of the player - analysis of player reactions: gestures and verbal
reactions whilst playing and interviews after playing. The way that a player
learns from, interacts and participates within the gameplay
(iii) Interactions – interaction between players and player and the game.
16

17.
Sites of Modalities and Methods of Game Interpretation
SiteoftheGame
17

18.
Sites of Modalities and Methods of Game Interpretation
SiteoftheGame
18

19.
Collecting Research Data
1. Game Design and Production: Background Information
Data type: a) Game development process, b) Game concept and story design,
c) Core design and game play, d) Conceptual design and detailed design.
e) Other
Research tools:
- Interview
- Sound recording
- Photo shooting
-Video ethnography
2. Online research: Site of the Game
- joining the online game: online ethnography
- Capture Game data
3. Gameplayers: Site of the Player and Interaction
Data type: visual language data – collecting data from gameplayers: ‘how are they reading
the game elements?’
Research tools:
- Interview
- Gameplay captured through screen and video capturing.
- Sound recording
-Video ethnography
19

20.
Research Process to develop the Visual Grammar
Framing the theoretical and analytical tools
Game online research The first data observation and
interviewing with game designers
and gameplayers
1st Data analyzing
Reframing the theoretical and
analytical tools
Developing the model of a visual
grammar
The second data observation and
interviewing with game
designers and gameplayers
2nd Data analyzing
Testing the model of the visual grammar
Developing the visual
grammar
Framework of Visual
Grammar
20

21.
Two sites of analysis
Site of the Game Site of the Player
21

22.
Site of the Game
22

23.
Theoretical Frameworks for Site of the Game
The Theoretical Foundations of Visual Grammar
Saussure
Signs Sign Distinctions
Signifier
Form
Substance
Signified
Form
Substance
Hjemslev
Planes – equivalent to Saussure’s signs
Four strata – equivalent to Saussure’s
distinctions
Expression plane - signifier
Expression-form
Expression-substance
Content plane - signified
Content-form
Content-substance
Eco
Sign – equivalent to Hjemslev’s plane Planes – equivalent to Hjemslev’s srata
Expression-form
Expression-plane
Content-plane
Content-form
Expression-plane
Content-plane
Arnheim
Principles of Visual Peception
Balance
Shape
Form
Growth
Space
Light
Colour
Movement
Dynamics
Expression
23

24.
Theoretical Frameworks for Site of the Game
Saint-Martin
Concepts Levels of variables
Variables
groups
Distinct categories
Basic plane – two
dimensional
The primary level of variables –
coloremes
Plastic variables
Texture
colour
Conceptual variables
Dimension
Boundaries
Vectoriality
Implantation or position
The second level of variables – syntactics
Virtual cube – constructed
from basic plane – three
dimensional
The primary level of variables –
coloremes
Plastic variables
Texture
colour
Conceptual variables
Dimension
Boundaries
Vectoriality
Implantation or position
The second level of variables – syntactics
Environment – constructed
from basic plane and virtual
cube – multi dimensional
The primary level of variables –
coloremes
Plastic variables
Texture
colour
Conceptual variables
Dimension
Boundaries
Vectoriality
Implantation or position
The second level of variables – syntactics 24

25.
Screen Analysis
25

26.
Developing Visual Grammar
Site of Game
Game-content Plane
(basic plane, virtual cube, and
environment plane)
The primary level: Colorematic
Phonological: Visual Variables
on the Basic plane
Game-expression Plane
- Visual Rules
- Visual Operators
The second level: Syntactic
By Syntax Operators of visual
language
Visual Perception Approach
and
Gestalt Theory 26

27.
Game-content Plane
Concept of ‘Basic Plane’
• Primary Visual Elements
• Secondary Visual Elements
27

28.
Primary Visual Elements
Plane Visual Elements
Content Plane
Planes Properties Attributes Definitions of Attributes
Primary
point
shape
scale
colour
line
thick
thin
colour
hue purity
shade darkness (by mixing black)
tint lightness (by mixing white)
value brightness
28

29.
Secondary Visual Elements
Plane Visual Elements
Content Plane
Planes Properties Attributes Definitions of Attributes
Secondary
shape
type
size
space
leveling and sharpening
similarities and differences
form
size
scale
colour
dimension
shape
space
spacing
third dimension
relative size and position
perspective
texture
size
scale
value
light
range
colour
direction
intensity
placement
quality and quantity
motivation
purpose
sound
volume or dynamic
rhythm and metre
tempo level or speed
form random sequence of music
pitch transposition
Shape and Form
29

30.
Game-expression Plane
Visual Principles (Art Theory)
 Balance
 Emphasis
 Movement or Rhythm
 Dynamics
 Perspective
Plane Visual eEements
Content
Plane
Planes Properties Attributes
Definitions of
Attributes
Expression
Plane
Rules
Visual
Principles
balance
emphasis
movement or rhythm
dynamics
perspectives
Syntactic
Rules
basic laws of visual perception
laws of grouping
principles of composition
principles of visual interaction
interactivity and game mechanics
Rule 1: The Basic Laws of Visual Perception
– fundamental rules of visual design
Rule 2: The Laws of Visual Grouping
– objects are visually systematized into groups. Groups are formed from objects which share visual elements and
properties.
Rule 3: Composition Principles
– a guide to integration of all visual elements and operators and other components of the game including storyline
and gameplay
Rule 4: The Principle of Visual Interactions
– interactions between player and the game.
Rule 5: Interactivity and Game Mechanics
– Interactivity encourages players to continue playing; gameplay, however, is systemized and controlled by the
‘game mechanics.’
Syntactic Rules - the Rules of Visual Syntax (Gestalt theory and Visual Perception)
30

31.
The diagram of visual grammar and game interaction
Visual Grammar of the WoW game
Game-content Plane
(basic plane, virtual cube,
and environmental plane)
Visual Elements:
- The primary visual elements
- The secondary visual elements
Game-expression Plane
Visual Operators (Visual
Syntactic)
- Five Operators
Visual Rules (The Rules of Visual Syntax)
Visual Composition
Visual landscape
31

32.
The Site of Player
and
Interaction
32

33.
Player Perspectives In Three-dimensional Online Games
(I) Social Semiotics (Halliday‘s concept): the three components of Systematic Function
Theory were adapted as follows for this study to define modes of the gameplay and
player interaction.
• Field of Game Discourse – applied to gameplay activities in both real and virtual
worlds.
• Tenor of Game Discourse – applied to profiles of the players such as gender, skill
levels and so on, and to relationships between players and their roles.
• Mode of Game Discourse – applied to interaction between player and the game and
between players including dialogue, gestures, movement; and symbolism such as
visual elements.
These are used to define the modes of gameplay and player
interaction.
33

34.
(II) Multimodal interaction (Norris’s concept) of two components is
integrated into the analysis of interaction to investigate a cohesive
chain of action between modes.
• Lower-Level Action – the smallest interactional unit of meaning
• Higher-Level Action – a multiplicity of chains of lower-level actions
bracketed by opening and closing
This is used to define the cohesive chains between modes.
34

35.
Gameplay Workshop
35

36.
Interviews with Players after Gameplay
Open-question interviews with gameplayers whilst watching and analyzing their own
gameplay 36

37.
Mapping the site of player and the site of game
37

38.
Analytical Methodology for
Site of the Player
38

39.
The analytical methodology was set up in five stages.
These stages are constructed from WoW literature online
and from several other sources such as official game
manuals.
Stage 1: Conceptualisation – the nature and characteristics of the
game
(to be adapted Halliday framework)
Stage 2: Categorisation and Quantification
(to be adapted Halliday framework)
Stage 3: Analysis of Gameplay to Define Interaction
(to be adapted Norris’s concept )
Stage 4: Game Literacy
Stage 5: Interconnection Between Modes (Norris’s concept)
39

40.
Conceptualise
• Game concept
• Features
• Gameplay
Categorize and
Quantify Modes
-modes of game
components and
elements
Investigate Game
Mechanics from
Gameplay
Investigate Game
Activities and Symbolism
(Visual Design)
Investigate Player
Interaction and
Experience
Analyse Game
Literacies
• Learning
• Skills/Training
• Perspectives
Examine
Interconnections
between Modalities
of Game
Components
-Facial expressions
- Head Movement
- Gaze
- Body Movement
-Proxemics
-Self-speaking
-Yell
-Laugh
Physical Space Virtual Space (Gamescape)
Player
Players
Storyline
Quest stories
Sounds
Visual Game Objects
-Props
- Sets
- Interface, Navigators
- Game Objects
-Avatar, NPCs
Text
Game mechanics
Interactions
Analytical Methodology for Analyse Player Experience in Online
Games
Halliday theory
Norris concept
40

41.
Interaction between
the Site of the Game
and
the Site of the Player
41

42.
Integrating Player Perceptions Into Visual Grammar Of
Online Games
Visual Components
Sites & Interaction Research Approaches
Game Interaction Player
Multimodal
Analysis
Visual
Semiotics
and
Art Theory
Visual Elements  × × × 
Visual Principles  × × × 
Visual
Operators
1.Basic Laws of Visual Perception 
   
2.Laws of Visual Grouping  × × × 
3.Composition Principles  × ×
× 
4.Principle of Visual Interaction 
 
 
5.Interactivity and Game Mechanics 
 
 
42

43.
Diagram of Visual Grammar (site of the game) and
Game Interaction (site of the player)
The visual Elements:
- The primary visual elements
- The secondary visual elements
The visual Operators:
Balance, Emphasis, Movement or
Rhythm, Dynamics, and Perspective
The visual Rules: Rules of Visual Syntax
- The Basic Laws of Visual Perception
- The Laws of Grouping
- Composition Principle
- Visual Interactions Principle
- Interactivity and Game mechanics
The Game World
The Macro Visual DesignThe Micro Visual Design
Gameplay or Interactions
43

44.
Breaking the Rules
44

45.
Breaking the Rules:
During play, players have to interpret multiple texts, written, visual and sound
through interaction. Visual interpretation can be erroneous, caused by the
visual grammar itself, player experience or devices. Misinterpretation can arise
for reasons such as:
• unbalanced visual design with, for example, vivid colours or oversized
objects.
45

46.
Breaking the Rules: (continue)
• visual clues which do not provide enough detail or information: there
is no prominent clue; the player cannot see the clue properly.
• objects are arranged too densely and superimposed, causing
problems of selection and slowing the progress of the game.
46

47.
Conclusion
47

48.
Interconnections between Modalities in the Virtual
World and the Real World
Visual Objects
Players Gameplay
Experience
48

49.
In summary
• My visual grammar can inform the design of game.
• My visual grammar can inform game studies.
• My visual grammar can be used to interpret the online
game and other digital media.
49

50.
Thank you
50

51.
References
Arnheim, R., 1974. Art and Visual Perception: A Psychology of the Creative Eye-the New Version-Expanded and Rev. Ed, University of California Press.
B. Hardt, M., 2010. Michael B. Hardt : Aesthetics, semiotics and visual communication. Available at: http://www.michael-hardt.com/Professor/aesthetics.htm
[Accessed March 15, 2013].
Baldry, A.P., 2004. Phase and transition, type and instance: patterns in media texts as seen through a multimodal concordancer. In K. L. O’Halloran, ed. Multimodal
discourse analysis: Systemic-functional perspectives. Continuum Intl Pub Group, pp. 83–108.
Buckingham, D., 2006. Defining digital literacy: What do young people need to know about digital literacy. Nordic Journal of Digital Literacy, 4(1), pp.263–276.
Buckingham, David, 2010. Einleitung: Medien und Bildung im dramatischen kulturellen Wandel. In B. Bachmair, ed. Medienbildung in neuen Kulturräumen. pp. 59–
71.
Calabrese, O., 2011. The bridge: suggestions about the meaning of a pictorial motif. Journal of Art Historiography, 5.
Eco, U., 1979. A theory of semiotics, Indiana University Press.
Eco, U., 1980. Function and sign: The semiotics of architecture. Signs, symbols and architecture, pp.11–69.
Goodman, N., 1976. Languages of art: An approach to a theory of symbols, Hackett Publishing Company.
Halliday, M.A.K. & Ruqaiya Hasan, 1985. K and Hasan, R.(1985). Language Context and Text: Aspects of Language in a Social Semiotic Perspective. Victoria:
Deakin university press.
Kenyon, S.F.G., 1909. Aristotle on the Athenian Constitution, G. Bell and sons.
Kress, G.R. & Van Leeuwen, T., 2006. Reading images: The grammar of visual design, Routledge.
Van Leeuwen, T., 1999. Speech, music, sound.
Van Leeuwen, T. & Jewitt, C., 2010. Handbook of visual analysis, Sage Publications Ltd.
Mann, D., 2010. Gestalt Therapy 100 Key Points & Techniques., Hoboken: Taylor & Francis. Available at:
http://public.eblib.com/EBLPublic/PublicView.do?ptiID=574588 [Accessed March 13, 2013].
Martinec, R., 1998. Cohesion in action. Semiotica, 120(1-2), pp.161–180.
Martinec, R., 2000. Types of process in action. SEMIOTICA-LA HAYE THEN BERLIN-, 130(3/4), pp.243–268.
Nöth, W., 1990. Handbook of semiotics, Bloomington: Indiana University Press.
O’Toole, M., 1994. The language of displayed art, Leicester Univ Pr.
Rock, I. & Palmer, S., 1990. The legacy of Gestalt psychology. Scientific American, 263(6), pp.84–90.
Rose, G., 2001. Visual methodologies: An introduction to the interpretation of visual materials, Sage Publications Ltd.
Saint-Martin, F., 1990. Semiotics of visual language, Bloomington: Indiana University Press.
Saussure, F., 1974. Course in. General Linguistics, 191(6).
Saussure, F., 1916. Course in general linguistics, ed. by Charles Bally, Albert Sechehaye, and Albert Riedlinger, tr. by Wade Baskin, 1959. New York: Philosophical
Library.
Sebeok, T.A., 2001. Signs: An introduction to semiotics, University of Toronto Press.
Stenglin, M.K., 2004. Packaging curiosities: Towards a grammar of three-dimensional space. University of Sydney.
Theo Van Leeuwen, 2004. Ten Reasons Why Linguists Should Pay Attention to Visual Communication. In P. Levine & R. Scollon, eds. Discourse and technology:
Multimodal discourse analysis. Georgetown Univ Pr.
www.michael-hardt.com/MichaelHardt/conferences.htm
51