Uses of cross-media strategies for new ways of communicating

1. Storytelling 2.0
uses of cross-media strategies for new ways
of communicating
IPIN summer school
August 8th 2012
Kjetil Sandvik, associate professor, Media, Cognition and Communication,
University of Copenhagen

2. Agenda
• Focus on strategic storytelling, particularly in the
field of communicating culture/cultural heritage in
the light of digital, network-based and mobile
media and the increasing use of web 2.0/social
media-services.
• From a cross-media perspective we will focus on
the opportunities and challenges which these new
media technologies, platforms and services
represent to cultural organizations and institutions:
• Storytelling 2.0 and a perpetual beta way of
communicating focusing on dynamic and easy
changeable formats with a strong focus on user
participation, collaboration and co-creation).

3. MA in dramaturgy
PHD on computer-
games
Head of master pro-
gram in Cross-Media
Communication
Research: strategic
communication, new
media, storytelling etc.

4. This lecture
• Some brief words about storytelling and
cross-media communication – the general
idea
• Rich media experiences: from ’experi-
ence+’ to ’experience universes’ – case
studies: X factor and Harry Potter
• Storytelling 2.0 – case study: an
augmented reality game

5. Storytelling classic
• A chain of events in time and space
• Told by someone (a narrator) to somebody
else (a reader or spectator)
• through a specific media (novel, movie, TV
series…)
• And in a specific discourse (a genre
defining the structure of the plot/storyline)

6. Storytelling 2.0: participation
• The ingredients are the same, but the role
of the recipient has changed
• The story hands out the possibility for
interaction:
– to influence the course of events
– to gain control over one or more characters
– to play a part in the storyline
• Storytelling  Storydwelling

7. Labyrinthine story structure
Myst 2: Riven

8. Storytelling 2.0: co-creation
• Added the possibility for the participants to
be a part of creating the story,
– adding new parts to it,
– adding new characters,
– adding new narrative spaces and so on
• Storytelling  Storyprocessing

9. Storyspace
World of Warcraft

10. Sandbox
Concept
Environment
Design tools
Second Life

11. Cross-media communication
• Collaborative interplay between different
media
• Each media playing its specific role and
delivering its part of the overall story
• Putting to play the specific strengths of
each media (the media does what it does
best!)
• Cross-media storytelling: putting both
‘storytelling classic’ and the two modes of
storytelling 2.0 to effective use!

12. Cross-media communication
• It is about getting through to the user
• It is about giving the user a broader and
richer media experience
• It is about giving the user the possibility to
get engaged and to be involved in the
media experience on different levels and
to various degrees
• It is about giving the user the possibility for
participation and co-creation.

13. Cross-media communication
The art of having different (old and
new) media communicating together
• Each media has its special qualities
• Context: media evolution
– CMC challenges the role of the media types
• Context: participatory culture
– CMC challenges our models of
communication

14. Challenges of digital media
Participatory (social) media/web 2.0:
• radical possibilities for dialogic processes, for
collaboration and co-creation
• Communication as dynamic processes
• Fixed solutions  changeable, adaptive and
user-centered solutions
• Uses of web 2.0 apps mashups:
combinations of cheap, effective and
constantly updated and improved media
technology
• Storytelling 2.0: perpetual beta way of
communication

15. Context: participatory
culture and 2G experience
economy

16. Participatory culture
• “Patterns of media consumption have been
profoundly altered by a succession of new
media technologies which enable average
citizens to participate in the archiving,
annotation, appropriation, transformation,
and recirculation of media content. Partici-
patory culture refers to the new style of
consumerism that emerges in this environment.”
» Henry Jenkins

17. 2G experience economy:
participation  co-creation

18. Co-creation
• Boswijk et.al. focuses on the creative
dialogue between supplier and customer
instead of the supplier deciding what the
customer wants:
• It builds upon communication as sharing of
knowledge and the idea that value creation
no longer takes place within the company but
is created in the individual:
• “The development of meaningful-experience
concepts cannot take place without the direct
participation of the (potential) customer”.

19. Participation-based
communication
• We do not just want to be communication
to (classical mass-media communication
format: one-to-many).
• We need new communication models
which focuses on various forms of user
involvement and user experiences (one-
to-one and many-to-many communication)
– personalization: online-services which adapt
to the users’ actions

22. Participation based
communication
• We do not just want to be communication to
(classical mass-media communication format:
one-to-many).
• We need new communication models which
focuses on various forms of user
involvement and user experiences (one-to-
one and many-to-many communication)
– personalization: online-services which adapt to
the users’ actions
– enabling dialogue (e.g. blogs), user
participation (interactive elements creating
unique user experiences) and user co-creation
(possibility to create your own content).

23. LEGO Factory
• A co-creative story:
The user in centre of
the design process in
accordance with
LEGO’s corporate
values:
• Stimulating
creative play!

24. Users want to create their own toys

30. Co-creation: sharing and reworking design

32. Users want to design their own kitchens

33. Users want to tell their own stories

34. Users want to solve the crime mystery themselves

35. Users want to produce TV
themselves

36. Users want to write the news themselves
Citizen Journalism

37. Collective intelligence: crowdsourced, co-
creative creation of knowledge

38. ’Traditional’ media com-
munication (storytelling classic)
Control of flow
Producer Content User
Interpretation/use
Media
Inspired by Randy Haykin:
Multimedia demystified. A guide to
the world of multimedia from Apple Computer, 1994

39. Dialogic media communication
Performance
Control of flow
Producer Content User
Interpretation/use
Media
Feedback

40. Participatory media
communication
Production of content
Control of flow
Producer Content User
Interpretation/use
Produser Performance/Feedback
Media
Produsage
Reconfiguration (editing)

41. Producer
prodUser prodUser
Production of content
prodUser Content prodUser
Use of content
Media
Platform
prodUser prodUser
Co-creation based communication model

42. Modes of user engagement
• Communication as composition (the combination of
related media contents by established media (the
book, the movie, the game, the website) and/or the
combined use of various media and applications by
audiences (using a player to watch a TV program,
using a browser to monitor its website, and news
applications to get updates)).
• Communication as collaboration (e.g., participating in
debates relating to media content (chats, blogs,
forums))
• Communication as participation (e.g., influencing the
content of television, such as using SMS to vote for
one’s favorite in a talent show)
• Communication as co-creation (the independent
creation of media content, e.g. designing new features
on Facebook)

43. • A networked, participatory environment enables all
participants to be users as well as producers of
information and knowledge - frequently in a hybrid
role of produser where usage is necessarily also
productive.
• Produsers engage not in a traditional form of
content production, but are instead involved in
produsage - the collaborative and continuous
building and extending of existing content in pursuit
of further improvement.
Axel Bruns: Blogs, Wikipedia, Second Life, and Beyond:
From Production to Produsage, 2008

44. This lecture
• Some brief words about storytelling and
cross-media communication – the general
idea
• Rich media experiences: from
experience + to experience universes –
case studies: X factor and Harry Potter
• Storytelling 2.0 – case study: an
augmented reality game

45. The elements of the media
cirquit (John Fiske 1987)
• the primary text (the movie/tv-series)
• the secondary text (pr/marketing,
background material, bonus material:
surrounding the primary text)
• the tertiary text (the user’s own texts: are
produced on the background of the
primary and secondary text)
• Cross-media productions (and their
new media cirquits) changes this
hierarchy

46. New media cirquits
• Cross-media production:
• Connects primary, secondary and tertiary
texts into one common media text
• Embeds possibilities for participation
• Uses several communication matrixes:
• One-to-many (the TV show in itself)
• One-to-one (chats)
• Many-to-many (debate forums, quizzes, games…)
• One-to-one-as-group (communities on e.g. FB)
• Attempt to create a sense of belonging in
the user based on identification AND
interaction

47. Convergence culture
• This circulation of media content - across
different media systems, competing media
economies, and national borders - depends
heavily on consumer's active participation.
• Convergence should NOT be understood
primarily as a technological process bringing
together multiple media functions within the
same devices.
• Instead, convergence represents a cultural
shift as consumers are encouraged to seek
out new information and make connections
among dispersed media content.
» Henry Jenkins

48. Rich media experiences
• Experience through
• engagement and identification
• participation
• collaboration
• co-creation
• Two types of rich media experience
• Experience + (the augmentation of experience of
one specific media by implementing other media in
the communication-structure, e.g. a website to a
TV-show)
• Experience universe (interplay between different
media: e.g. book, movies, games)

49. Experience +
X factor

50. Engagement and identification
• The use of the TV media’s strenghts: ‘Storytelling classic’
to create emotional intensification
– A dramatic plot: the contest-format
– The use of classical dramatic agents (most prominent in the first two
seasons of the show): the good vs. the bad
– Use of personal and emotionally loaded stories
– Use of emotionally manipulative editing: production of ’magic
moments’ (the Poul Potts-trick!): close-ups, cross-editing, tears, tears
and more tears…
– Website: augmentary media with a surplus of background materials
about participants, their reactions to the judges and so on: extends the
possibility for engagement and interaction and introduces a possibility
for participation (guestbooks, chats, blogs…)
– Web 2.0: connecting and spreading the experience through the users’
own networks

51. There can only be one
winner…
51

52. Engagement and identification
• The use of the TV media’s strenghts: ‘Storytelling classic’
to create emotional intensification
– A dramatic plot: the contest-format
– The use of classical dramatic agents (most prominent in the first
two seasons of the show): the good vs. the bad
– Use of personal and emotionally loaded stories
– Use of emotionally manipulative editing: production of ’magic
moments’ (the Poul Potts-trick!): close-ups, cross-editing, tears, tears
and more tears…
– Website: augmentary media with a surplus of background materials
about participants, their reactions to the judges and so on: extends the
possibility for engagement and interaction and introduces a possibility
for participation (guestbooks, chats, blogs…)
– Web 2.0: connecting and spreading the experience through the users’
own networks

53. The good… …the bad…
…and the crybaby

54. 54

55. Engagement and identification
• The use of the TV media’s strenghts: ‘Storytelling classic’
to create emotional intensification
– A dramatic plot: the contest-format
– The use of classical dramatic agents (most prominent in the first two
seasons of the show): the good vs. the bad
– Use of personal and emotionally loaded stories
– Use of emotionally manipulative editing: production of ’magic
moments’ (the Poul Potts-trick!): close-ups, cross-editing, tears, tears
and more tears…
– Website: augmentary media with a surplus of background materials
about participants, their reactions to the judges and so on: extends the
possibility for engagement and interaction and introduces a possibility
for participation (guestbooks, chats, blogs…)
– Web 2.0: connecting and spreading the experience through the users’
own networks

56. The Outsider taking the prize
56

57. Engagement and identification
• The use of the TV media’s strenghts: ‘Storytelling classic’
to create emotional intensification
– A dramatic plot: the contest-format
– The use of classical dramatic agents (most prominent in the first two
seasons of the show): the good vs. the bad
– Use of personal and emotionally loaded stories
– Use of emotionally manipulative editing: production of ’magic
moments’ (the Poul Potts-trick!): close-ups, cross-editing, tears,
tears and more tears…
– Website: augmentary media with a surplus of background materials
about participants, their reactions to the judges and so on: extends the
possibility for engagement and interaction and introduces a possibility
for participation (guestbooks, chats, blogs…)
– Web 2.0: connecting and spreading the experience through the users’
own networks

60. Engagement and identification
• The use of the TV media’s strenghts: ‘Storytelling classic’
to create emotional intensification
– A dramatic plot: the contest-format
– The use of classical dramatic agents (most prominent in the first two
seasons of the show): the good vs. the bad
– Use of personal and emotionally loaded stories
– Use of emotionally manipulative editing: production of ’magic
moments’ (the Poul Potts-trick!): close-ups, cross-editing, tears, tears
and more tears…
– Website: augmentary media with a surplus of background
materials about participants, their reactions to the judges and so
on: extends the possibility for engagement and interaction and
introduces a possibility for participation (guestbooks, chats,
blogs…)
– Web 2.0: connecting and spreading the experience through the users’
own networks

61. 61

62. 62

63. 63

64. 64

65. Engagement and identification
• The use of the TV media’s strenghts: ‘Storytelling classic’
to create emotional intensification
– A dramatic plot: the contest-format
– The use of classical dramatic agents (most prominent in the first two
seasons of the show): the good vs. the bad
– Use of personal and emotionally loaded stories
– Use of emotionally manipulative editing: production of ’magic
moments’ (the Poul Potts-trick!): close-ups, cross-editing, tears, tears
and more tears…
– Website: augmentary media with a surplus of background materials
about participants, their reactions to the judges and so on: extends the
possibility for engagement and interaction and introduces a possibility
for participation (guestbooks, chats, blogs…)
– Web 2.0: connecting and spreading the experience through the
users’ own networks: RSS-feeds, apps for mobile phone,
Facebook profile, Twitter profile.

66. 66

67. 67

68. 68

69. 69

70. 70

71. 71

72. Live integration of
social media during
shows
72

73. Second Screen
73

74. Engagement and identification
• The use of the TV media’s strenghts: ‘Storytelling classic’
to create emotional intensification
– A dramatic plot: the contest-format
– The use of classical dramatic agents (most prominent in the first two
seasons of the show): the good vs. the bad
– Use of personal and emotionally loaded stories
– Use of emotionally manipulative editing: production of ’magic
moments’ (the Poul Potts-trick!): close-ups, cross-editing, tears, tears
and more tears…
– Website: augmentary media with a surplus of background
materials about participants, their reactions to the judges and so
on: extends the possibility for engagement and interaction and
Storytelling 2.0
introduces a possibility for participation (guestbooks, chats,
blogs…) - participation
– Web 2.0: connecting and spreading the experience through the
- co-creation
users’ own networks: RSS-feeds, apps for mobile phone,
Facebook profile, Twitter profile.

75. X-factor cross-media
Other media
communication experience+
May not be (fully) controlled Other DR
radio and TV
shows
Website Backstage TV-show
Aftenshowet
Updates: Mobile phone
RSS, app,
FB, Twitter
Live events
Viewers
DR blogs
+ Aftenshowet’s and other
Red arrows = participation and co-creation DR TV and radio shows’ website

76. X-factor is more than just at TV
show
• As a media event X-factor transgresses its
boundaries as a stand-alone TV show
• It invites the viewer not just to a TV
experience but to become a participant in a
collective course of events
• The viewer can get involved, participate and
have influence on several levels
• And different media play specific – and
coordinated – roles according to their
strengths in creating this cross-media
experience.

77. Experience universe
Harry Potter

78. Rich media experience
• The cross-media story about Harry Potter is not
told by one single media which the other media
relates to in a hierarchical sense.
• Although it all starts with the novels of J.K.
Rowlings, the movies based on the novels can be
seen quite independent of the novels.
• And the games (primarily) based on the movies,
may also be played quite independently.
• As such the cross-media structure of Harry Potter
as an experience universe consists of 7 books, 7
movies and 7 games in three interconnected
series each dealing with the same narrative across
the 3 media – a year in the life of Harry Potter at
the Hogwarts school of sorcery.

79. Rich media experience
• The possibility for engagement and
participation is ensured by the
implementation of websites related to each
novel/movie/game.
• The experience is richened by the
existence of websites (J.K. Rowling’s own
Potter-site, various fansites etc.), books
and games relating to the entire Harry
Potter-universe across the 7-year episodic
plot-structure etc.

91. JK Books based Games based on
Fansites, fx on the entire
Rowlings the entire Merchan
Harry Potter universe, e.g. universe, e.g.
official -dise
Fan Zone Quidditch LEGO Harry
website
throuout times Potter Years 1-4
The entire Harry Potter universe
Book series Film series Game series
Harry Potter Harry Potter Harry Potter
and the and the and the
Sorcerers Sorcerers Sorcerers
Stone Stone Stone
(book) (movie) (game)
Website Website Website
User

92. Experience universe
• As a cross-media production Harry Potter
produces not just an augmentation of the
experience of a specific media.
• It creates an experience universe in which the
user is offered a rich media experience in words,
moving pictures and interactive action.
• Storytelling classic: novels, movies
• Storytelling 2.0 (participation): computer games,
playable merchandise (e.g. LEGO), interactive
features on official websites (e.g. jkrowling.com)
• Storytelling 2.0 (co-creation): fan-sites and other
forums for users expanding on the HP-universe
(e.g. by writing fanfiction)

93. This lecture
• Some brief words about storytelling and
cross-media communication – the general
idea
• Rich media experiences: from experience
+ to experience universes – case studies:
X factor and Harry Potter
• Storytelling 2.0 – case study: an
augmented reality game

94. Trust no-one!
A conspiracy play in the King’s
Kolding
“Mixed reality, ubiquitous
computing and augmented places
as format for communicating
culture”

95. Project scope
• Mobile phones (smart phones) used for
communicating culture
• Fiction used for communicating history
• Experiments with Augmented Reality (at low
costs)
• Creating an unorthodox city walk:
– instead of an exhibition about renaissance
Kolding, we let the renaissance pop up in the city
space
• The audience as participants and co-
creators

96. Format not just for
the design process,
but for ’the exhi-
bitions’ itself

97. Project scope
• Mixed media:
– mobile phone as ’swizz army knife’
– mash-up of variety of services: low-cost and easy to
adjust (Layar, Google Maps, Youtube and other file-
sharing services)
• Ubiquitous computing:
– not so much embedded in the fabric of physical
location
– but accessible everywhere by ways of…
• Mobile and location sensitive media:
• Over-layering locations with digital information:
• Augmentation!

99. Augmentation
• an informational, aesthetical and/or
emotional enhancement of our sense and
experience of place by use of various
framing strategies (e.g. Ian Rankin’s
Edinburgh) and media technologies (e.g. a
guided Rebus Tour).

100. Augmentation of places
• Construction of a kind of mixed reality
• the place has a status both as an actual
location in the physical world and as a
storyspace
• blend of fact and fiction
• blend of physical and mediated space
• blend of presentation and (user)
performance
• ‘charged spaces’
100

101. 101

102. Split reality vs Mixed reality
• Split reality: switching between mediated
space (e.g. inside the mobile phone) and
physical space
• Mixed reality: blending between mediated
and physical space (e.g. looking at physical
space through an ‘augumented reality
browser’ on the mobile phone)
• Mixed reality implies a certain way of telling
stories connecting the actual and the fictional
space/the physical space and the mediated
space
• (this is where Hikuin’s Vendetta goes wrong – and we
try to make things right) 102

104. 104

105. Kolding as augmented
storyspace
• Creating a dramatic meta-story connecting different location specific
narrative tableaus containing various actual historical characters
and events
– (e.g. the co-operation between the public executioner and the
pharmacist selling human fat and pulverized sculls for medical use)
• within the same fiction frame providing connections between the
narrative tableaus
– (the castle is on fire (which is an actual event), a messenger is found
murdered, a conspiracy against the King may be afoot).
• The tale is taking place in the city space and interfaces with specific
locations with historical significance
– (e.g. the square where executions took place, the building housing the
pharmacy)
• Thus: a mediated version of renaissance Kolding is mapped onto
the physical – and present-day – version of the city.

108. Kolding as augmented
storyspace
• Creating a dramatic meta-story connecting different location
specific narrative tableaus containing various actual historical
characters and events
– (e.g. the co-operation between the public executioner and the
pharmacist selling human fat and crushed sculls for medical use)
• within the same fiction frame providing connections between
the narrative tableaus
– (the castle is on fire (which is an actual event), a messenger is
found murdered, a conspiracy against the King may be afoot).
• The tale is taking place in the city space and interfaces with
specific locations with historical significance
– (e.g. the square where executions took place, the building
housing the pharmacy)
• Thus: a mediated version of renaissance Kolding is mapped
onto the physical – and present-day – version of the city.

109. Physical space as media
• The physical space is to some degree
functioning as media communicating specific
types of information, specific types of stories.
• the city quarters with its streets, alleys, buildings,
ornamentations such as statues, gargoyles and so on
function as a narrative architecture like a theme/themed
park like Disneyland including buildings and landscapes
known from the catalog of Disney fairytales
• Several parts of the city of Kolding used as
location for the “Trust No-one!” project have
these qualities of being media in themselves,
as carriers of the story of Kolding.
109

111. Physical space as media
• With the use of mobile phones equipped
with navigation tools and augmented
reality browsers this information residing in
the very architecture and infrastructure of
the city may be pulled forth and made
visible, accessible and interactive from
the perspective of communicating history
and cultural heritage.
111

118. 118

119. Storyspace
Meta-story Narrative tableaus

120. Summing up
• Augmentation as a storytelling 2.0-strategy
makes us see things in new ways:
• Buildings are not just buildings, streets are
not just streets – the carry stories, they carry
cultural meaning
• This meaning may be experienced through
an interplay between the physical locations of
the city and the ubiquitous and locative
information layers provided by mobile media.
• Connecting the dots, moving through physical
and media space guessing the answer to
who the murderer is constitutes the
participatory and co-creative dimension.

121. Visit the project on Facebook
• https://www.facebook.com//Stolpaaingen#!
/Stolpaaingen
• Online, open-accessed development site