This talk is about how language plays a pivotal role in creating meaningful experiences beyond interaction design. It was delivered by David Sherwin at Emily Carr University of Art and Design on March 30, 2011.

1. the language of interaction
david sherwin
emily carr university
march 30, 2011
Tree of Codes by Jonathan Safran Foer, from Visual Editions

2. so, what do you do?
interaction designer
Photo 1314251438 by HeyRocker on Flickr / CC share remix license

3. uh… what’s occupation?
What’s your that?
hmm… well, how much time
do you have for me to tell you?
Photo 1314251438 by HeyRocker on Flickr / CC share remix license

4. “interaction designers design for everyday life
by shaping those human experiences that sit at
the intersection of needs [desireability], business
goals [viability], and technology [feasibility].”
—www.ecuad.ca/programs/undergrad/bdesign/interaction
"…interaction design is first and foremost
the design of behavior that occurs over time."
—Alan Cooper, David Cronin, Robert Reimann, About Face 3
"interaction is how people relate to other
people through the mediating influence
of product."
—Richard Buchanan
"interaction designers make awesome
products, services, and systems that

5. “my husband studies the ways
people interact with technology,
then determines if any thing
should be made better,
and if so, how.”
—Mary Paynter Sherwin

6. “my husband studies the ways
people interact with technology,
then determines if any thing
should be made better,
and if so, how.”
—Mary Paynter Sherwin

7. technology: “the sum of the
ways in which social groups provide
themselves with the material
objects of their civilization.”

8. frogMOB:
Work Bikes

9. Jan Chipchase, The Mobile Frontier:
A study of mobile money use in Afghanistan

10. technology changes human behavior,
human behavior changes technology

11. “my husband studies the ways
people interact with technology,
then determines if any thing
should be made better,
and if so, how.”
—Mary Paynter Sherwin

12. “my husband studies the ways
people interact with technology,
then determines if any thing
should be made better,
and if so, how.”
—Mary Paynter Sherwin

13. 10,000
omission and/or transcription errors
can occur per year for a typical
200-bed hospital or clinic
how can we minimize these errors
and their impact on patient care?

15. prototypes from
participatory design testing
sessions at frogSF with
doctors and nurses

16. testing tasks via a high-fidelity
interactive prototype on a
touch tablet PC screen

17. industrial
design
interaction
design
communication
design

18. Tree of Codes by Jonathan Safran Foer, from Visual Editions

19. let’s try a little experiment…
let’s read a poem three ways:
1 you read it in your mind
2 all of you read it out loud
3 i read it and you listen

20. “Thought is thought, and not
anything more. Seeing is seeing.
What is is what is. These three together
are what they are, and their total is one,
which is what there is and is equal
to zero. A is not A; one is not one;
this too is the rule.
Only insofar as one is
speechless can one really
think with words.”
—from Robert Bringhurst’s “Saraha’s Exercise for Beginners,”
in the book Pieces of Map, Pieces of Music

21. what just happened?
we illustrated the 3 challenges
both artists and designers face
in trying to craft interactions

22. material you are
processing
metaphor the poem in
practice your head,
formulating
its intent

23. “In interaction design,
the system is the material.”
—Scott Nazarian, Creative Director, frog design

24. TEXT
WATCH
EMAIL
CALL
READ
language is the first material
we may apply to an interaction
PLAY
STATUS CHAT

25. language is an interface
for prototyping an experience…
…and a well-designed product or
service expresses a vocabulary
that matches people’s needs

26. how do you cook pasta?
Photo 197840294 by Oskay on Flickr / CC share remix license

27. how do you cook pasta?
verbs connote action
Photo 197840294 by Oskay on Flickr / CC share remix license

28. how do you cook pasta?
who will initiate or receive the action?
Photo 197840294 by Oskay on Flickr / CC share remix license

29. how do you cook pasta?
nouns are the objects acted upon
Photo 197840294 by Oskay on Flickr / CC share remix license

30. how do you cook pasta al dente?
adjectives are metadata
Photo 197840294 by Oskay on Flickr / CC share remix license

31. can you cook pasta al dente for dinner tonight?
providing the activity’s context
Photo 197840294 by Oskay on Flickr / CC share remix license

32. can you cook pasta al dente for dinner tonight?
Illustration by Mark Baskinger from the forthcoming book Drawing Ideas

33. and make me a cup of iced tea?
“A cup of…” by industrial designer Tithi Kutchamuch from Creative Workshop

34. and can you help me resist eating dessert?
Temptd, a social support application by frog design, partnering with MTV’s I Used to Be Fat

35. individual actions USE CASES
VOCABULARY
IS ACTED UPON
sum of actions
required to TASKS
complete task
CONTEXT
CHANGES
time, place, and CONTEXT
people involved
motivations for GOALS CONSEQUENCES
initiating a task CHANGE
DESIRED BEHAVIOR:
LOSE WEIGHT

36. through research,
we capture the real and the incorporeal:
desires, hopes, beliefs, needs, wants,
frustrations, motivations…

37. through design,
we capture the real and the incorporeal:
desires, hopes, beliefs, needs, wants,
frustrations, motivations…

38. through painting,
we capture the real and the incorporeal:
desires, hopes, beliefs, needs, wants,
frustrations, motivations…

39. through film,
we capture the real and the incorporeal:
desires, hopes, beliefs, needs, wants,
frustrations, motivations…

40. through sculpture,
we capture the real and the incorporeal:
desires, hopes, beliefs, needs, wants,
frustrations, motivations…

42. with the right insight, we can
clarify intent and increase our
support of desired behaviors

43. over time, desired interactions
can accumulate into a liveable story
we hope to inhabit

44. “The Future of Health Care Is Social” by frog design

45. “The secret is in listening to the words,
and arranging, and listening,
and listening again.
—Robert Wallace, Writing Poems

46. material you are
processing
metaphor the poem in
practice your head,
formulating
its intent

47. material i map the intent
of my words
metaphor
to the intent
practice of yours, finding
where A = B

48. “metaphor: a figure of speech
in which a term or phrase is applied
to something to which it is not
literally applicable in order
to suggest a resemblance…”
—Dictionary.com

49. i’m not talking about
small metaphors…

50. …or big, poorly constructed ones
Photo 3928200642 by x-ray delta one on Flickr / CC share remix license

51. metaphors
are at the heart
of how we model
interactive systems

55. PATTERNS
repetitive pathways
and structures
PRINCIPLES
guidelines preserving
the integrity of the above
FEATURE CLUSTERS
functionality organized
fit to the patterns
BEHAVIOR OVER TIME
use of animation,
transition, and motion

58. understand the metaphor of a system,
know how to shape interactions within it

59. new modes of input
allow new metaphors to emerge
“SAM Gets Dirty” by designer Mark Notermann from Creative Workshop

60. material you are
processing
metaphor the poem in
practice your head,
formulating
its intent

61. material i map the intent
of my words
metaphor
to the intent
practice of yours, finding
where A = B

62. material focusing your
individual voice,
metaphor
informed by
practice the experiences
of your peers

63. “A new poem is created by
everyone that reads poetically—
not that its raw material is original…
but that every individual brings with him,
when he exercises his individuality,
a way of seeing and feeling that
in its interaction with old material
creates something new, something
previously not existing in experience.”
—from John Dewey’s Art as Experience

64. so it goes for a designer…
our experience is summative

66. Photo 2425646723 by Bekathwia on Flickr / CC share remix license

67. material art and design,
stitched together
metaphor
by technology
practice and language

68. digital + interaction
interactive arts design
animation
film, video +
integrated
technology
media
ceramics
critical and industrial
sculpture design
cultural practice
print media
photography
language arts
visual arts
drawing
illustration communication
painting design
ART “THE INTERFACES” DESIGN

69. language may be our v1 interface
for summarizing experience

70. but what we paint, draw, film, sculpt,
and design with technology…

71. human emotion lives in
the spaces between

72. “Happiness is not a noun or a verb.
It’s a conjunction.”
—Eric Weiner, The Geography of Bliss