A lecture delivered for Human-Computer Interaction, a post-graduate level discipline taught by Dr. Sabin Buraga at Faculty of Computer Science, Alexandru Ioan Cuza University of Iasi, Romania.
Visit also https://profs.info.uaic.ro/~busaco/teach/courses/hci/hci-film.html
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
HCI 2018 (9/10) Affective Factors. From Emotion to Persuasive Technologies
1. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
affective factors
Human-Computer Interaction
www.edsurge.com/news/2013-02-19-the-new-face-of-adaptive-learning
2. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
“Designers think, so people can feel.”
Juan-Carlos Fernandez
3. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Affective aspects of interaction (Rogers, 2007)
expressivity
how the interface look & feel could affect the user?
4. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Affective aspects of interaction (Rogers, 2007)
frustration
how we can detect and reduce it?
5. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Affective aspects of interaction (Rogers, 2007)
persuasive technologies
how we can change the user attitude and behavior
by using the existing technologies?
6. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Emotion is a reaction to events deemed relevant
to the needs, goals, or concerns of an individual
Scott Brave & Clifford Nass, 2008
7. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Emotion represents the experience in UX
Kristina Hook, “Affective Computing”, The Encyclopedia
of Human-Computer Interaction (2nd Edition), 2015
www.interaction-design.org/encyclopedia/affective_computing.html
8. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Emotion encompasses physiological, affective,
behavioral, and cognitive components
Scott Brave & Clifford Nass, 2008
9. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion in the cognitive context – Moore (2005)
10. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
Emotions as discrete categories
basic emotions (Paul Ekman et al., 1972):
anger
disgust
fear
happiness
sadness
surprise
11. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
Emotions as discrete categories
additional emotions (Paul Ekman, 1990):
amusement contempt contentment
embarrassment excitement guilt
pride in achievement relief satisfaction
sensory pleasure shame
12. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
Emotions as multi-dimensional models
circumplex model (James Russell, 1980)
2 axes:
arousal (calm↔excitement) + valence (pleasantness)
13. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
14. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
Emotions as multi-dimensional models
circumplex model (James Russell, 1980)
used to test stimuli of emotion words,
emotional facial expressions, and affective states
15. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
Emotions as multi-dimensional models
Plutchik’s wheel of emotions (Robert Plutchik, 1980)
8 primary bipolar emotions:
joy vs. sadness; anger vs. fear;
trust vs. disgust; surprise vs. anticipation
+
more advanced emotions
16. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
18. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
Emotions as multi-dimensional models
positive activation—negative activation (PANA)
Watson and Tellegan, 1985
PAD (Pleasure, Arousal & Dominance)
emotional state model – Mehrabian, 1997
used to study nonverbal communication
19. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
Lövheim cube of emotion (Hugo Lövheim, 2012)
a theoretical model considering the relationships
between monoamine neurotransmitters
(serotonin, dopamine, noradrenaline) and emotions
20. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
Emotions as multi-dimensional models
Parrott’s classification (Gerrod Parrott, 2001)
primary emotions: love, joy, surprise, anger, sadness, fear
21. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: classification
Emotions as multi-dimensional models
Parrott’s classification (Gerrod Parrott, 2001)
primary emotions: love, joy, surprise, anger, sadness, fear
secondary emotions – e.g., love: affection, lust, longing
24. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: modeling languages
Emotion Annotation and Representation Language
HUMAINE Project, 2006
emotion-research.net/projects/humaine/earl/
<complex-emotion
xlink:href="user-face-when-using-instant-messaging-mobile-app.jpg">
<emotion category="pleasure" intensity="0.7"/>
<emotion category="worry" intensity="0.5"/>
</complex-emotion>
25. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: modeling languages
Emotion Markup Language (EmotionML) 1.0
W3C Recommendation, 2014
manual annotation of material involving emotionality
automatic recognition of emotions from sensors
generation of emotion-related system responses
www.w3.org/TR/emotionml/
26. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: modeling languages
Vocabularies for EmotionML
W3C Working Group Note, 2014
www.w3.org/TR/emotion-voc/
representing emotions
in terms of scientifically valid descriptors:
categories, dimensions, appraisals, action tendencies
27. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
<emotionml xmlns="http://www.w3.org/2009/10/emotionml"
xmlns:meta="...">
<info><meta:name>Emotional state of the robot</meta:name></info>
<!-- Robot's current global state configuration:
negative, active, powerless (using PAD model) -->
<emotion dimension-set=
"http://www.w3.org/TR/emotion-voc/xml#pad-dimensions">
<dimension name="pleasure" value="0.2"/>
<dimension name="arousal" value="0.8"/>
<dimension name="dominance" value="0.3"/>
</emotion>
<!-- Robot's action tendencies: want to recharge -->
<emotion action-tendency-set=
"http://human-robot-interfaction.info/custom/action/robot.xml">
<action-tendency name="charge-battery" value="0.9"/>
<action-tendency name="seek-shelter" value="0.7"/>
<action-tendency name="pickup-boxes" value="0.1"/>
</emotion>
28. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
<!-- Appraised value of incoming event:
obstacle detected, appraised as novel and unpleasant -->
<emotion appraisal-set=
"http://www.w3.org/TR/emotion-voc/xml#scherer-appraisals">
<appraisal name="suddenness" value="0.8" confidence="0.4"/>
<appraisal name="intrinsic-pleasantness" value="0.2"
confidence="0.8"/>
<reference role="triggeredBy" uri="file:data.xml#obstacle27"/>
</emotion>
<!-- Robot's planned facial gestures: will frown -->
<emotion category-set=
"http://human-robot-interfaction.info/custom/robot-emotions.xml"
expressed-through="face">
<category name="frustration"/>
<reference role="expressedBy"
uri="file:behavior-repository.xml#frown"/>
</emotion>
</emotionml>
29. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
neurological structure of emotion (LeDoux, 1996)
30. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
limbic system – the “seat of emotion” – evaluates
the need/goal relevance of its inputs
31. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
thalamus + limbic system mainly deal with
primitive (basic) emotions
e.g., startle-based fear, anger, sadness, joy, disgust
32. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Emotions that require more extensive cognitive
(i.e., knowledge-based) processing are called secondary
33. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Emotions that require more extensive cognitive
(i.e., knowledge-based) processing are called secondary
examples:
frustration
pride
satisfaction
in the cortex
34. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
An emotion can result from a combination of both
thalamic-limbic and cortex-limbic mechanisms
35. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
An emotion can result from a combination of both
thalamic-limbic and cortex-limbic mechanisms
an event causing an initial fear reaction
can be later recognized as harmless
by more extensive, rational evaluation
examples: screensaver initialization, a warning message
36. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Emotions could be learned from the social environment
37. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion versus mood
emotions are intentional
38. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion versus mood
emotions are intentional
“they imply and involve relationships
with a particular object”
Frijda, 1994
39. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion versus mood
moods are not directed at any object in particular and
are experienced as more diffuse, global, and general
40. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion versus mood
moods are not directed at any object in particular and
are experienced as more diffuse, global, and general
a person can be sad about something (an emotion)
or generally depressed (a mood)
41. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion versus mood
from a functional point of view,
emotions could determine the execution of task(s)
42. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion versus mood
emotions cause or contribute to moods
43. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion versus sentiment
sentiments are not states of an individual,
but assigned properties of an entity (person, object)
44. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion versus sentiment
sentiments can persist indefinitely, by contrast with
emotions (seconds) or moods (hours/days)
45. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion versus sentiment
sentiments are often generalizations about a class of
objects with a given recognizable property
46. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
emotion versus sentiment
sentiments are often generalizations about a class of
objects with a given recognizable property
these generalizations must not necessarily be logical
“I hate spreadsheet applications!”
47. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
How about the role of emotion in HCI?
www.affectiva.com/product/emotion-as-a-service/
48. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Emotion has influences on
attention
memory
performance
opinion (assessment)
Scott Brave & Clifford Nass, 2008
49. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Attention
an interface capable of detecting – or at least predicting –
a user’s emotional or mood state could similarly assume
an affect-regulation role, helping to guide attention away
from negative and toward more positive stimuli
50. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Memory
negative events, which tend to be highly arousing,
are typically remembered better than positive events
51. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Memory
negative events, which tend to be highly arousing,
are typically remembered better than positive events
avoid error messages + patronizing contents
52. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Memory
emotionality “improves memory for central details,
while undermining memory for background details”
Heuer & Reisberg, 1992; Parrott & Spackman, 2000
53. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
User performance
keeping a user happy may, therefore,
not only affect satisfaction,
but may also lead to efficiency and creativity
54. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
the circumplex model in the context of user experience
Pieter Desmet & Paul Hekkert, 2007
55. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
P. Desmet & P. Hekkert, “Framework of Product Experience”,
International Journal of Design, Vol. 1, No. 1 (2007)
www.ijdesign.org/index.php/IJDesign/article/view/66/15
product
emotional
experience
user-product interaction
aesthetic
experience
experience
of meaning
56. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
model of product impact
S. Fokkiga et al., “From Product to Effect. Towards a Human-Centered Model of
Product Impact”, Design Research Society, 2014
www.drs2014.org/media/655163/0386-file1.pdf
57. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
User performance
negative example:
objects that appear or move unexpectedly
– e.g., pop-ups, animations – and loud or sharp noises
are likely to trigger startle-based fear
58. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Assessment
mood influences judgment and decision making
users in a good mood will likely judge
both the interface and their work more positively,
regardless of any direct emotional effects
59. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Causes:
needs & goals
Maslow’s hierarchy of basic needs
A. H. Maslow, Motivation and Personality, Harper & Brothers, 1954
physiological, safety/security, social, esteem,
cognitive, aesthetic, self-actualization, transcendence
60. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
commons.wikimedia.org/wiki/File:Maslow's_hierarchy_of_needs.svg
61. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Causes:
needs & goals
interfaces can also directly address
the basic needs of a certain user
62. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Causes:
needs & goals
example (Fogg, 1998):
establishing a trusting and safe relationship with users
63. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Causes:
needs & goals
example (Brave & Nass, 2006):
educational software should address users’ emotional
needs, not only teaching the relevant content,
but also ensuring users believe that they are learning
64. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Causes:
obstacles & control
Ellsworth (1994)
if user has the control over the software,
the negative affective consequences are least important
65. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Causes:
social context – “agency”
(Ellsworth, 1994; Friedman & Kahn, 1997)
“This system failed to understand your command.” – blaming itself
“The command was not understood.” – blaming no one
“You did not speak clearly enough for your command
to be understood.” – blaming the user
66. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
blaming
itself
67. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Causes:
previous emotional state
repeated pleasurable affective states, therefore,
become expected and thus gradually lose intensity
68. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
How emotion could be detected?
neurological response – via electroencephalogram (EEG)
or magneto-resonance imaging (MRI)
69. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
How emotion could be detected?
autonomous activity of the human organism
breath
facial expression
voice
70. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
categorization of facial responses to determine
which emotions they express
Facial Action Coding System – FACS
(Paul Ekman et al., 1978, 2002)
71. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
interpreting micro-expressions
(e.g., pushed up cheeks, dropping upper eyelids,
eyebrows raised, nose wrinkling,…) – via FACS –
and body language to detect emotions
see also Applying Facial Imaging To Understanding Consumer
Emotional Response (2011) – tinyurl.com/k47dhen
temasys.com.sg/vidyoedm22102013/
72. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
revealing true emotions through micro-expressions
current approaches use machine/deep learning
see also github.com/topics/emotion-recognition
Web-based solutions: Emotion As A Service
temasys.com.sg/vidyoedm22102013/
insights.sei.cmu.edu
73. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
How emotion could be detected?
behavioral changes
74. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
6 + 1 emotions with universal facial gestures
context: game design
(N. Lazzaro, Why We Play: Affect and the Fun of Games, 2008)
Emotion Example
Frustration Figuring out how to get character off a roof in Tom Clancy’s Splinter
Cell (usability issues that detract from the player experience)
Fear Falling into boiling lava,
fast-moving projectiles aimed at the player in Doom
Surprise Using Myst’s linking books for the first time to transport to
a new world
Sadness When the young magician Aerith, in Final Fantasy VII is murdered
Amusement When two Sims get married in The Sims, or rolling
over and picking up sumo wrestlers in Katamari Damacy
Disgust Becoming a social outcast (social disgust)
after losing the dancing challenge in Sid Meier’s Pirates
Curiosity Wanting to know what happens by driving the race track
the wrong way in Project Gotham Racing 3
75. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Expressivity
induces an emotional state via UI elements:
color, icons, sound, graphics,…
76. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Expressivity
lack of expressivity in communication at a textual level
emoticons + Emoji
:) :| >:-( 👀 👍 💆
77. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Expressivity
lack of expressivity in communication at a textual level
emoticons + Emoji
findings (N. Aldunate & R. González-Ibáñez, 2017):
“emoticons contribute to a greater social presence” +
“constitute a valuable resource for language compre-
hension by providing expressivity to text messages”
www.ncbi.nlm.nih.gov/pmc/articles/PMC5216035/
78. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
Expressivity
abbreviations could have emotional connotations
see leet speak
I 12 CU 2NITE
79. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
To stimulate emotions, products must have
attributed a personality based on appearance
and interaction means
Reeves & Nass, 1989
80. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
How to create emotions in a person?
81. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
Ekman, 2004
82. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
a dialog window
appears
83. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
thinking about
previous message
84. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
remembering
last game played
85. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
thinking about
what happens
now on Web
86. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
discussing UI
features with others
87. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
another person’s
facial expression
(video-chat)
88. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
another person’s
review of a task
89. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
deleting others’ files
90. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion
1. Auto appraisal
2. Reflective appraisal
3. Memory of an emotion
4. Imagination
5. Talking about
6. Empathy or witnessing another’s emotion
7. Instruction by others on how to feel
8. Violation of social norm
9. Making facial expression of an emotion
laughing after defeat
in front of friends makes
it feel more positive
91. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
metaphors
+
idioms
92. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
metaphors + idioms
“Metaphors create insight. But they also distort.
They have strengths. But they also have limitations.”
Gareth Morgan
93. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
metaphors + idioms
use metaphors to convey and/or create
mental associations
Dan Saffer, 2005
94. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
metaphors + idioms
proper metaphors used to denote UI objects/devices
contribute to the personification of the software,
increasing the usability
95. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
icons
denote a relation between a visual representation
and a certain concept
based on similarity – e.g., an empty page = new file
analogue – example: scissors = cut operation
arbitrary (based on an idiom) – e.g., X = close window
96. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
icons
most effective are the icons designed with respect to
similarity between visual representation and the concept
Rogers, 2007
97. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
arbitrary applications
system programs
remark: the orientation is different
98. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
icons
beware of the cultural context
99. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
map symbols/icons: Japanase (top) vs. Finish (bottom)
example
volcanic crater, fire station, government office building,…
computer-based cross-cultural communication design
A. Heimbürger et al., “Cross-cultural communication with icons & images”, 2014
www.researchgate.net/publication/289548098_Cross-cultural_communication_with_icons_and_images
100. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
earcons
earcon = the aural equivalent of an icon
101. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
earcons
earcon = the aural equivalent of an icon
using sounds to denote a specific concept, event, or object
M. Blattner et al, “Earcons and Icons”,
Human-Computer Interaction, Vol. 4 (1989)
www.daimi.au.dk/~dsound/DigitalAudio.dir/Papers/Earcons_and_Icons.pdf
102. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
earcons
design guidelines (Karen Kaushansky, 2012):
choose the right type of audio
embed meaning in audio earcons
design in context
consider the “non-use cases”
decide between recorded prompts and text-to-speech
www.smashingmagazine.com/2012/09/14/guidelines-for-designing-with-audio/
103. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
earcons
examples:
mobile phones, operating system sound schemes,
instant messaging, e-games, smart watches,
3D environments,…
commonly, not (to be) used in the Web context
104. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
tactons
communicate non-visual information
by using the tactile sense
“structured tactile messages” (Brewster & Brown, 2004)
eprints.gla.ac.uk/3443/1/tactons_aussi.pdf
105. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
tactons
communicate non-visual information
by using the tactile sense
a common example:
coded tactile representations (Braille code)
text informationnon-visual form
106. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
tactons
context:
wearable computing
107. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
tactons
examples:
haptic I/O devices – e.g., sensors
case study:
Lorna Brown, Tactons: Structured Vibrotactile Messages
for Non-Visual Information Display, Microsoft Research (2006)
www.microsoft.com/en-us/research/video/tactons-structured-vibrotactile-messages-for-
non-visual-information-display/
108. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
abstract
representation
cognitive
dimension
usage
icon visual conveying information
earcon audio attention
tacton haptic appeal to a known touch
emoticon emotion conveying
an emotional state
gesticon dynamic indicating a state change
kineticon dynamic state change, progress
some are more suitable to (visually) impaired users
109. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
persuasive computing
110. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Positive psychology
hedonomics
focusing on the pleasant or enjoyable aspects of HCI
Peter Hancock et al., “Hedonomics: The Power of Positive and
Pleasurable Ergonomics”, Ergonomics in Design 13(1), 2005
journals.sagepub.com/doi/abs/10.1177/106480460501300104
111. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Persuasive (seductive) technologies
interactive computational systems deliberately designed
to change human attitudes and behaviors
captology
B.J. Fogg (1996) – computers as persuasive technologies
captology.stanford.edu
112. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
mobile devices
Web sites/apps
e-games
exercise
equipment
smart
environments
augmented/
virtual reality
…
attitude change
behavior change
user motivation
change in
worldview
compliance
cap-
tology
persuasion
computers
see also C. Allen, Tactics of Persuasion & Influence (2010)
www.slideshare.net/ChristopherA/tactics-of-persuasion-influence-bgiedu
113. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Persuasive (seductive) technologies
persuasion in the context (Maria Miceli et al., 2011):
accidental vs. intentional
communicative vs. non-communicative
coercive vs. non-coercive
115. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Persuasive (seductive) technologies
case studies:
games with social impact – for people with special needs
applications helping the cure of certain phobias
infant simulators
virtual pets – to diminish “bad habits” of children
…
116. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Design-A-Plant – a knowledge-based learning
environment project to investigate interactive problem-
solving with animated pedagogical software agents
(Elliot et al., 1997) – condor.depaul.edu/elliott/papers/ijcai97/ij.html
case study
117. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Evaluating self-help mobile apps providing
support for people with depression
117 apps identified
only 10.26% (n = 12) have support that seems to be
consistent with evidence-based principles of
CBT (cognitive behavioral therapy)
or BA (behavioral activation)
UI evaluated according to Nielsen’s usability heuristics
A. Hugget et al., “A systematic review of cognitive behavioral therapy
and behavioral activation apps for depression”, PLoS ONE 11(5), 2016
journals.plos.org/plosone/article?id=10.1371/journal.pone.0154248
case study
118. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Persuasive (seductive) technologies
interesting uses in the HCI context:
get understanding of user behavior
get understanding of cognitive factors
119. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
case study
an iOS app that encourages
kids to talk about feelings
context:
emotional learning
for other examples, browse
www.participate.com/search
?q=Emotional%20Learning
120. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Persuasive (seductive) technologies
multimodal factors (Guerini, 2011):
embodied conversational agents – ECAs
kinetic typography
music
use of images
virtual characters
121. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Virtual characters
avatars – artificial (visual) representations of people
122. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Virtual characters
user-facing agents
showing simulated human behavior
(knowledge, inferring, choices,…artificial intelligence)
for details, study C. Grimme et al., “Social Bots:
Human-Like by Means of Human Control?”, Big Data, 5(4), 2017
arxiv.org/pdf/1706.07624.pdf
123. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Virtual characters
autonomy no human intervention
reactivity takes stimuli from environment
proactivity takes initiative to act
collaboration able to collaborate with other agents
124. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Virtual characters
autonomy no human intervention
reactivity takes stimuli from environment
proactivity takes initiative to act
collaboration able to collaborate with other agents
multi-agent systems
125. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
intelligent virtual agents having human
appearance and the capability to teach
foreign language vocabulary
M. Macedonia et al., “Intelligent virtual agents as language trainers
facilitate multilingualism”, Frontiers in Psychology (2014)
www.frontiersin.org/articles/10.3389/fpsyg.2014.00295/full
case study
126. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
The use of anthropomorphism
adding human qualities to virtual things
reducing the user’s anxiety regarding computers
(computer and/or Internet phobia)
127. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
The use of anthropomorphism
adding human qualities to virtual things
reducing the user’s anxiety regarding computers
(computer and/or Internet phobia)
larger context: assistive technologies
www.interaction-design.org/literature/article/understand-the-social-needs-
for-accessibility-in-ux-design
128. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
The use of anthropomorphism
example:
“Hi, Dan, welcome back! What we learnt last time?
Oh, yes, that experiment on virtual reality…
Let’s repeat it!”
versus
“User 33, start the experiment 69!”
129. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
The use of anthropomorphism
warnings:
user might feel inferior, anxious, or stupid
130. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Affective interaction
goal:
a program/device able to register emotions
and other stimuli and to generate a correct response
to these stimuli
131. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Affective interaction
input:
sensors measuring (or learning) the user affective state
facial expression, gestures,
heartbeat rate, blood pressure, transpiration, speech
132. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Affective interaction
output:
expressing empathy using computational approaches
“intelligently” / “emotionally” reacting on a situation
133. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
case study
human-robot interaction (HRI)
experiments using 3 robot embodiments:
Elaine Schaertl Short et al., “Understanding Agency in Interactions Between
Children With Autism and Socially Assistive Robots”,
Journal of Human-Robot Interaction, 6(3), 2017
mobile humanoid robot vs. mobile box robot vs. non-mobile toy
134. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Aspect of interest: computational humour
humour is a powerful generator of emotions
theories of humour applied in computer science
details in P. Petta et al. (Eds.),
Emotion-Oriented Systems, Springer, 2011
135. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Aspect of interest:
computational humour
humour generation
building computational models
to generate humourous content (text)
www.sirifunny.com
136. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: design solutions
Aspect of interest: computational humour
humour recognition
identifying humour in human-generated content
(e.g., natural language, images) and/or human behavior
Emotion: design solutions
137. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: abuse on software
Humans could have an offensive behavior
regarding the digital entities
www.agentabuse.org
138. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: abuse on software
Humans could have an offensive behavior
regarding the digital entities
example (Tatsuya Nomura, 2005):
impolite behavior of the patients,
when they discovered that the therapist is a virtual agent
139. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: abuse on software
Humans could have an offensive behavior
regarding the digital entities
example (Antonella De Angeli & Rollo Carpenter, 2005):
“injuries” on conversational – e.g., IRC – robots:
“liar”, “stupid”, “dumb” software
140. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: abuse on software
Humans could have an offensive behavior
regarding the digital entities
experiment (Christoph Bartneck et al., 2005):
humans tend to hurt more easily robots/devices
than people
141. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: open questionHow does emotion play out
in computer-mediated communication?
hypothesis:
communication of emotions is more difficult
in computer-mediated communication (CMC)
than in face-to-face (F2F) communication
discussion
142. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: open questionHow does emotion play out
in computer-mediated communication?
findings:
there is no indication that CMC is a less emotional or
less personally involving medium than F2F
Daantje Derks et al., “The Role of Emotion in Computer-Mediated
Communication: A Review”, Computers in Human Behavior (2007)
www.researchgate.net/publication/222428783_The_role_of_emotion_in_computer-
mediated_communication_A_review
discussion
143. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: open question
With which emotion
should HCI designers be most concerned?
144. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: open question
When and how should interfaces attempt
to directly address users’ emotions and
basic needs (versus application-specific goals)?
145. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Emotion: open question
When and how should users be informed
that their affective states are being monitored
and adapted to?
146. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
IBM Watson Developer Cloud
Emotion Analysis service detects anger, disgust, sadness,
fear, and joy, from text (comments, online reviews,…)
Tone Analyzer provides insights about the emotion,
social and writing tones reflected in text
github.com/watson-developer-cloud/
case study
147. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
IBM Watson Personality Insights extracts and analyzes
a spectrum of personality attributes to help discover
actionable insights about people and entities
demo: personality-insights-demo.ng.bluemix.net
case study
148. Master on Software Engineering :: Human-Computer Interaction
Dr. Sabin-Corneliu Buraga – profs.info.uaic.ro/~busaco/
Face API – detect, identify, analyze, tag faces
+ recognize emotions in photos
www.microsoft.com/cognitive-services/ aischool.microsoft.com
case study
Microsoft Cognitive Services