Hire 💕 9907093804 Hooghly Call Girls Service Call Girls Agency
Modeling Gaze Dynamics in Human-Agent Interactions
1. Modeling the Dynamics of Gaze-
Contingent Social Behaviors in
Human-Agent Interaction
University of Augsburg, Germany
Human Centered Multimedia
Elisabeth André
2. 2
My Background
Social Robotics and Virtual Agents
European and BMBF Projects on Affective
Computing
4. 4
Explicit versus Implicit
Interaction with Eye Gaze
Explicit Interaction:
Open interaction with a system where
humans intentionally input discrete
commands to explicitly express their needs
Implicit Interaction:
Information that people convey indirectly in
a conversation, but which may be derived
from dialogue and context information.
Unconscious Interaction:
Continuous (often nonverbal) behavior
people not voluntarily control, but which
may be (but are not necessarily expected
to be) interpreted as the implicit expression
of a particular need or intention
http://www.vision-systems.com/
5. 5
Eye Gaze to Initiate Contact with
a Human User
Breaking the Ice in Human-Agent
Communication: Eye-Gaze Based
Initiation of Contact with an
Embodied Conversational Agent
Tober et al. IVA 2009
6. 6
Five Phases of Flirting [Givens, 1978]
Attention Phase
Men and women arise each other’s attention
Ambivalent non-verbal behavior
Recognition Phase
One interactant recognizes the interest of the other
He or she may then signal readiness to continue the interaction,
e.g., by a friendly smile.
Interaction Phase
After mutual interest has been established, the man or woman
may be initiated the interaction phase and engage in a
conversation
Sexual-Arousal and Resolution Phases are somehow
missing relevance to human-agent communication.
8. 8
Interaction Modes
Interactive version
Non-interactive version with ideal flirt behavior:
In the non-interactive ideal version the virtual agent behaves like
in the interactive version except for that it does not respond to
the user’s eye gaze behavior, but assumes a perfect eye gaze
behavior from the user and thus follows a fixed sequence.
Non-interactive version with anti-flirt behavior:
Duration of mutual gaze is increased from 3 s to 7 s
Facial expression remains neutral (which can be interpreted as a
bored attitude towards the user)
Virtual agent looks away upwards after gazing at the user
instead of downwards
9. 9
Results
1. In the interactive and the ideal mode, the agent was
able to show the users that Alfred had an interest in
them and the users also had the feeling that he was
flirting with them.
2. We found that the effect was increased when moving
from the ideal to the interactive mode.
3. The interactive version contributed to the user’s
enjoyment, increased their interest to continue the
interaction or even to engage in a conversation with
Alfred.
10. 10
Conclusions
Alfred was lacking of attractiveness, but the eye gaze
enabled agent improved the flirting interaction.
Flirting tactics as implemented in this work are of
benefit to a much broader range of situations with agents
than just dating, e.g. initiate human-agent interaction or
regulating turn-taking in dialogues.
11. 11
Setting
Discovering eye gaze behavior
during human-agent conversation
an interactive storytelling
application. ICMI-MLMI 2010.
12. 12
Gaze Model
Parameters set on the basis of data found in the
literature
Non-interactive Interactive
Looks around
4.0 s
(2-6 s)
4.0 s
(2-6 s)
Gazes at user
(Wait for gaze)
2.0 s
(1-3 s)
2.0 s
(1-3 s)
Mutual gaze n/a
1.0 s
(0.75-1.25 s)
13. 13
Evaluation
Compared the 2 different gaze behavior models:
non-interactive vs. interactive
Study with 19 subjects
How do people respond to different gaze models?
Does the gaze model affect their sense of social presence?
Order of the 2 gaze models was randomized for each
subject to avoid any bias due to ordering effects
17. 17
Results
In total users were much more looking at Emma
compared to human-human interaction
Argyle & Cook Kendon Our Study
Looking at interlocutor 58%
50%
(28% - 70%)
76%
(46% - 98%)
Looking at interlocutor
while listening
75% 81%
Looking at interlocutor
while speaking
41% 71%
18. 18
Conclusions
Interactive gaze mode led to a better user experience
compared to the non-interactive gaze mode
Users adhere to patterns of gaze behaviors for speaker
and addressee that are also characteristic of dyadic
human-human interactions
They looked more often to the virtual interlocutor than is
typical of human-human interactions.
20. 20
Empathetic Artificial Listener
Attention: pay attention to the signals produced by a speaker
Perception of signals
Comprehension: understand meaning attached to signals
Internal reaction: the
comprehension of the
meaning may create cognitive
and emotional reaction
Decision: communication or
not of the internal reaction
Generation: display
behaviors
21. 21
Generation of Facial
Expressions
FACS (Facial Action Coding System) can be used to
generate and recognize facial expressions.
Action Units are used to describe emotional
expressions.
Seven Action Units were identified for the robotic face
(out of 40 Action Units for the human face)
Upper face:
inner brows raiser (AU 1),
brown lowerer (AU 4),
upper lid raiser (AU 5)
and eye closure (AU 43).
Lower face:
lip corner puller (AU 12),
lip corner depressor (AU 15)
and lip opening (AU 25).
22. 22
Social Signal Interpretation:
SSI by Augsburg University
Multiple Sensor Input
ECG, Skin Conduction, Blood
Glucose Level,
Speech, Acceleration, …
Preprocessing and Feature Analysis
Filtering,
Frequency
Analysis,
…
Pattern Recognition
Fusion and
Final Decision
Physiological and
Affective State,
Context Information
SSI is freely available under:
http://www.openssi.net
Johannes Wagner, Florian Lingenfelser, Tobias
Baur, Ionut Damian, Felix Kistler, Elisabeth André:
The social signal interpretation (SSI) framework:
multimodal signal processing and recognition in
real-time. ACM Multimedia 2013: 831-834
29. 29
Multimodal Dialogue with a
Robot
G. Mehlmann, M. Häring, K. Janowski, T.
Baur, P. Gebhard, E. André: Exploring a
Model of Gaze for Grounding in Multi-
modal HRI. ICMI 2014: 247-254
30. 30
Research Strategy
Model of Human Social
Behaviors
Multimodal Behavior Simulation
Corpus on Human Social Behaviors
Build
Simulate
Evaluate
Refine
Statistics
33. 33
Gaze Recognition
The glasses provide the video image and the gaze
coordinates
-, -
-, -
...
156, 543
189, 527
145, 567
211, 542
34. 34
Gaze-Based Disambiguation
3
1
42
Do you mean this red object there?
„Do you
mean this
red object
there?“
1 1 1 2 2 3 3 3 2 1 1 1 2 3Gaze:
Speech:
Use this information
for disambiguation!
Gaze
36. 36
Robot Behavior
The robot‘s behavior depends on the role.
In the speaker role, the
robot awaits the dialog
manager‘s decision to play
a behavior.
In the addressee role, the robot
shows some idle gaze behavior,
occasionally reacting to the users‘
gaze movements, emotional
expressions and other cues.
37. 37
Gaze-based Interaction
Object grounding:
The robot follows the user’s hand movements.
The robot follows the user’s gaze.
Social grounding:
The robot seeks and recognizes mutual gaze.
Turn management:
The robot recognizes when the user yields the turn.
39. 39
Results of a Study
Object grounding was more effective than social grounding.
People were able to interact more efficiently with object grounding.
Social grounding did not improve the perception of the interaction.
Assumption:
People were rather concentrating on the task instead of the
social interaction with the robot.
42. 42
Conclusions
Effect of gaze-aware agents:
Gaze-aware agents have a positive effect on user perception.
Gaze-aware agents improve grounding.
Side effects:
Midas Touch Problem:
• The agent should not respond to each detected gaze behavior.
Unnatural user behavior:
• Use of gaze as a pointing device
Timing is the key.