1. 2010 CRC PhD Student Conference
Effective Tutoring with Affective Embodied Conversational
Agents
Sharon Moyo
menziwa@hotmail.com
Supervisors Dr Paul Piwek
Dr Neil Smith
Department/Institute Computing
Status Part‐time
Probation viva After
Starting date Oct 2007
This natural language generation project aims to investigate the impact of affect
expression using embodied conversational agents (ECAs) in computer‐based learning
environments. Based on the idea that there is a link between emotions and learning,
we are developing an affect expression strategy. We will implement the strategy
within a tutoring system in two domains: Information Technology (IT) and Business
Studies.
Current research has not firmly established the impact of affect expression strategies
within tutorial feedback which supports learners in computer‐based learning
environments [1]. Our approach is to provide affective support through empathy.
Empathy is described as expressing emotion that is based on another’s situation
(target) and not merely one’s own [2]. An individual can show: parallel empathy that
mirrors the target’s emotion; or reactive empathy that might be different to the
target’s emotion [2].
The empathic tutor interventions will be designed to support positive emotions [3]
and reduce negative learner emotions [4] using a range of verbal and non‐verbal (or
multimodal) interventions. These interventions will be combined with corrective and
meta‐cognitive feedback [5] and presented to users as a hint or summary.
We will conduct a series of studies. Initially, we intend to develop implement and
evaluate an algorithm that generates multimodal empathic behaviours using an ECA.
The experiment conditions will include multimodal channels of communication:
speech vs. speech and facial expression vs. speech and gesture vs. speech, facial
expression and gesture. We hypothesize that participants will identify the ECA’s
expression most accurately in the condition using three channels to generate
affective expressions in comparison to the other conditions.
Page 58 of 125

2. 2010 CRC PhD Student Conference
Additionally we aim to evaluate when and how parallel or reactive empathy can be
used to best effect in learning environments. Subsequently, we will integrate the
algorithm into a web‐based tutoring environment and conduct an evaluation in the
domain of Business Studies. Finally, in the main study we will evaluate the empathic
tutoring system in a classroom setting over several weeks in the domain of
Information Technology (IT).
We intend to contribute to current research by describing how an ECA can
effectively express multimodal [6] empathic behaviour within computer‐based
learning. More specifically, we aim to create a framework to model parallel and
reactive empathy and the learning contexts where they can be used in a quiz‐based
web environment. We intend to validate these results through evaluations across
two domains: Information Technology and Business demonstrating that the
framework can be applied to other quiz‐based learning environments.
References:
1. Arroyo, I., et al. Designing Affective Support to Foster Learning, Motivation
and Attribution. in AIED 2009. 2009. Brighton, UK: IOS.
2. Davis, M., Empathy: A Social Psychological Approach. 1994, Madison, WI:
Brown and Benchmark.
3. Bickmore, T. and D. Schulman, Practical approaches to comforting users with
relational agents, in CHI '07 extended abstracts on Human factors in
computing systems. 2007, ACM: San Jose, CA, USA.
4. Burleson, W., Affective learning companions: Strategies for empathetic
agents with real‐time multimodal affective sensing to foster meta‐cognitive
and meta‐affective approaches to learning, motivation, and perseverance. .
2006, Massachusetts Institute of Technology: Cambridge, MA.
5. Tan, J. and G. Biswas. The Role of Feedback in Preparation for Future
Learning: A Case Study in Learning by Teaching Environments. in ITS 2006.
2006: Springer‐Verlag.
6. Cassell, J., et al. Animated Conversation: Rule‐Based Generation of Facial
Expression, Gesture and Spoken Intonation for Multiple Conversational
Agents. . in Siggraph 94, ACM SIGGRAPH. 1994: Addison Wesley.
Page 59 of 125