1. ABE:
An
Agent-­‐Based
So/ware
Architecture
for
A
Mul9modal
Emo9on
Recogni9on
Framework
Javier
Gonzalez-­‐Sanchez,
Maria-­‐Elena
Chavez-­‐Echeagaray,
Robert
Atkinson,
Winslow
Burleson
Mo9va9onal
Environments
Group
|
Learning
Science
Research
Lab
School
of
Compu9ng,
Informa9cs,
and
Decision
Systems
Engineering
This
work
was
supported
by
Office
of
Naval
Research
under
Grant
N00014-­‐10-­‐1-­‐0143

2. I.
Background
Discovering Importance for Emotions*,
MIT Media Lab
!"#$%&'(#)*%&$&+),*-&.+/$&+)0102*%)
Human-Computer =*+0/+.)) ;*,-*:$&+) ?+2*.,($&+)
Interaction >*6/-*0) %*-<(+/0%0) @#.&,/2<%)
(HCI)
4,(/+5(6*0)
71*)%&6*%*+20)
8(-/(#)*9:,*00/&+0)
Affective
Computing ;<10/&#&./-(#)0/.+(#0)
30*,) A(5)'(2() 4*#/*B0) =2(2*)
* Picard, Rosalind W.; , "Affective Computing: From Laughter to IEEE," Affective Computing, IEEE Transactions on , vol.1, no.1, pp.11-17, Jan.
2010. doi: 10.1109/T-AFFC.2010.10
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

3. I.
Background
!"#$%&'(#)*%&$&+),*-&.+/$&+)0102*%)
=*+0/+.)) ;*,-*:$&+) ?+2*.,($&+)
>*6/-*0) %*-<(+/0%0) @#.&,/2<%)
4,(/+5(6*0)
71*)%&6*%*+20)
8(-/(#)*9:,*00/&+0)
;<10/&#&./-(#)0/.+(#0)
30*,) A(5)'(2() 4*#/*B0) =2(2*)
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

4. I.
Background
“Most software today is very much
like an Egyptian pyramid with
millions of bricks piled on top of each
other, with no structural integrity, but
Affective just done by brute force and
Computing
thousands of slaves” *
* ACM Queue A Conversation with Alan Kay Vol. 2, No. 9 – Dec/Jan 2004-2005
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

5. 2.
Mo2va2on
(a) product line = empathetic systems
(b) software qualities
(c) save time and take advantage of resources
Affective Computing
Researcher
Software Architect
(a) proof-of-concept system to collect data and
validate technology approaches
(b) one-of-a-kind applications
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

6. 3.
Macro-­‐Architecture
Agent
Federation
* B. Horling, and V. Lesser, “A survey of multi-agent organizational paradigms,” The Knowledge Engineering Review, Cambridge University
Press, 2005, vol. 19,! pp. 281-316, doi: 10.1017/S0269888905000317.
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

7. 3.
Macro-­‐Architecture
Agent
Federation
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

8. 4.
Quali2es
Pattern
Based Reusable
Keeping objects small and specialized promotes loose coupling.
Integrability
To develop components separately and then put them together.
Flexible
“Patterns encapsulate the concept that varies”
Objects delegate responsibility to smaller specialized objects. Objects
differing slightly in functionality are composed of smaller objects rather
than implementing most of the functionality in a common base class.
Extendability
It is easier to change or add an object implementing an interface rather
than anticipate all future functionality.
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

9. 4.
Quali2es
Run-time
and
Deployment Performance
Total response time.
Latency
Delay incurred in communicating a message.
Easy to Install
“one-click set up”.
Variable group of programmers
A programmer is in the project between 4 months and 1 year
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

10. 4.1
Scenarios
40 students
independently
Data Logger
Agent Data Visualizer
Agent
Agent
Centre
Multimodal
Tutoring
System
37 student concurrently
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

11. 4.1.
Scenarios
Third-party
Systems
VISUALIZER
gaze-x, gaze-y, time
frustration
threshold = 0.75
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

12. Conclusions
!
ABE
as
our
architectonical
proposal
for
a
mul9modal
emo9on
recogni9on
framework
that
supports
the
crea9on
of
empathe9c
systems.
!
It
is
rooted
in
an
agent-­‐based
approach
under
a
mul9layer-­‐distributed
architecture
oriented
to
create
highly
reusable,
flexible,
integrable
and
extensible
so/ware
components.
!
It
integrates
both
novel
and
well-­‐known
sensing
devices,
including
brain
computer
interfaces,
eye
tracking
systems,
computer
vision
systems
and
physiological
sensors.
!
It
achieves
good
performance
results.
!
Future
work
includes
improving
latency
for
real-­‐9me
reac9ve
systems.
Javier
Gonzalez-­‐Sanchez
|
Maria-­‐Elena
Chavez-­‐Echeagaray

13. Thanks !
Javier Gonzalez-Sanchez
javiergs@asu.edu
Maria-Elena Chavez-Echeagaray
helenchavez@asu.edu
This
work
was
supported
by
Office
of
Naval
Research
under
Grant
N00014-­‐10-­‐1-­‐0143