Unlike other technologies, AR and VR has the potential to enable people to experience what someone else is seeing, hearing and feeling. This talk describes the coming age of Empathic Computing and how AR and VR technology can be combined with wearable physiological sensors to create shared empathic experiences. Examples will be shown from current research projects from leading groups from around the world, and important directions for future work will be presented. Augmented World Expo (AWE) is back for its seventh year in our largest conference and expo featuring technologies giving us superpowers: augmented reality (AR), virtual reality (VR) and wearable tech. Join over 4,000 attendees from all over the world including a mix of CEOs, CTOs, designers, developers, creative agencies, futurists, analysts, investors, and top press in a fantastic opportunity to learn, inspire, partner, and experience first hand the most exciting industry of our times. See more at http://AugmentedWorldExpo.com

1. THE COMING AGE OF
EMPATHIC COMPUTING
Mark Billinghurst
mark.billinghurst@unisa.edu.au
June 2nd 2016
AWE 2016 Conference
Santa Clara, USA

3. Current AR/VR/Wearable Technology

4. Applications Enabled
•Gaming
•Education
•Health
•Medicine
•Training
•Tourism
•Etc

5. Looking to the Future

6. Empathy
“Seeing with the Eyes of another,
Listening with the Ears of another,
and Feeling with the Heart of another..”
Alfred Adler

8. Empathic Computing
1. Understanding: Systems that can
understand your feelings and emotions
2. Experiencing: Systems that help you
better experience the world of others
3. Sharing: Systems that help you better
share the experience of others

9. 1. Understanding: Affective Computing
• Ros Picard – MIT Media Lab
• Systems that recognize emotion

10. Example: Affectiva
https://www.youtube.com/watch?v=6QDMNPhMJI4

11. 2. Experiencing: Virtual Reality
"Virtual reality offers a whole different
medium to tell stories that really connect
people and create an empathic connection."
Nonny de la Peña
http://www.emblematicgroup.com/

12. Using VR for Empathy
• USC Project Syria (2014)
• Experience of Terrorism • Project Homeless (2015)
• Experience of Homelessness

13. Project Homeless
https://www.youtube.com/watch?time_continue=1&v=SSLG8auUZKc

14. CHILDHOOD (Siggraph 2015)
• Kenji Suzuki, University of Tsukuba
• What does it feel like to be a child?
• VR display + moved cameras + hand restrictors

15. CHILDHOOD Demo
https://www.youtube.com/watch?v=V3OghiKZxz4

16. Sharing: Augmented Reality
Can we develop systems
that allow us to share what
we are seeing, hearing and
feeling with others?

17. Empathy Glasses (CHI 2016)
• Combine together eye-tracking, display, face expression
• Impicit cues – eye gaze, face expression
++
Pupil Labs Epson BT-200 AffectiveWear
Masai, K., Sugimoto, M., Kunze, K., & Billinghurst, M. (2016, May). Empathy Glasses. In Proceedings of
the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM.

18. AffectiveWear – Emotion Glasses
• Photo sensors to recognize expression
• User calibration
• Machine learning
• Recognizing 8 face expressions

19. Empathy Glasses in Use
• Eye gaze pointer and remote pointing
• Face expression display
• In future integrated eye-tracking/display

20. Empathy Glasses Demo

21. Lessons Learned
• Pointing really helps in remote collaboration
• Makes remote user feel more connected
• Gaze looks promising
• Shows context of what person talking about
• Establish shared understanding/awareness
• Face expression
• Used as an implicit cue to show comprehension
• Limitations
• Limited implicit cues
• Task was a poor emotional trigger
• AffectiveWear needs improvement

22. Empathic VR Environments
• Player and Viewer
• Viewer slaved to player
• Share emotional signals
• Heart rate, GSR
• Remote affect measuring

23. Demo: Empathic VR

24. AR and VR for Empathic Computing
• VR systems are ideal for trying experiences:
• Strong story telling medium
• Provide total immersion/3D experience
• Easy to change virtual body scale and representation
• AR systems are idea for live sharing:
• Allow overlay on real world view/can share viewpoints
• Support remote annotation/communication
• Enhance real world task

25. Conclusions
•Trend towards Empathic Computing
• Understanding, Experiencing, Sharing
•AR/VR Enables Empathic Experiences
• Changes perspective
• Sharing space/experience
• Supports annotation/communication
•Many directions for future research

26. www.empathiccomputing.org
@marknb00
mark.billinghurst@unisa.edu.au