What we learn about technology from people living with dementia
1. What we learn about technology from
people living with dementia
Jim Ang
PanoteSiriaraya
School of Engineering and Digital Arts
University of Kent
1
2. Jim Ang
• Game and virtual world enthusiast
• Email junky
• Dream about uploading my conscious to the net
• Lecturer in multimedia and digital systems
• Backgrounds in human computer interaction/man-
machine interaction
2
3. PanoteSiriaraya
3
• Background in Human-Computer Interaction with
an emphasis on designing technology to support aging
• Likes to build and program stuff
(web app, simulators for rescue robots and recently virtual
worlds)
• Research Associate at University of Kent
4. The project
• Funded by EPSRC for 13 months (from Oct 2012)
• In collaboration with Silverfit and Avante care homes in
UK and the Netherlands
• AgeUK and Society Alzheimer’s Society
4
7. 3D virtual worlds
• Aim:
– How can 3D virtual worlds be designed to support older people
(in the care environment) and carers
• Why virtual ?
– Access to certain objects which may be otherwise not accessible
physically
– Able to (in principle) visit any locations
– Potentially cheaper and safer than physical activity
7
8. Natural user interface
• Interaction with 3D virtual worlds is complex
• No straightforward mapping between 2D interaction
devices (keyboard and mouse) to 3D space.
• Gesture-based interaction
– Kinect
– Leap motion
8
11. Initial observations
• Residents were disoriented and confused when using the virtual
room
• They were unable to associate their actions with the on screen
avatar
• The “picking up objects” system was confusing
– Avatar’s hand went through the table
• giving them complete interactivity freedom (just do whatever you
want and the avatar will reflect this) is not supported by Kinect yet.
11
15. Lesson learnt about virtual world technology
for dementia
• They love technology!
• The experience of virtual worlds could help in the
recovery of self through place-based reminiscence
• Safe and expressive virtual wonderland
• Virtual worlds could provide a “sanctuary” to maintain the
continuity of self
15
16. Muscle memory
• Visual, audio memory – used a lot in reminiscence
therapy
• Research has suggested that the ways in which we think
is an extension of our existence as embodied minds. The
way we think is driven by our physical motions, which in
turn drive the way that we think.
• Triggering muscle memory with gesture interaction ?
16
17. Presence
• For virtual world to be effective, users should be able to
suspend their disbelief and maintain the illusion that the
alternative
• almost all residents were able to see the projected
screen as a place they were situated in and perceive
themselves as actually performing the activity
17
18. Tangible user interface/physical computing
• Although these prototypes worked for many residents,
they didn’t work for those with a more severe condition
• Touch appeared important!
• Internet of things / ubiquitous computers
• Linking physical to virtual
18
23. Day-to-day objects as interaction devices
• NFC
23http://www.youtube.com/watch?v=Q3jjFjIJJRU
24. Lesson learnt about (virtual world)
technology
• Our interaction with technology focuses too much audio
and visual – what about physical touch?
• Virtual world technology should not just be “virtual.” It
can include a physical element
• Fully immersive vs mixed reality
24
We had the idea that instead of using the normal mouse and keyboard, that we would use some kind of gesture based interactiontechnology instead.Basically, moving around in a 3D space is hard. What normally happens, if you play these kind of 3D games is if you want to do something like say pick up an apple, you have to press the buttons in your keyboard or mouse and move your character, turn left , turn right and go through that complex interaction to move your character to the apple and then press another button to pick it up.So we felt that it would not be so straightforward to learn to map these kind of 2D interactions to a 3D space for older people.So in the end, we decided to use gesture interaction, specifically the Microsoft Kinect which had just been released.