Developing games for rehabilitation and the medical field requires input from a variety of sources and stakeholders. A game for rehabilitation can have multiple potential end users, all of whom can have different requirements for the game. Rehabilitation also occurs in a variety of settings all of which have different demands on the player and the game/system. It is a challenge to incorporate the needs of multiple end users and requirements of multiple treatment settings into a single game for rehabilitation. This talk will discuss the challenges of creating a rehabilitation game for multiple end users and the methods used to overcome the challenges. A prototype game for rehabilitation, Mystic Isle, will be used as an example to highlight specific methods, including the user centered design cycle, multiple iterations, and stakeholder engagement through user testing and focus groups.

1. The work depicted here was sponsored by the U.S. Army. Statements and opinions expressed do not necessarily
reflect the position or the policy of the United States Government, and no official endorsement should be inferred.
Rachel Proffitt, OTD, OTR/L
Assistant Professor of Clinical Occupational
Therapy
Division of Occupational Science and
Occupational Therapy
University of Southern California
ONE GAME, MANY USERS
Inclusive Design of Interactive
Technologies for Rehabilitation

2. Acknowledgments
 U.S. Army Research Lab SFC Paul Ray Smith, Simulation and
Training Technology Center (STTC), the Telemedicine and
Advanced Technology Research Center (TATRC) at the US Army
Medical Research and Materiel Command (USAMRMC) (W911NF-
04-D-0005) (PI: Lange)
 National Institute on Disability and Rehabilitation Research (NIDRR)
grant: Optimizing Participation Through Technology: Rehabilitation
Engineering Research Center (OPTT:RERC) (PI: Winstein)
(H133E080024)
 NIH T32 Institutional Postdoctoral Training Grant- TREET: Training
in Rehabilitation Efficacy and Effectiveness Trials (5T32HD064578-
02) (PI: Clark)
 Division of Occupational Science and Occupational Therapy,
Herman Ostrow School of Dentistry, University of Southern
California

3. Game-Based Rehabilitation

4. 4
• Neuroplasticity
• Motor Learning/
Motor Control
Principles
Traditional
Therapy
• Wii-Fit
• Wii-Motion Plus
• EyeToy
• DDR
Wii-Hab

5. 5
What is the quality of movement we are
asking our clients to do?

6. 6
User Testing: Challenges with Off-Shelf-Devices
Level Difficulty
Compensatory Movements
Graphics
Feedback
Data Management
Dynamic
Difficulty
Adjustment

7. 7
• Neuroplasticity
• Motor Learning/
Motor Control
Principles
Traditional
Therapy
• Game Design
• Learning Theories
• Game Play
mechanics
Research and
Development • Wii-hab
• Telemedicine
• Rehab Games
• Virtual Reality
Game-Based
Rehab
• Stakeholders
• Needs Assessment
• Focus Groups
• Usability Testing
Testing
•Pilot Studies
•Case
Controlled
Trials
•Cohort
Studies
•RCTs
Evidence

8. The process

9. 9
Initial Injury
Family and Friends
Timing of Therapy
Skillful Clinicians
Successful Rehabilitation Depends On:

10. 10
Key Stakeholders & User Centered Design
 Patients
 Clinicians
 Occupational Therapists
 Physical Therapists
 Speech Therapists
 Psychiatrist/Neuropsychiatrists
 Physicians/PM&R/Physiatry
 Caregivers
 Family
 Friends
 Paid/unpaid

11. 11
Functional tasks (motor, sensory, cognitive) that need to be included in the game
Is there something that already exists? (off the shelf or developed in our lab: what
technologies (hardware) is available and what games are available?)
Yes No
FOCUS GROUP: What are the user’s thoughts
about the current games? How do they interact
with these games? What are the user’s thoughts
about aspects that could be added to the game?
FOCUS GROUP: User’s thoughts on their
impairment and associated therapy? User’s
suggestions for game play characteristics,
mechanics and goals?
GAME DESIGN and HARDWARE DESIGN: Depending on user feedback and requirements,
hardware is designed and games are designed. These games are designed using the Iterative
design process (outlined by Fullerton et al 2004). This iterative design process involves input
from user groups throughout the process.
USABILITY STUDIES: Users provide feedback about all aspects of the hardware and games.
REVISIONS: Games and hardware revised based on user feedback.
CLINIC BASED ASSESSMENT

12. 12
Functional tasks (motor, sensory, cognitive) that need to be included in the game
Is there something that already exists? (off the shelf or developed in our lab: what
technologies (hardware) is available and what games are available?)
Yes No
FOCUS GROUP: What are the user’s thoughts
about the current games? How do they interact
with these games? What are the user’s thoughts
about aspects that could be added to the game?
FOCUS GROUP: User’s thoughts on their
impairment and associated therapy? User’s
suggestions for game play characteristics,
mechanics and goals?
GAME DESIGN and HARDWARE DESIGN: Depending on user feedback and requirements,
hardware is designed and games are designed. These games are designed using the Iterative
design process (outlined by Fullerton et al 2004). This iterative design process involves input
from user groups throughout the process.
USABILITY STUDIES: Users provide feedback about all aspects of the hardware and games.
REVISIONS: Games and hardware revised based on user feedback.
CLINIC BASED ASSESSMENT

13. 13
Functional tasks (motor, sensory, cognitive) that need to be included in the game
Is there something that already exists? (off the shelf or developed in our lab: what
technologies (hardware) is available and what games are available?)
Yes No
FOCUS GROUP: What are the user’s thoughts
about the current games? How do they interact
with these games? What are the user’s thoughts
about aspects that could be added to the game?
FOCUS GROUP: User’s thoughts on their
impairment and associated therapy? User’s
suggestions for game play characteristics,
mechanics and goals?
GAME DESIGN and HARDWARE DESIGN: Depending on user feedback and requirements,
hardware is designed and games are designed. These games are designed using the Iterative
design process (outlined by Fullerton et al 2004). This iterative design process involves input
from user groups throughout the process.
USABILITY STUDIES: Users provide feedback about all aspects of the hardware and games.
REVISIONS: Games and hardware revised based on user feedback.
CLINIC BASED ASSESSMENT

14. 14
Functional tasks (motor, sensory, cognitive) that need to be included in the game
Is there something that already exists? (off the shelf or developed in our lab: what
technologies (hardware) is available and what games are available?)
Yes No
FOCUS GROUP: What are the user’s thoughts
about the current games? How do they interact
with these games? What are the user’s thoughts
about aspects that could be added to the game?
FOCUS GROUP: User’s thoughts on their
impairment and associated therapy? User’s
suggestions for game play characteristics,
mechanics and goals?
GAME DESIGN and HARDWARE DESIGN: Depending on user feedback and requirements,
hardware is designed and games are designed. These games are designed using the Iterative
design process (outlined by Fullerton et al 2004). This iterative design process involves input
from user groups throughout the process.
USABILITY STUDIES: Users provide feedback about all aspects of the hardware and games.
REVISIONS: Games and hardware revised based on user feedback.
CLINIC BASED ASSESSMENT

15. 15
Focus Group Protocol
 Clinicians
 Current patient populations
 Current rehabilitation protocols
 Use of technology with patients
 Issues with current rehabilitation protocols
 Feedback on game concepts
 Ideas for tasks within game-based tool
 Patient / Client and Caregiver groups
 Current exercise protocols
 Lifestyle and recreational activities
 Social and community participation
 Barriers to exercise and aging with/into disability
 Thoughts on use of technology in the clinic/home
 Feedback on game concepts
 Ideas for maintaining motivation

16. 16
Functional tasks (motor, sensory, cognitive) that need to be included in the game
Is there something that already exists? (off the shelf or developed in our lab: what
technologies (hardware) is available and what games are available?)
Yes No
FOCUS GROUP: What are the user’s thoughts
about the current games? How do they interact
with these games? What are the user’s thoughts
about aspects that could be added to the game?
FOCUS GROUP: User’s thoughts on their
impairment and associated therapy? User’s
suggestions for game play characteristics,
mechanics and goals?
GAME DESIGN and HARDWARE DESIGN: Depending on user feedback and requirements,
hardware is designed and games are designed. These games are designed using the Iterative
design process (outlined by Fullerton et al 2004). This iterative design process involves input
from user groups throughout the process.
USABILITY STUDIES: Users provide feedback about all aspects of the hardware and games.
REVISIONS: Games and hardware revised based on user feedback.
CLINIC BASED ASSESSMENT

17. 17
Functional tasks (motor, sensory, cognitive) that need to be included in the game
Is there something that already exists? (off the shelf or developed in our lab: what
technologies (hardware) is available and what games are available?)
Yes No
FOCUS GROUP: What are the user’s thoughts
about the current games? How do they interact
with these games? What are the user’s thoughts
about aspects that could be added to the game?
FOCUS GROUP: User’s thoughts on their
impairment and associated therapy? User’s
suggestions for game play characteristics,
mechanics and goals?
GAME DESIGN and HARDWARE DESIGN: Depending on user feedback and requirements,
hardware is designed and games are designed. These games are designed using the Iterative
design process (outlined by Fullerton et al 2004). This iterative design process involves input
from user groups throughout the process.
USABILITY STUDIES: Users provide feedback about all aspects of the hardware and games.
REVISIONS: Games and hardware revised based on user feedback.
CLINIC BASED ASSESSMENT

18. 18
Demographics
Common Data Element
Structured Interview
Game Play
 Overall perception of the game
 Overall perception of the technology
 Instructions
 Game elements
 Comparison to current exercise program
 Game Ideas
 Future use of game
User Testing Protocol

19. 19
Functional tasks (motor, sensory, cognitive) that need to be included in the game
Is there something that already exists? (off the shelf or developed in our lab: what
technologies (hardware) is available and what games are available?)
Yes No
FOCUS GROUP: What are the user’s thoughts
about the current games? How do they interact
with these games? What are the user’s thoughts
about aspects that could be added to the game?
FOCUS GROUP: User’s thoughts on their
impairment and associated therapy? User’s
suggestions for game play characteristics,
mechanics and goals?
GAME DESIGN and HARDWARE DESIGN: Depending on user feedback and requirements,
hardware is designed and games are designed. These games are designed using the Iterative
design process (outlined by Fullerton et al 2004). This iterative design process involves input
from user groups throughout the process.
USABILITY STUDIES: Users provide feedback about all aspects of the hardware and games.
REVISIONS: Games and hardware revised based on user feedback.
CLINIC BASED ASSESSMENT

20. 20
Functional tasks (motor, sensory, cognitive) that need to be included in the game
Is there something that already exists? (off the shelf or developed in our lab: what
technologies (hardware) is available and what games are available?)
Yes No
FOCUS GROUP: What are the user’s thoughts
about the current games? How do they interact
with these games? What are the user’s thoughts
about aspects that could be added to the game?
FOCUS GROUP: User’s thoughts on their
impairment and associated therapy? User’s
suggestions for game play characteristics,
mechanics and goals?
GAME DESIGN and HARDWARE DESIGN: Depending on user feedback and requirements,
hardware is designed and games are designed. These games are designed using the Iterative
design process (outlined by Fullerton et al 2004). This iterative design process involves input
from user groups throughout the process.
USABILITY STUDIES: Users provide feedback about all aspects of the hardware and games.
REVISIONS: Games and hardware revised based on user feedback.
CLINIC BASED ASSESSMENT

21. 21
Functional tasks (motor, sensory, cognitive) that need to be included in the game
Is there something that already exists? (off the shelf or developed in our lab: what
technologies (hardware) is available and what games are available?)
Yes No
FOCUS GROUP: What are the user’s thoughts
about the current games? How do they interact
with these games? What are the user’s thoughts
about aspects that could be added to the game?
FOCUS GROUP: User’s thoughts on their
impairment and associated therapy? User’s
suggestions for game play characteristics,
mechanics and goals?
GAME DESIGN and HARDWARE DESIGN: Depending on user feedback and requirements,
hardware is designed and games are designed. These games are designed using the Iterative
design process (outlined by Fullerton et al 2004). This iterative design process involves input
from user groups throughout the process.
USABILITY STUDIES: Users provide feedback about all aspects of the hardware and games.
REVISIONS: Games and hardware revised based on user feedback.
CLINIC BASED ASSESSMENT

22. 22
Tracking Wheelchair Users

23. 23
Tracking User when Clinician is Present

24. 24
User Feedback in the Clinic
Iterative User Group Feedback during Design Process

25. 25
Collaborative Partners and Test Sites

26. The Game

27. 27
Microsoft Kinect Sensor
Field-of-view: 58 degrees horizontal and 45 degrees vertical
Resolution: 640x480 at 30
frames per second.

28. 28
Microsoft Kinect Skeletal Tracking

29. 34
Game-based rehabilitation tool
 Tailored to individual level of ability
 Option for individualized exercise prescription
 Interchangeable graphics and environments
 After action review and data management

30. 35
Calibration

31. 36
Performance Results

33. 38
Jewel Mine: Stepwise menus

34. 39
Jewel Mine: Avatar Representation

35. 40
Jewel Mine: Game Options

36. 41
Jewel Mine: Game Options

38. 43

39. 44

40. 45

41. Evaluation

42. 47
Case-study Clinical Setting
 68 yo female with Parkinsonism (onset in 2009)
 Intervention Dose: 8 x 1-hour sessions over 4 weeks
 Customized JewelMine Intervention
 Cross body and backwards reaching interventions
 Dual tasking with Simon game
 Calibration to patient’s limits of stability
 Clinician programmed gem number/ sequence
 Forward functional reach
 Improved from 6 inches to 9.5 inches
 Maintained at 4 week follow-up: 9.25 inch
 Number of falls between Pre and Post testing
 6 falls reported - None during reaching, turning or dual tasking

43. Case Study & Case Series: Home Setting
 55 year old Male, 39 months post-Stroke (left hemiparesis)
 Intervention Dose: 30-90 minutes/day, 3-7 days/wk – Total 6 weeks
 Customized JewelMine Intervention
 Sitting, Sit to stand, Step up
 Standing right and left, Standing right hand only
 Standing with leg exercises Monitor
Kinect Sensor
Laptop
Wireless
Mouse

44. 49
Real World Comparison: Community Dwelling Older
Adults
• Understood the importance of exercise in
maintaining function and health
(Proffitt & Lange, 2013)
• Perceived virtual environments as more engaging
than real environments for reaching tasks
(Proffitt et al., in press)
• Virtual environments required more attentional
demand than real environments
(Chen et al., in submission)
• Used different reaching strategies in virtual
environments compared to real environments
(Wade et al., in submission)
• Sample of 30 older adults
• Age: 75.2±8.6yrs (range = 59-92)
• Right hand dominant
• 2 x 2 cross-sectional design
• 2 task conditions (virtual and real targets)
• 2 postural demands (standing and stepping)

45. The future

46. 51
Clinical Research – Clinic and Home Settings
 Phase II Clinical Trial
 Collaborations!
 Provide feedback and assist with future
development
 Independent or collaborative studies with
different clinical populations