Entertainment Software Accessibility - Improving Game Accessibility with Vibrotactile-Enhanced Hearing Instruments - Tessendorf, Bernd (f)

1. 13th International Conference on Computers
Helping People with Special Needs
July 2012, Linz, Austria
Improving
Game Accessibility with
Vibrotactile-Enhanced
Hearing Instruments
Bernd Tessendorf, Peter Derleth, Manuela Feilner, Daniel Roggen, Thomas Stiefmeier, Gerhard Tröster

2. Video: http://www.ingenhindring.shdir.no/

3. 3

4. Locating sound sources
comfort issue
safety issue,
e.g. traffic situation
4
4

5. 5

6. Vibrotactile Feedback
sight hearing haptic
~ 70% ~ 20%
6
[1] Raising awareness about space via vibro-tactile notifications, Andreas Riener, Alois Ferscha, EuroSSC, 2008, Zürich.

7. Vibrotactile Feedback
Fast user response time
Information parallel to
sound
sight hearing haptic
Integration into
hearing instruments
~ 70% ~ 20%
7

8. Coin shaped vibration motor
10mm
...how to integrate into hearing instruments?
8

9. Commercial FM-Receiver
9

10. 10

11. Integration of Vibrotactile Feedback
11

12. Architecture
Matlab USB Bit Whacker Vibration motors
PWM
Potentiometer
Xsens MT9 IMU
12

13. Architecture
Matlab
User-in-the-loop
USB Bit Whacker
PWM
Vibration motors
Flexible generation of
vibration patterns Potentiometer
Xsens MT9 IMU
Affordable off-the-shelf
components
13

14. Application to Gaming
Can vibrotactile feedback
support HI users in gaming?
14

15. Gaming is very impressive and popular...
à gaming market 2008: 50 Billion US$,
with growing tendency [1]
[1] PricewaterhouseCoopers: Entertainment and Media Outlook 2009–2013 15

16. Gaming is very impressive and popular...
à gaming market 2008: 50 Billion US$,
with growing tendency [1]
à impulses for, e.g., GPU development due to
extreme resource requirements
à impulses also for assistive technologies and HCI
[1] PricewaterhouseCoopers: Entertainment and Media Outlook 2009–2013 16

17. Gaming is very impressive and popular...
à gaming market 2008: 50 Billion US$,
with growing tendency [1]
à impulses for, e.g., GPU development due to
extreme resource requirements
à impulses also for assistive technologies and HCI
à accessible gaming: e.g. Closed Captioning
...but can also be very violent.
[1] PricewaterhouseCoopers: Entertainment and Media Outlook 2009–2013 17

18. DOOM 3[CC] http://vimeo.com/13340812 Nominated IGF Choice Award for Best Doom3 MOD by Reid Bryant Kimbal

19. 3D Shooter as an evaluation scenario
• High cognitive load on user
• fast gameplay, reaction time critical
• sound cues are important information here
• background music is present
• Reproducible daily-life scenario
• Open Source: possible to enhance
with assistive technology

20. Explorative Evaluation with HI Users
Tactograms
User Studies
20

21. Pre-Study: Tactograms
21

22. Variants of Tactograms
Resolution, #segments
Intensity levels
Patterns
22

23. Tactograms
Left front
Right
Left right
Right
Left back
Right
0 sec 0,2 sec 0,4 sec 0,6
motor 'on' motor 'off'
23

24. Tactograms
24

25. Pre-study
16 normal hearing participants:
7 males, 9 females, age 23 – 61
Sessions of 90 minutes each
Measurement of accuracy and
response times
25

26. Calculation of angular deviation
26

27. Calculation of angular deviation
27

28. Summary ofRMS Lateralization Error
User Error Rate and
Pre-Study Results
Trade-Off:
Minimal quantization(4error
Segments, 1 Intensity)
(due to the encoding)
(6 Segments, 1 Intensity)
VS the number of user errors
(due to misinterpretation of (8 Segments, patterns)
presented 1 Intensity)
(12 Segments, 1 Intensity)
Trade-Off:
(8 Segments, 3 Intensities)
Response time
VS minimum lateralization error Intensities)
(12 Segments, 3
Choosing the most suitable
pattern is application-specific!
28

29. Vibrotactile Pattern
...to encode direction of rockets fired at the user.
29

30. 2nd Pre-Experiment (4 HI users, 5 normal hearing people)
...only sounds (no visual) of rockets
Behind Left Right
No Game Music
Game Music

31. 2 nd Pre-Experiment
31

32. 2 nd Pre-Experiment
Need to support HI users
with sound localization
32

33. User Study (4 HI users, 5 normal hearing people)
• HI users with and without system
• Normal hearing without system
• >10h of gaming data, >4500 rockets fired
• Measure: hit rate in gameplay

34. 34

35. User Study
Normal hearing people
outperform HI users
All HI users perform
sight better with system than
hearing haptic
without the system20%
~ 70% ~
35

36. Conclusion
Vibrator
Integration of a vibrotactile Feedback
into hearing instruments!
36

37. Outlook
• Standardize API for vibrotactile
feedback, e.g. like in HTML5
• Deployment in parallel to other assistive
technologies
• Confirmation of, e.g., volume change
• Alarm in emergencies
• Warning if battery is empty

38. 13th International Conference on Computers
Helping People with Special Needs
July 2012, Linz, Austria
Questions?
Comments?
Feedback?
Bernd Tessendorf
tessendorf@ife.ee.ethz.ch
http://www.wearable.ethz.ch/