This presentation presents a novel scanning mechani

1. Navigating a 3D Avatar using a Single Switch
Eelke Folmer, Fangzhou Liu, Barrie Ellis - FDG’11, Bordeaux
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

2. Interaction
★2D ★3D
★Text ★Visual
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

3. Users with Disabilities
“ The eiffel tower
was built in ...”
Screen reader
Assistive
Technology
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

4. “Gamers” with Disabilities
“ ...........”
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

5. Severe Motor Impairments
Quadriplegics
one handed eye tracker quad controller switch
controller Player Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

6. Adapted Controllers
1 analog
? 2-5 analog
more 13 buttons
5-7 binary
Ability
mapping
1 analog
less
1 binary
Human Computer Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

7. Switch Controller
jellybean sip/puff pinch
Severe Motor/Cognitive impairments
Sturdy / Cheap
No calibration / easy to use
Binary input (on/off)
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

8. Input Reduction
?
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

9. Input Reduction
?
Automate
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

10. Input Reduction
?
Automate
Remove
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

11. Input Reduction
?
Automate
Remove
Scanning
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

12. Scanning 101
scanning rate
A B C
Selection Set Input generated
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

13. Scanning 101
09
scanning rate
A B C C
Selection Set Activate Input generated
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

14. Scanning Pattern
A
E B
A B C
D E F G C
G H I F D
E
Cartesian Polar
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

15. Scanning Pattern
A
E B
A B C
D E F G C
G H I F D
E
Cartesian Polar
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

16. Scanning Pattern
A
E B
A B C
D E F G C
G H I F D
E
Cartesian Polar
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

17. Scanning Control
hold to scan
A B C
Not always possible
Unable to hold switch (Arthritis)
Sip and puff
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

18. Scanning Control
release to select
A B C C
Not always possible
Unable to hold switch (Arthritis)
Sip and puff
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

19. Discrete Selections
Text Entry H E L L O
Web Navigation
Menu Navigation
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

20. Avatar Navigation
400ms 600ms 600ms 800ms
Selection set [FORWARD,LEFT,RIGHT,BACK]
Types of input:
Continuous [FORWARD] for 300ms
Mixed [FORWARD + TURN RIGHT]
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

21. Continuous input approximation
[Forward, Le1, Right, Back] selection set
[Forward] 300ms input to generate
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

22. Continuous input approximation
[Forward, Le1, Right, Back] selection set
[Forward] 300ms input to generate
[Forward] [Forward] [Forward] 100 ms
3 switch activations
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

23. Continuous input approximation
[Forward, Le1, Right, Back] selection set
[Forward] 300ms input to generate
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

24. Continuous input approximation
[Forward, Le1, Right, Back] selection set
[Forward] 300ms input to generate
[Forward] 200 ms
1 switch activations
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

25. Continuous input approximation
[Forward, Le1, Right, Back] selection set
[Forward] 300ms input to generate
[Forward] 200 ms
1 switch activations
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

26. Mixed Inputs Approximation
[Forward+Le1] 200 ms
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

27. Mixed Inputs Approximation
[Forward+Le1] 200 ms
[Forward] [Le1] 200 ms
2 SA
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

28. Mixed Inputs Approximation
[Forward+Le1] 200 ms
[Forward] [Le1] 200 ms
2 SA
[Forward] [Le1] [Forward] [Le1] 100 ms
4 SA
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

29. Navigating an avatar
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

30. Navigating an avatar
[Forward] [Le1] [Forward] [Le1] [Forward] [Right] [Forward] [Forward] [Forward] [Right] [Right]
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

31. Navigating an avatar
Slow
Cumbersome
Unnatural
[Forward] [Le1] [Forward] [Le1] [Forward] [Right] [Forward] [Forward] [Forward] [Right] [Right]
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

32. Research Question
Can we develop a more
efficient scanning system
that can generate continuous
and mixed inputs?
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

33. Improved Scanning Control
Continuous input
[Forward]
Discrete [Forward] [Forward] [Forward] [Forward]
hold release
hold and release [Forward]
less efficient more efficient
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

34. Generating Mixed Inputs
★Extending selection set
[Forward, Le%, Right, Back, Forward+Right, Forward+Le%, Back
+Le%, Back+Right]
[Forward+Le1]
★Multistep selection
[Forward, Le%, Right, Back] dynamically generated
[Forward] [Cancel, Le%, Right]
[Le1]
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

35. Selection set order
[Forward] [Forward, Le%, Right, Back] cost
RT
[Back, Le%, Right, Forward]
wait wait wait 3SR+RT
More frequently used inputs should be at the
front of the selection set
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

36. Parameters to Compare
control Discrete vs Hold and Release 2
mixing Extension vs Multistep
dia 100,200....1,000 ms 10
selection [Forward, Le%, Right, Back] 24
sets
[Forward, Le%, Right, Back, Forward+Right, 40,320
Forward+Le%, Back+Le%, Back+Right]
[Forward, Le%, Right, Back] [Cancel, Input, Input] 144
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

37. Parameters to Compare
control Discrete vs Hold and Release 2
mixing Extension vs Multistep
dia 100,200....1,000 ms 10
~400k
selection [Forward, Le%, Right, Back] 24
sets
[Forward, Le%, Right, Back, Forward+Right, 40,320
Forward+Le%, Back+Le%, Back+Right]
[Forward, Le%, Right, Back] [Cancel, Input, Input] 144
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

38. Characterizing Avatar Navigation
Forward 873
Forward+Left 220
Left 293
Left+Forward 167
Forward 675
Second Life Key Logger
8 Able bodied users
Focus on grounded navigation
Navigated in SL Environment for 8 minute
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

39. Keystroke frequency
be the
Table 1: Characteristics of Avatar Navigation. rearran
Input Single Multi Total Time of occu
FORWARD 74.9 - 46.8 79.4
RIGHT 10.6 - 6.6 6.1
LEFT 12.7 - 8.0 7.0 4.4
BACK 1.9 - 1.2 1.8 Adding
FORWARD+RIGHT - 46.7 17.5 10.3 facilita
FORWARD+LEFT - 52.4 19.6 8.9
BACK+RIGHT - 0.5 0.19 0.12 1. E
se
BACK+LEFT - 0.5 0.19 0.21
total 100% 100% 100% 100% 2. M
F
key that is held the longest (86.92% of single & multi) of the
4,300 keystrokes totalLEFT are used m
total time participants held keys. RIGHT and
Single: 62.6% comprise 46.21% and ca
less frequently in single strokes but
Multi: 37.4%
52.71% of multistrokes. This confirms our assumption that
Each s
multistrokes are primarily used to adjust the course of an
Player-Game Interaction Research ciency
avatar. BACK is rarely used. We consider the collected data Reno
University of Nevada,
to be representative of 3D navigation behavior in general to scan
Thursday, June 30, 2011

40. keystroke length
min: 26ms
max: 30,606ms
average: 804 ms (δ=1,404)
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

41. Parameters to Compare update
control Discrete vs Hold and Release 2
mixing Extension vs Multistep
40
dia 25, 50,....1,000 ms
selection [Forward, Le%, Right, Back] 7 sets total 24
sets
[Forward, Le%, Right, Back, 120
Forward+Right, Forward + Le%]
[Forward, Le%, Right, Back, Forward+Right, 40,320
Forward+Le%, Back+Le%, Back+Right]
[Forward, Le%, Right, Back] [Cancel, Input, Input] 144
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

42. Parameters to Compare update
control Discrete vs Hold and Release 2
mixing Extension vs Multistep
40
dia 25, 50,....1,000 ms
selection [Forward, Le%, Right, Back] 7 sets total 24 ~3.3 M
sets
[Forward, Le%, Right, Back, 120
Forward+Right, Forward + Le%]
[Forward, Le%, Right, Back, Forward+Right, 40,320
Forward+Le%, Back+Le%, Back+Right]
[Forward, Le%, Right, Back] [Cancel, Input, Input] 144
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

43. Simulator
Simulator
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

44. Simulator
keystrokes
Forward 873
Forward+Left 220
Left 293 Simulator
Left+Forward 167
Forward 675
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

45. Simulator
keystrokes
Forward 873
Forward+Left 220
Left 293 Simulator
Left+Forward 167
Forward 675
[Forward, Le1, Right, Back]
selection sets
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

46. Simulator
keystrokes
Forward 873
Forward+Left 220
Left 293 Simulator
Left+Forward 167
Forward 675
[Forward, Le1, Right, Back] [Forward, Back, Right, Le1]
[Back, Le1, Forward, Right]
[......]
selection sets permutations
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

47. Simulator
25<i<1,000, i+=25
dia
keystrokes
Forward 873
Forward+Left 220
Left 293 Simulator
Left+Forward 167
Forward 675
[Forward, Le1, Right, Back] [Forward, Back, Right, Le1]
[Back, Le1, Forward, Right]
[......]
selection sets permutations
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

48. Simulator
25<i<1,000, i+=25
dia
keystrokes
Forward 873 average per stroke:
Forward+Left 220
★time (ms)
Left 293 Simulator ★switch activations
Left+Forward 167
Forward 675 ★error (ms)
[Forward, Le1, Right, Back] [Forward, Back, Right, Le1]
[Back, Le1, Forward, Right]
[......]
selection sets permutations
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

49. Simulator
25<i<1,000, i+=25 SR = 1,000ms
dia RT = 650ms
keystrokes
Forward 873 average per stroke:
Forward+Left 220
★time (ms)
Left 293 Simulator ★switch activations
Left+Forward 167
Forward 675 ★error (ms)
[Forward, Le1, Right, Back] [Forward, Back, Right, Le1]
[Back, Le1, Forward, Right]
[......]
selection sets permutations
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

50. Results
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

51. Results
Discrete
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

52. Results
Hold and release approximates keystrokes the best.
H&R [Multistep] Significantly faster p<.01
»time: 1,539 ms per stroke
»error: 122 ms (~35 cm) H&R 2.4x slower
than keyboard
Discrete [Extended]
»time: 3,442 ms per stroke
»error: 114 ms (~32 cm)
H&R [Extended] Significant fewer errors p<.01
»time: 2,444 ms per stroke
»error: 0 ms Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

53. Demo / Implementation
Human Computer Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

54. Limitations
User error modeling is limited
assume user has perfect timing skills
»selection accuracy >90% but not 100%
correction may be difficult --> extended set
model error using Bayesian noise & error
correction strategies.
switch users may develop different behavior --
>remove [Back]
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

55. Future Work
Other VW functions
»Object interaction
User studies with
switch users
Real time response
environments (FPS/
Racing game)
Hold and Release
integrated in console
OS as HUD
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

56. Questions
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011

57. Questions
?
Player-Game Interaction Research
University of Nevada, Reno
Thursday, June 30, 2011