Presentation slides from Usability Professionals Association Conference (UPA 2010) in Munich, May 26 2010. Please email me for more context and details.

1. Evaluating Touch
Gesture Usability
Kevin Arthur
Synaptics
UPA 2010
1

2. 2

3. Multi-Finger TouchPad Gestures
Two-Finger Two-Finger Three-Finger
Pinch Zoom Rotate Flick
Two-Finger Two-Finger Three-Finger
Scroll Pivot Rotate Press
3

4. Motivation for Testing Gestures
• Formative usability tests for feedback to
developers
Audience • Summative and competitive usability
tests for sales, OEM partners
• Need for repeatable procedures
• Do users understand the gestures?
Questions • Can users successfully perform the
gestures?
• Are gestures satisfying to use?
• Holistic evaluation; test with users in a
realistic scenario on a working system
Context • Different from pure data collection for
testing gesture recognizer offline
4

5. Important Properties of Gestures
• Gestures must be taught; testing the
Lack of Affordances documentation is important
• Performance is not always predictable
Nondeterministic • Recognition trade-offs exist between
and Interdependent gestures in same parameter space
• Gestures should be evaluated as a set
Interface • TouchPad’s primary use is still for
pointing and scrolling; gestures
Overloading shouldn’t interfere
User Variation • Hand size, long fingernails
5

6. Phases in Performing a Gesture
• Introduction through
Exposure documentation or prior use
• Initial touch contact and motion
Registration for gesture recognition
• Gesture is registered and
“locked in” – mode switch
Continuation • Dynamic phase with relaxed
motion requirements
Termination • End position, fingers lifting
6

7. Test Framework
Step Objective
Assess understanding, help
1 Gesture Introduction
materials
[Exposure phase]
Familiarization and
2 Train to basic performance
Practice Task
[All phases]
Obtain rates of correct and incorrect
gesture recognition
3 Accuracy Task
[Registration phase]
Assess satisfaction, ease of use
Satisfaction Questionnaire
4
and Debrief [All phases]
7

8. Gesture Introduction
Simulate new user experience
• “Out of box experience”
material
• Help videos
• Try the gesture until success
• If no success, moderator
assists
8

9. Familiarization and Practice Tasks:
Pinch Zoom
Zoom in on the South
Residences, near the top of
the map, and find the
building called The Knoll.
Zoom all the way in on The
Knoll and then zoom all the
way back out.
2000 × 2000 pixels or higher.
Approximately five pinch-zooms
required.
9

10. 10

11. Flick and Rotate
1. Use the rotate gesture to make the image upright.
2. Type the image’s title into the caption field and press Enter.
3. Use the flick gesture to go to the next image. 11

12. Flick and Rotate
12

13. Accuracy Task
User performs a set number of gesture attempts
Moderator records table of system responses. Example:
No Misrecognized Other/
Gesture Correct
Response As Pinch As Rotate Notes
Pinch Zoom
7 2 - 1
In
Pinch Zoom
8 2 - 0
Out
“requires
Rotate
6 3 1 - too large a
Clockwise
motion”
Rotate
Counter- 7 1 2 -
clockwise
13

14. Accuracy Results
Unified
Measure
100%
90%
80%
70%
60%
50% Misrecognized
40% No Response
30% Correct
20%
10%
0%
Pinch Pinch Rotate CW Rotate Average
Zoom In Zoom Out CCW
14

15. Accuracy Results, Competitive Study
System A System B
Average Correct 82% Average Correct 89%
100% 100%
90% 96 90% 94 96
94
90
88
80% 80% 85
82 83
70% 70% 73
60% 60%
61
50% 50%
40% 40%
30% 30%
20% 20%
10% 10%
0% 0%
2F Pinch 2F Rotate 3F Flick 3F Flick 2F Scroll 2F Pinch 2F Rotate 3F Flick 3F Flick 3F Press 2F Scroll
Zoom LR UD Zoom LR UD
Correct No response Incorrect Correct No response Incorrect
15

16. Accuracy Results, Competitive Study
Correct Gesture Recognition Rates
120
100
80
Correct (%)
System A
60 System B
System C
40
20
98 100 97 62 77 68 92 86 93 97 97 100
0
Pinch Rotate Flick Three-Finger Press
16

17. Satisfaction Questionnaire
17

18. Satisfaction Questionnaire
18

19. Questionnaire Result
I would use this gesture if it were available
(strongly disagree = 1, strongly agree = 9)
Pinch 7.38
Rotate 4.38
3F
7.88
Flick
3F
8.75
Press
2F
8.25
Scroll
1 2 3 4 5 6 7 8 9
Strongly Strongly
Disagree Agree
19

20. Questionnaire,
Competitive Rating Format
How well do the gestures work on each system?
Please rate from 1 (Poor) to 5 (Excellent)
Gesture System A System B System C
Pinch Zoom
Rotate N/A
Flick Left-Right
Flick Up-Down N/A
Three-Finger Press N/A N/A
Two-Finger Scrolling
20

21. Ratings Results
User Ratings With 95% Confidence Intervals
Excellent 5
4
3
2
Poor 1
Pinch Zoom 2F Rotate 1F Rotate Flick LR Flick UD 1F Scroll 1F Scroll 2F Scrolling
(Chiral) (Linear) (Circular)
System A System B System C
21

22. Related Tests & Gesture Side-Effects
Other usability tests for notebook input devices
• Pointing (target acquisition)
• Drag and drop
• Scrolling
• Typing and accidental TouchPad input
Assess gesture side-effects
• Unintended gestures
• Other issues
22

23. Conclusions
Properties of gestures call for careful testing
• Gesture introduction and documentation is key
• Gestures are nondeterministic – less predictable
• Gestures are interdependent; test as a set
Framework of gesture tests
1. Gesture Introduction
2. Familiarity and Practice Tasks
3. Accuracy Tasks
4. Satisfaction Questionnaire and Debrief
5. Related Tests & Gesture Side-Effects
23

24. Resources
References
• Sylvia Le Hong and Dan Mauney, “Cultural Differences and Similarities
in the Use of Gestures on Touchscreen User Interfaces,” UPA 2010.
blog.humancentric.com/gesture-research/
• Mark Billinghurst and Bill Buxton, “Gesture Based Interaction” in Human
Input to Computer Systems (draft), www.billbuxton.com/inputManuscript.html
• Dan Saffer, Designing Gestural Interfaces, O’Reilly 2008.
• Craig Villamor, Dan Willis, Luke Wroblewski, Touch Gesture Reference
Guide, www.lukew.com/touch/
• Jacob Wobbrock , Meredith Ringel Morris, Andrew Wilson, “User-
Defined Gestures for Surface Computing,” CHI 2009.
• Mike Wu, Chia Shen, Kathy Ryall, Clifton Forlines, Ravin Balakrishnan,
“Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-
Touch Surfaces,” IEEE Tabletop 2006.
Contact
• karthur@synaptics.com, touchusability.com
Acknowledgments
• Usability colleagues at Dell, HP, Lenovo, Synaptics
24