Comparing different techniques for typing on touchscreens like BrailleTouch and the Google Talkback

1. Objective comparisons between
Exploration and Gesture based
keyboards for the blind
Adit Gupta,
IIT Gandhinagar
Dr. Nikhil Balram,
President and CEO,
Ricoh Innovations Corporation
11/26/2013

2. Introduction
Motivation
About 8.075 million of the total
35 million
blind population is in India (about 21%)
But only 5% of them use or receive support from
assistive technology of any form! – WHO report
11/26/2013
Pascolini D, Mariotti SPM. Global estimates of visual impairment: 2010. British Journal Ophthalmology Online First published December 1, 2011 as 10.1136/bjophthalmol-2011-

3. Technology
Current Assistive Technology for
text typing
Range from Braille Sense Plus (6000 USD) with a
Braille Display to GalaTee (400 USD) in the USA
Braille Sense Plus
11/26/2013
Images from Nanopac.com and amazon.com respectively
GalaTee

4. What about making information
available to everyone on the go?
11/26/2013

5. Research
Goal
But how does a blind individual
type and reach information without
tactile feedback?
This question became the backbone of our
research
11/26/2013

6. Braille
Before going further,
A bit of background in Braille
3 by 2 binary matrix, encodes upto 63 characters
11/26/2013Brian Frey, Caleb Southern, and Gregory D. Abowd. 2011. BrailleTouch: designing a mobile eyes-free soft keyboard. In Proceedings of the 13th International
Mario Romero,
Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 707-709.

7. The current cutting edge research
on Eyes-Free text typing.
- Highly researched area in the West.
- We focused on the two which are in vogue
keeping Indian Blind participants in mind
11/26/2013

8. BrailleTouch
BrailleTouch
Georgia Tech Research
Drawbacks
Numeric keypad is not simple to use
Both hands are required
Highly Memory dependent (Braille)
11/26/2013Brian Frey, Caleb Southern, and Gregory D. Abowd. 2011. BrailleTouch: designing a mobile eyes-free soft keyboard. In
Mario Romero,
Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New
York, NY, USA, 707-709.

9. Talkbac
k
Google’s Android TalkBack
Google Inc.
Enables Explore-by-Touch
Drawbacks
Numeric keypad is not simple to use
11/26/2013

10. So we put them Head-to-Head
and compared their results
objectively
BrailleTouch vs Android Talkback Keyboard
11/26/2013

11. BPA
BPA, Ahmedabad
Established in 1947, has 15 campuses across
Gujarat, with 300 odd employees and many more
students
Taught Braille in 3 languages- English, Hindi &
Gujarati
11/26/2013 bpa.org
Photograph from

12. Demographics
We recruited 4 completely Blind
participants for our study…
Each participants used mobile phones with a TTS on a
daily basis.
Nokia C5 series was a phone commonly used by them.
Quite often keep in touch with me via email!
Participant
Age
Braille education Smartphone
education
P1
52
K-12
0
P2
27
K-10
0
P3
29
K-12
0
P4
30
K-12
4 months
11/26/2013

13. Usability testing strategy
11/26/2013
Research
strategy

14. App
The Experiment App
Simple Key logger; No need for a very aesthetically
appealing design
BrailleTouch
Android Talkback
11/26/2013

15. Aggregated results – Keystroke
Timings
BrailleTouch
14000
12000
Time required (milliseconds)
10000
8000
6000
4000
2000
0
11/26/2013
results
Android Stock Keyboard

16. Can we now conclude that
BrailleTouch is a better technique?
Let’s take a closer look at the typing errors by our
subjects
11/26/2013

17. Aggregated results – Keystroke
Timings
BrailleTouch
3.5
3
2.5
Number of errors total
2
1.5
1
0.5
0
11/26/2013
Results
Android Stock Keyboard

18. Data manipulation
Keystroke time inclusive of errors
Typed
character
Target
character
Match
Timestamp
t
j
backspace
h
e
space
t
h
h
h
e
space
1
0
0
1
1
1
16:11:32
16:11:35
16:11:38
16:11:41
16:11:45
16:11:48
Timestamp
Textbox Input
(Milliseconds)
1925859247
1925862314
1925865627
1925869157
1925872317
1925875522
t
tj
t
th
the
the
Hence,
Actual time required to type ‘h’
= 3067 + 3313 + 3530
= 9910 millisecs! (inclusive of errors)
Whereas it’s 3530 millisecs in the previous method
11/26/2013
Keystroke Time
(Milliseconds)
2539
3067
3313
3530
3160
3205

19. Insights
Aggregated results –
Error Inclusive Keystroke Timings
11/26/2013

20. Learnings
Take home #1
Is BrailleTouch or Android
Talkback keyboard ideal?
Maybe a hybrid approach between exploration and
gesture might be ideal since both have huge
drawbacks.
A longer term study on the participants would make
it clear
11/26/2013

21. Learnings
Take home #2
Indian Blind users prefer the British
accented voice feedback more than
the American accent!
A simple reason for this is that the colloquial terms
are crisp and clear in the British accent
11/26/2013

22. Learnings
Take home #3
Have lesser motor control in their
ring finger to type
Clear from the errors increasing with the dots in the
Braille cell
11/26/2013

23. Learnings
Take home #4
Editing previously typed words
functionality is essential
Through my direct observations and the user’s
feedback
11/26/2013

24. Learnings
Future: A Hybrid approach?
A longer term study would be required for this
research.
11/26/2013

25. Thanks!
11/26/2013