2. Input evolution
• As computers become smaller and ubiquitous
(everywhere and in everything), they can not grab
all of our attention (sitting in front of a screen)
and their size doesn’t allow us to use are fingers
to communicate with them
• Alternative input methods (speech, movement,
gazing) are considered as ways to make
computing less attention demanding and more
intuitive (Accessibility benefits)
Amir Dotan - www.amirdotan.com
3. Input evolution
• Punch cards
• Keyboard
• Mouse
• Face recognition
• Facial expression
• Gaze / eye movement
• Touch / Multi-Touch
• Speech recognition
• Body movement (gestures)
Amir Dotan - www.amirdotan.com
6. Text Entry Input - Keyboards
and Keypads
• The primary mode of textual data entry
• Keyboard size and packaging influences users
satisfaction and usability
• Large keyboards with many keys give an
impression of professionalism and complexity
but may deter novice users
• Small keyboards seem lacking in power to some
users, but their compact size is an attraction for
mobile devices
Amir Dotan - www.amirdotan.com
7. The QWERTY layout
• The most commonly used keyboard layout is the
QWERTY layout
• Also known as the Sholes keyboard, it was
invented by Christopher Sholes who invented the
typewriter in 1868
• His layout slowed down users enough that key
jamming was infrequent
Amir Dotan - www.amirdotan.com
8. Dvorak Layout (Schneiderman, 2005)
• Developed in the 1920’s, supposedly reduces
finger travel, thereby increasing the typing rate of
expert typists from about 150 words per minute
to more than 200 words per minute
• Acceptance of the Dvorak layout has been very
limited, despite the work of devotees
• People who have tried it report that it takes about
a week of regular typing to make the switch, but
most users have been unwilling to invest much
effort
Amir Dotan - www.amirdotan.com
12. Alternative Adjustable
Keyboard
Optimus Maximus keyboard - http://www.artlebedev.com/everything/optimus/
Amir Dotan - www.amirdotan.com
13. Keyboards and Keypads for
Mobile Devices
http://amirdotan.multiply.com/video/item/98/The_Evolution_of_Mobile_Phones_1985_-_Today
Amir Dotan - www.amirdotan.com
14. Keyboards and Keypads for
Mobile Devices
• Mobile-phone functionalities are increasing
considerably while their size is still shrinking
• Most devices combine normal static keys with
dynamically labelled softkeys whose functions
are dependent on status and context
Amir Dotan - www.amirdotan.com
15. Alternative Mobile Phone
Keypads
http://www.digitwireless.com Nokia 7600
Amir Dotan - www.amirdotan.com
16. Pointing Devices
• A device such as a mouse, trackball, or graphics
tablet that allows users to move a pointer about
on the workspace and point to graphical objects
• A direct-manipulation approach - No need to
learn commands or type, attention is focused on
the display
• Important for small devices and large wall
displays that make keyboards impractical as
input devices
Amir Dotan - www.amirdotan.com
17. Pointing Devices
• Direct control devices
– Lightpen
– Touchscreen (Single input point)
– Multitouch (Multiple and simultaneous input points)
– Stylus
• Indirect control devices
– Mouse
– Trackball
– Joystick
– Touchpad
– Trackpoint (for laptops)
– Graphics tablet
• Novel devices and strategies
– Foot controls
– Eye tracking
– DataGloves
– Tangible user interfaces
– Digital paper
– Haptic feedback
Amir Dotan - www.amirdotan.com
18. Criteria for Success (Schneiderman, 2005)
• Speed and accuracy
• Learning time
• Cost and reliability
• Size and weight
Amir Dotan - www.amirdotan.com
20. Direct-Control Pointing Device
- The Lightpen
• Had several disadvantages:
– Users’ arm got tired
– Users’ hand obscured part of the screen
– Users had to remove their hands from the
keyboard to pick up the lightpen
– The lightpen was too fragile for public-access
environments
Amir Dotan - www.amirdotan.com
23. An Alternative to the Stylus -
Shift (Microsoft Research)
http://research.microsoft.com/users/baudisch/projects/shift/index.html
http://uk.youtube.com/watch?v=kkoFlDArYks
Amir Dotan - www.amirdotan.com
24. Indirect-Control Pointing
Devices (Schneiderman, 2005)
• Eliminate the hand-fatigue and hand-obscuring-
the-screen problem
• They do however require the hand to locate the
device and demand more cognitive processing
and hand/eye coordination to bring the onscreen
cursor to the desired target
• Among the indirect pointing devices the mouse
has been the most popular
Amir Dotan - www.amirdotan.com
25. Pointing Device - The Mouse
On December 9, 1968 in San Francisco, Douglas Engelbart
presented for the first time the computer mouse
http://uk.youtube.com/watch?v=1MPJZ6M52dI
Amir Dotan - www.amirdotan.com
30. Touch Input - Considerations
(Windows Vista User Experience "Touch" guidelines)
• Small controls are difficult to use
– The size of the controls greatly affects your ability
to interact effectively. Controls that are at least
23x23 pixels are usable with a finger, but larger
controls of at least 40x40 pixels are even more
comfortable to use
Amir Dotan - www.amirdotan.com
31. Touch Input - Considerations
(Windows Vista User Experience "Touch" guidelines)
• Task locality helps
– While you can move the pointer across a 14-inch
screen with a 3-inch mouse movement, using
touch requires you to move your hand the full 14
inches
– Repeatedly moving between targets that are far
apart can be tedious, so it's much better to keep
task interactions within the range of a resting hand
whenever possible
– Context menus are convenient because they
require no hand movement
Amir Dotan - www.amirdotan.com
32. Touch Input - Considerations
(Windows Vista User Experience "Touch" guidelines)
• Hover must not be required
– Most touchscreen technologies don't detect a
hovering finger, even if they can detect a hovering
pen. If a program has tasks that depend on hover,
you won't be able to perform them efficiently using
touch
Amir Dotan - www.amirdotan.com
33. Touch Input - Considerations
(Windows Vista User Experience "Touch" guidelines)
• Text input and selection are difficult
– Lengthy text input is especially difficult using
touch, so auto-completion and acceptable default
text values can really simplify tasks. Text selection
can also be quite difficult
Amir Dotan - www.amirdotan.com
34. Touch Input - Considerations
(Windows Vista User Experience "Touch" guidelines)
• Small targets near the edge of the display can
be very difficult to touch
– Some touchscreen technologies are less sensitive
at the edges, making controls near the edge
harder to use. For example, the Minimize,
Maximize/Restore, and Close buttons on the title
bar can be harder to use when a window is
maximized
Amir Dotan - www.amirdotan.com
35. Basic Touch Design Principles
(Windows Vista User Experience "Touch" guidelines)
• Don't assume that if a UI works well for a mouse, it also works
well for touch
– While good mouse support is a start, a good touch experience has a
few additional requirements
• You can assume that if a UI works well for a finger, it also
works well for a pen
– Making your program touchable goes a long way to providing good pen
support. The primary difference is that fingers have a blunter tip, so
they need larger targets. And again, hover must be optional
• Don't depend on touch pointer to fix touch UI problems
– Because the touch pointer isn't as easy to use as direct input, view the
touch pointer as a last resort for programs that haven't been designed
for touch
Amir Dotan - www.amirdotan.com
36. Comparison of Pointing
Devices
• Each input device has its strengths and
weaknesses
• The keyboard is best for text input and giving
commands with minimal hand movement
• The mouse is best for efficient, precise pointing
• Touch is best for object manipulation and giving
simple commands
• A pen is best for freeform expression, as with
handwriting and drawing
Amir Dotan - www.amirdotan.com
37. Comparison of Pointing
Devices (Schneiderman, 2005)
• Early studies found that direct pointing devices
such as a lightpen or touchscreen were often the
fastest but the least accurate
• Studies have shown that the mouse is the more
accurate and efficient than alternative devices
• Users’ tasks matter when comparing devices
• Joysticks and trackball are often preferred over
mice by users with motor disabilities
Amir Dotan - www.amirdotan.com
38. Comparison of Pointing
Devices (Schneiderman, 2005)
• Pointing devices, especially indirect-control
devices such as the mouse, are particularly
challenging for users who have vision
impairments - Well designed cursors and
adjustable size and shape can help
• Alternative keyboard or keypad navigation
options should be provided whenever possible
Amir Dotan - www.amirdotan.com
39. Novel Devices (Schneiderman, 2005)
• The popularity of pointing devices and the quest
for new ways to engage diverse users for diverse
tasks has led to provocative innovations
• Since users’ hands might be busy on the
keyboard, designers have explored other
methods for selection and pointing
• In some cases, the device is too small or too big
to be used with a keyboard or a conventional
pointing device
Amir Dotan - www.amirdotan.com
42. Gestural Interface - Example
G-Speak http://vimeo.com/2229299
Amir Dotan - www.amirdotan.com
43. Gestural Interface - Example
http://amirdotan.multiply.com/video/item/148/HITACHI_Gesture_operation_TV
Amir Dotan - www.amirdotan.com
44. Gestural Interface - Example
http://amirdotan.multiply.com/video/item/83/Second_Life_gestural_interface_-_Navigation_Demo
Amir Dotan - www.amirdotan.com
45. Gestural Interface - Example
http://amirdotan.multiply.com/video/item/80/ShadowReaching_New_Perspective_on_Wall_Display_Interaction
Amir Dotan - www.amirdotan.com
46. Imaging and Video Input
Device - Webcam Tracking
http://uk.youtube.com/watch?v=GDINqY9kekU
Amir Dotan - www.amirdotan.com
47. Gestural Interface - Example
http://uk.youtube.com/watch?v=7uixUmxH-Z0
Amir Dotan - www.amirdotan.com
48. Gestural Interfaces - Wii and
Wii Fit
http://uk.youtube.com/watch?v=mCufArSg-SQ http://uk.youtube.com/watch?v=5oNVIcMnZh4
Amir Dotan - www.amirdotan.com
50. Input - Using physical objects
http://amirdotan.multiply.com/video/item/101/reactable_basic_demo_
Amir Dotan - www.amirdotan.com
51. Eye tracking systems
• Our eyes are an essential output device with is
used to point to objects. We use it to
communicate to the world our intents (Eye
contact)
• Combined with head gesture, this method of
pointing can be a powerful communication
channel
Amir Dotan - www.amirdotan.com
52. Eye tracking - Example
• Squidoo EyeTracking
– http://www.youtube.com/watch?v=ilq9qeyVjT0
Amir Dotan - www.amirdotan.com
53. The future?
Gamers will soon be able to interact
with the virtual world using their
thoughts and emotions alone.
BBC NEWS 20.02.08
http://news.bbc.co.uk/1/hi/technolo
gy/7254078.stm
Amir Dotan - www.amirdotan.com
55. Speech and Auditory Input
(Schneiderman, 2005)
• Hardware designers have made dramatic
progress with speech recognition, generation,
and processing but current success are far from
achieving the results we see in Sci-Fi movies
• They also recognize that voice commanding is
more demanding of user’s working memory than
is hand/eye coordination and thus may be more
disruptive to users while they are carrying out
tasks
Amir Dotan - www.amirdotan.com
56. Speech and Auditory Input
(Schneiderman, 2005)
• Planning and problem solving can proceed in
parallel with hand/eye coordination, but they are
more difficult to accomplish while speaking
• Background noise and variations in users speech
performance make the challenge of speech
recognition still greater
• The benefits to people with certain disabilities
can be immense, but general users are not
rushing to implement speech input
Amir Dotan - www.amirdotan.com
57. Speech Systems -
Opportunities (Schneiderman, 2005)
• When users have vision impairments
• When the speaker’s hands are busy
• When mobility is required
• When the speaker’s eyes are occupied
• When harsh or cramped conditions preclude use
of a keyboard
Amir Dotan - www.amirdotan.com
58. Speech Systems -
Technologies (Schneiderman, 2005)
• Discrete-word recognition -
– Recognize individual words spoken by a specific
person
– 90%-98% reliable
– 100-10000-word or larger vocabularies
• Continuous-speech recognition
Amir Dotan - www.amirdotan.com
59. Speech recognition -
Examples
Flight Simulator Voice Recognition Speech recognition in Windows Vista
http://www.youtube.com/watch?v=wZf9Ni6Qocc http://www.youtube.com/watch?v=v9o72XJcFQI
Amir Dotan - www.amirdotan.com
60. Speech recognition -
Examples
Google Mobile App for iPhone, now with Voice Search
http://uk.youtube.com/watch?v=y3z7Tw1K17A
Amir Dotan - www.amirdotan.com