This presentation covers the basics of how users of screen readers interact with widgets. The goals of this presentation are: 1) Make you comfortable enough with using screen readers so that you have a basic understanding of the types of information users need/expect to hear for widgets. 2) Make you familiar with the common keyboard patterns used for existing widgets on the web and the desktop. Use the information in this presentation as a foundation for implementing expected keyboard shortcuts for custom widgets as well as for learning how test custom widgets you develop for the web using JavaScript.

1. The Audio
User Experience
for Widgets
Todd Kloots
@todd

2. Goals
1. Understand what users of screen readers
expect or need to hear when interacting
with widgets
2. Understand the common keyboard and
touch interaction patterns for widgets

3. Screen Readers
Read the code (HTML)
Read the focused element
What is read:
Element/Control type
Label
Properties
State changes
For web-based interfaces most of the information that screen readers announce originates from the code—the HTML.

4. Navigation
Some fundamentals. First, how do users of screen readers navigate through a page or application?

5. Navigating web pages or applications using a screen reader is very much the opposite of the experience for sighted users.
As a sighted person, when you first see a web page or application you see the gestalt.

6. Then you can focus your eyes on the task you want to accomplish, often ignoring the other parts of the page or application.
For example, if you want to search, you focus your eyes on the search box and ignore the rest of the page.

7. Or if you want to read a section of a page, you move your focus to that section.

8. On the other hand, users of screen readers begin with a blank canvas and discover the page or application one element or section at a time. It is not until they traverse through a section that
they can begin to get a sense of the whole.

9. Imagine navigating through yahoo.com using a screen reader. You’d first encounter the Yahoo! logo.

10. Then the first of the search tabs at the top of the page: “Web”

11. Then “Images”

12. Then “Video”

13. Then “Travel”

14. Then “News”

15. Then the search box.

16. And finally the “Search” button.

17. And after navigating through those elements you might begin to perceive them as belonging to a section.

18. Widgets
That's navigation. Now, how do users of screen readers interact with wigets?

19. Demo
http://youtu.be/ZnlvBIfTQfI
Demo of interacting with a checkbox using VoiceOver—the screen reader for the Mac. Notice how VoiceOver announces the control type (“checkbox”), the label “Send me offers”, and the
current state (“checked” and “not checked”).
All this text is announced to ensure that users of screen readers can perceive and operate the widget.

20. See Hear
That's the experience for existing widgets. For custom controls, most often problems are caused when there is a gap between the visual and audio user experience such that users of screen
readers do not have access to the same information as sighted users.

21. http://youtu.be/lpU8IHiU8rw
This short video illustrates the screen reader user experience of a custom toolbar that doesn't use ARIA and whose buttons aren't properly labeled. Notice how the user is unaware of the fact
that they are interacting with a toolbar, and how each button is indistinguishable for one another.
This demonstrates a common problem accessibility problem for custom controls built using JavaScript: a perception gap between the visual and audio user experience.

22. Visual rendering of the screen reader user experience when contextual information and labels are not available.

23. http://youtu.be/3AdvVg2Cs9Q
This short video illustrates the user experience of a custom toolbar that uses ARIA to improve the user experience for users of screen readers. Notice how there is no gap in what sighted
users perceive and what the screen reader announces; the screen reader announces to the user they are in a toolbar and the label of each button.
This is made possible by ARIA—it provides developers with the necessary tools to close gaps between the visual and audio user experience.

24. http://youtu.be/WK6qN4OCwVU
This short video illustrates the screen reader user experience of a custom modal dialog that doesn't use ARIA. Notice how when the dialog is made visible the only information that is
announced is the label of the "OK" button. Other contextual information such as the dialog's title, and message are not announced.
This demonstrates a common accessibility problem for custom controls built using JavaScript: a perception gap between the visual and audio user experience.

25. http://youtu.be/XqEwuV2HySs
This short video illustrates the user experience of a custom modal dialog that uses ARIA to improve the user experience for users of screen readers. Notice how when the dialog is made
visible the same information visible to sighted users is also announced by the screen reader. This is made possible by ARIA—it provides developers with the necessary tools to close gaps
between the visual and audio user experience.

26. Screen readers are available on a variety of devices including: desktop, phones and tablets.

27. As for mobile: Screen readers are available for both iOS and Android.

28. Configuring the Mac
Every Mac has a built-in screen reader called VoiceOver.

29. Start/Stop VoiceOver
Cmd + F5
(or Fn + Cmd + F5)

30. Speech Rate
Press Ctrl + Option + Command + → until
you see/hear "Rate"
Press Ctrl + Option + Command + ↓ to To change the speech rate (how fast VoiceOver speaks)
• Press Ctrl + Option + Command + → until you see/hear "Rate"
make VoiceOver speak slower • Press Ctrl + Option + Command + ↓ to make VoiceOver speak slower
• Press Ctrl + Option + Command + ↑ to make VoiceOver speak faster
Press Ctrl + Option + Command + ↑ to
make VoiceOver speak faster
If you find VoiceOver is speaking too fast, you can adjust the rate.

31. VoiceOver Shortcuts
Task Shortcut
Toggle VoiceOver on/off Command + F5 (or Fn + Command + F5)
Stop speaking Ctrl
Read the focused item Ctrl + Option + A
Move to next/previous control Tab / Shift + Tab
Activate the focused item Spacebar OR Ctrl + Opt + Spacebar
Toggle keyboard help Ctrl + Option + K
Toggle the display off/on Ctrl + Option + Shift + F11
Some of the very basic shortcuts you’ll need to know to begin using VoiceOver and to test custom widgets.

32. nvda-project.org
The NVDA screen reader is a free screen readers for Windows.

33. Install NVDA
1. Download and install NVDA
2. Press Ctrl + Alt + N to start NVDA
3. After NVDA starts uncheck the
checkbox labeled "Show this dialog when
NVDA starts"

34. Disable Mouse Tracking
1. Right click on the NVDA icon in the
system tray
2. Go to Preferences > Mouse Settings
3. Uncheck the checkbox labeled "Mouse
Tracking"
Sighted users might want to disable mouse tracking in the interest of reducing "noise"

35. Save Preferences
1. Right click on the NVDA icon in the
system tray
2. Choose the "Save Configuration"
menu item

36. Configuring Speech Rate
Ctrl + Insert + ↓ Speak slower
Ctrl + Insert + ↑ Speak faster
If you find the screen reader is speaking too fast, you can adjust the rate.

37. NVDA Speech Viewer
Click on the
NVDA icon in
the system try
Go to Tools >
Speech Viewer
If you don't want to listen to the screen reader, the spoken text is available via NVDA's speech viewer.

38. NVDA Keyboard Shortcuts
Task Shortcut
Stop NVDA from speaking Ctrl
Toggle Browse Mode on/off Insert + Spacebar
Read the focused item Insert + Tab
Activate the focused item Enter or Spacebar
Move between objects in the tab order Tab or Shift + Tab
Toggle NVDA on Ctrl + Alt + N
Toggle NVDA off Insert + Q
Some of the very basic shortcuts you’ll need to know to begin using NVDA and to test custom widgets.

39. Configuring Your iOS Device

40. Enable VoiceOver on iOS
Settings > General > Accessibility > Triple-
click Home
Choose VoiceOver
As a developer, I recommend setting up VoiceOver on iOS such that triple-clicking the Home button toggles VoiceOver on and off.

41. Widget Touch Patterns
Task Gesture
Speak item or navigate to item Tap
Select the next or previous item Flick right or left
Activate (click) the selected item Double-tap
Change selection or value Flick up or down
Dismiss an alert Two-finger “scrub”
Some of the very basic shortcuts you’ll need to know to begin using VoiceOver and to test custom widgets for mobile.

42. Exercises
Some exercises to get you familiar with using existing web form controls and classic desktop controls using a screen reader.

43. Exercise
bit.ly/z86b0J
Use this simple page to begin to get comfortable using either VoiceOver and Safari on the Mac, or NVDA and Firefox on Windows.
Observable keyboard patterns for existing HTML form controls:
Tab and Shift+Tab navigate between widgets/controls
Arrow keys change selection in a group
Spacebar executes a click action
Enter executes the default action
Esc key closes menus and dialogs
Observable widget information announced by the screen reader:
Widget type
Widget label
Widget properties
Widget state

44. Exercise
Use the menubar with VoiceOver enabled. (Press Control + F2 to focus the menubar.)
Notice some of the menubar keyboard patterns:
Arrows navigate the menubar and submenus
Esc key closes the menu and blurs the menubar. When the menubar blurs, focus is restored to the control that previously had focus.
Notice how VoiceOver announces things like:
The label of the current menu or menuitem
The size of the menu when you first open it
Submenu items are announced as "submenu"
The state of menuitems—for example "dimmed" to indicate disabled menuitems
When a menu closes

45. Exercise
Use the tree control in the Finder with VoiceOver enabled.
Notice some of the tree control keyboard patterns:
Up and down arrow keys navigate between items
Left and right arrow key expand or collapse items
Notice how VoiceOver announces when a tree item expands or collapses.

46. Exercise
Use one of the tab controls on the Mac with VoiceOver enabled.
Notice how VoiceOver announces things like:
The selected state of a tab
The total number of tabs
Notice the tab control keyboard patterns:
Left and right arrows move focus between tabs
Spacebar changes the selected tab

47. Exercise
Use the menubar in IE on Windows with NVDA enabled. (Press Alt + F to move focus to the menubar.)
Notice some of the menubar keyboard patterns:
Arrows navigate the menubar and submenus
Esc key closes the menu and blurs the menubar. When the menubar blurs, focus is restored to the control that previously had focus.
Notice how NVDA announces things like:
The label of the current menu or menuitem
The size of the menu when you first open it
Submenu items are announced as "submenu"
The state of menuitems—for example "unavailable" to indicate disabled menuitems

48. Exercise
Tab controls on Windows work similar to tab controls on the Mac.
Move focus to the control by pressing Tab or Shift + Tab.
Change tabs via left/right arrow keys.

49. Widget Keyboard Patterns
Task Key
Focus Next or Previous Widget Tab / Shift + Tab
Change selection →, ←, ↑, ↓
Expand / Collapse →, ←, ↑, ↓
Close or Exit Esc
Click Spacebar
Default Action Enter
Although there are differences across browsers and platforms, there are many similarities with respect to keyboard patterns for widgets.

50. Mobile Exercise
bit.ly/z86b0J
Use this simple test page to begin learning how to interact with HTML form controls when VoiceOver is enabled.
Some observable patterns:
Swipe left/right to move between widgets
Once a widget has focus, double tab anywhere on the screen to execute a “click” or interact with the control
For controls that support selection, swiping up or down changes the value

51. Widget Touch Patterns
Task Gesture
Speak item or navigate to item Tap
Select the next or previous item Flick right or left
Activate (click) the selected item Double-tap
Change selection or value Flick up or down
Dismiss an alert Two-finger “scrub”

52. Review

53. Screen Readers
Read the code (HTML)
Read the focused element
What is read:
Element/Control type
Label
Properties
State changes

54. Widget Keyboard Patterns
Task Key
Focus Next or Previous Widget Tab / Shift + Tab
Change selection →, ←, ↑, ↓
Expand / Collapse →, ←, ↑, ↓
Close or Exit Esc
Click Spacebar
Default Action Enter
Although there are differences across browsers and platforms, there are many similarities and common patterns for widgets, and these patterns span the web and the desktop on both
platforms—Mac and Windows.

55. Widget Touch Patterns
Task Gesture
Speak item or navigate to item Tap
Select the next or previous item Flick right or left
Activate (click) the selected item Double-tap
Change selection or value Flick up or down
Dismiss an alert Two-finger “scrub”