Windows 8. Chromebook Pixel. Ubuntu Phone. These devices shatter another consensual hallucination that we web developers have bought into: mobile = touch and desktop = keyboard and mouse. We have tablets with keyboards; laptops that become tablets; laptops with touch screens; phones with physical keyboards; and even phones that become desktop computers. Not to mention new forms of input like cameras, voice control, and sensors. We've learned how to respond to screen size. Our next challenge is learning how to adapt to different forms of input.

1. Jason Grigsby • @grigs • cloudfour.com
Adaptive Input
http://www.flickr.com/photos/cobalt/7217055290/

2. Tweets: @grigs_talks
Slides: http://bit.ly/grigs-2013-07-22
http://www.flickr.com/photos/cobalt/7217055290/

3. http://www.flickr.com/photos/cdm/51747860/

4. http://www.flickr.com/photos/rheaney/4397863376
It started with TVs.

5. Designing for a 10-foot UI is very different.
http://www.flickr.com/photos/chrisbartow/5835428673

6. Larger text and fewer words.

7. Make up, down, left, right directions clear.
http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg

8. How do we know what is a TV?

9. This is HDTV.

10. This is HDTV.
1980 px
1080px

11. Resolution does not
define the optimal
experience.

12. Next came responsive web apps.
https://twitter.com/freediverx/status/354698695041744896

13. How do I make this responsive?
How do I make this responsive?

15. mobiledesktop
THE ART OF WEB DEVELOPMENTTHE ART OF WEB DEVELOPMENT
Web widgets
THE ART OF WEB DEVELOPMENTTHE ART OF WEB DEVELOPMENT
Mobile widgets

22. It’s not that we’re technically incapable, but adapting a phone UI to a
tablet UI is not so dissimilar from trying to automatically adapt desktop
UI to a phone. They are fundamentally different platforms with
different usability considerations, and something that makes sense on
phones may or may not belong on tablets.
—Todd Anglin, Kendo UI
http://www.kendoui.com/blogs/teamblog/posts/12-09-11/universal_mobile_apps_with_html5_and_kendo_ui.aspx

23. Sometimes it’s hard to envision a responsive version.
http://demos.kendoui.com/web/grid/editing.html

24. http://www.flickr.com/photos/jesuspresley/384080245/
We want people to be productive…

25. and stay in the zone.
http://www.flickr.com/photos/raccatography/8038855203

26. http://www.flickr.com/photos/shantellmartin/4543010568
Which seems very different from playing on an iPad.

27. For both the TV…

28. and the desktop web app…

29. input matters much more than screen size.

30. The grid is important to support d-pad interaction.
http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg

31. http://www.flickr.com/photos/royalsapien/2387707860

32. And keyboard and mouse are what we envision work is.
http://www.flickr.com/photos/royalsapien/2387707860

33. http://www.flickr.com/photos/hellogeri/6154034099/
Two years ago at BDConf, Jeremy talked about how…

34. http://www.flickr.com/photos/60415054@N00/14301113/
we told ourselves
that the web was…

35. http://www.flickr.com/photos/60415054@N00/14301113/

36. http://www.flickr.com/photos/60415054@N00/14301113/
640 px
480px

37. 640 px
480px

42. 1024 px
768px

43. http://www.flickr.com/photos/adactio/6153481666/

44. http://www.flickr.com/photos/adactio/6153481666/
Then mobile came and made us realize…

45. that it was a collective hallucination all along.
http://www.flickr.com/photos/garibaldi/303085857/

46. The web never had a fixed canvas.
http://www.flickr.com/photos/paulocarrillo/124755065/

47. Even our tools perpetuate the lie.

48. http://www.flickr.com/photos/69797234@N06/7203485148/
We’ve made tremendous progress.

49. But there is another collective hallucination.
http://www.flickr.com/photos/garibaldi/303085857/

51. =
=

53. Supports hover and pointer events.

56. Keyboard and touch.

59. Even the iPhone can have a keyboard.

62. Are these laptops or tablets?

65. Desktop computer with 23” touch screen

67. Luke nailed it.
http://static.lukew.com/unified_device_design.png

69. We can no longer make assumptions about input
based on screen size or form factor.
And we probably never should have.

70. http://www.flickr.com/photos/cblue98/7254221968

71. Input represents a bigger challenge than screen size.
http://www.flickr.com/photos/cblue98/7254221968

72. http://www.flickr.com/photos/taedc/9278192929

73. And it is changing more rapidly than ever before.
http://www.flickr.com/photos/taedc/9278192929

74. So let’s take a closer look…

75. Let’s start with futuristic input.
http://www.flickr.com/photos/jdhancock/3714748769/

76. http://uncyclopedia.wikia.com/wiki/File:Man_yelling_at_computer.JPG
VOICE

77. http://uncyclopedia.wikia.com/wiki/File:Man_yelling_at_computer.JPG
VOICE

78. http://www.98ps.com/viewnews-15222.html

79. Siri gets all of the hype…
http://www.98ps.com/viewnews-15222.html

80. but both Microsoft and Google have
compelling voice input in their products.

85. Explore the web on your TV with Internet Explorer for Xbox One. Use
your voice to browse your favorite sites with ease. Use Bing to find the
best of the web. Or use Xbox SmartGlass on your phone or tablet to
control your experience.
http://www.xbox.com/en-US/xboxone/what-it-does

86. Xbox 360 Voice Control
http://www.youtube.com/watch?v=xsfVJxakDa8

87. How would web pages change if we had voice control?

90. Google voice search

91. You can use speech recognition too.
http://www.google.com/intl/en/chrome/demos/speech.html
http://www.moreawesomeweb.com/demos/speech_translate.html

92. Web Speech API Specification
19 October 2012
Editors:
Glen Shires, Google Inc.
Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API
Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable
summary is available.
Abstract
This specification defines a JavaScript API to enable web developers to incorporate speech recognition and
synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to
use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web
pages to control activation and timing and to handle results and alternatives.
Status of This Document
This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on
the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)
other conditions apply. Learn more about W3C Community and Business Groups.

93. Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API
Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable
summary is available.
Abstract
This specification defines a JavaScript API to enable web developers to incorporate speech recognition and
synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to
use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web
pages to control activation and timing and to handle results and alternatives.
Status of This Document
This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on
the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)
other conditions apply. Learn more about W3C Community and Business Groups.
All feedback is welcome.
Table of Contents
1 Conformance requirements
2 Introduction
3 Use Cases
4 Security and privacy considerations
5 API Description
5.1 The SpeechRecognition Interface
5.1.1 SpeechRecognition Attributes
5.1.2 SpeechRecognition Methods
5.1.3 SpeechRecognition Events
5.1.4 SpeechRecognitionError
5.1.5 SpeechRecognitionAlternative
5.1.6 SpeechRecognitionResult

94. Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API
Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable
summary is available.
Abstract
This specification defines a JavaScript API to enable web developers to incorporate speech recognition and
synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to
use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web
pages to control activation and timing and to handle results and alternatives.
Status of This Document
This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on
the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)
other conditions apply. Learn more about W3C Community and Business Groups.
All feedback is welcome.
Table of Contents
1 Conformance requirements
2 Introduction
3 Use Cases
4 Security and privacy considerations
5 API Description
5.1 The SpeechRecognition Interface
5.1.1 SpeechRecognition Attributes
5.1.2 SpeechRecognition Methods
5.1.3 SpeechRecognition Events
5.1.4 SpeechRecognitionError
5.1.5 SpeechRecognitionAlternative
5.1.6 SpeechRecognitionResult
Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API
Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable
summary is available.
Abstract
This specification defines a JavaScript API to enable web developers to incorporate speech recognition and
synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to
use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web
pages to control activation and timing and to handle results and alternatives.
Status of This Document
This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on
the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)
other conditions apply. Learn more about W3C Community and Business Groups.
All feedback is welcome.
Table of Contents
1 Conformance requirements
2 Introduction
3 Use Cases
4 Security and privacy considerations
5 API Description
5.1 The SpeechRecognition Interface
5.1.1 SpeechRecognition Attributes
5.1.2 SpeechRecognition Methods
5.1.3 SpeechRecognition Events
5.1.4 SpeechRecognitionError
5.1.5 SpeechRecognitionAlternative
5.1.6 SpeechRecognitionResult

95. Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API
Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable
summary is available.
Abstract
This specification defines a JavaScript API to enable web developers to incorporate speech recognition and
synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to
use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web
pages to control activation and timing and to handle results and alternatives.
Status of This Document
This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on
the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)
other conditions apply. Learn more about W3C Community and Business Groups.
All feedback is welcome.
Table of Contents
1 Conformance requirements
2 Introduction
3 Use Cases
4 Security and privacy considerations
5 API Description
5.1 The SpeechRecognition Interface
5.1.1 SpeechRecognition Attributes
5.1.2 SpeechRecognition Methods
5.1.3 SpeechRecognition Events
5.1.4 SpeechRecognitionError
5.1.5 SpeechRecognitionAlternative
5.1.6 SpeechRecognitionResult
Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API
Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable
summary is available.
Abstract
This specification defines a JavaScript API to enable web developers to incorporate speech recognition and
synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to
use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web
pages to control activation and timing and to handle results and alternatives.
Status of This Document
This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on
the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)
other conditions apply. Learn more about W3C Community and Business Groups.
All feedback is welcome.
Table of Contents
1 Conformance requirements
2 Introduction
3 Use Cases
4 Security and privacy considerations
5 API Description
5.1 The SpeechRecognition Interface
5.1.1 SpeechRecognition Attributes
5.1.2 SpeechRecognition Methods
5.1.3 SpeechRecognition Events
5.1.4 SpeechRecognitionError
5.1.5 SpeechRecognitionAlternative
5.1.6 SpeechRecognitionResult

96. Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API
Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable
summary is available.
Abstract
This specification defines a JavaScript API to enable web developers to incorporate speech recognition and
synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to
use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web
pages to control activation and timing and to handle results and alternatives.
Status of This Document
This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on
the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)
other conditions apply. Learn more about W3C Community and Business Groups.
All feedback is welcome.
Table of Contents
1 Conformance requirements
2 Introduction
3 Use Cases
4 Security and privacy considerations
5 API Description
5.1 The SpeechRecognition Interface
5.1.1 SpeechRecognition Attributes
5.1.2 SpeechRecognition Methods
5.1.3 SpeechRecognition Events
5.1.4 SpeechRecognitionError
5.1.5 SpeechRecognitionAlternative
5.1.6 SpeechRecognitionResult
Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API
Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable
summary is available.
Abstract
This specification defines a JavaScript API to enable web developers to incorporate speech recognition and
synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to
use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web
pages to control activation and timing and to handle results and alternatives.
Status of This Document
This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on
the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)
other conditions apply. Learn more about W3C Community and Business Groups.
All feedback is welcome.
Table of Contents
1 Conformance requirements
2 Introduction
3 Use Cases
4 Security and privacy considerations
5 API Description
5.1 The SpeechRecognition Interface
5.1.1 SpeechRecognition Attributes
5.1.2 SpeechRecognition Methods
5.1.3 SpeechRecognition Events

97. Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API
Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable
summary is available.
Abstract
This specification defines a JavaScript API to enable web developers to incorporate speech recognition and
synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to
use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web
pages to control activation and timing and to handle results and alternatives.
Status of This Document
This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on
the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)
other conditions apply. Learn more about W3C Community and Business Groups.
All feedback is welcome.
Table of Contents
1 Conformance requirements
2 Introduction
3 Use Cases
4 Security and privacy considerations
5 API Description
5.1 The SpeechRecognition Interface
5.1.1 SpeechRecognition Attributes
5.1.2 SpeechRecognition Methods
5.1.3 SpeechRecognition Events
5.1.4 SpeechRecognitionError
5.1.5 SpeechRecognitionAlternative
5.1.6 SpeechRecognitionResult
Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API
Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable
summary is available.
Abstract
This specification defines a JavaScript API to enable web developers to incorporate speech recognition and
synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to
use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web
pages to control activation and timing and to handle results and alternatives.
Status of This Document
This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on
the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)
other conditions apply. Learn more about W3C Community and Business Groups.
All feedback is welcome.
Table of Contents
1 Conformance requirements
2 Introduction
3 Use Cases
4 Security and privacy considerations
5 API Description
5.1 The SpeechRecognition Interface
5.1.1 SpeechRecognition Attributes
5.1.2 SpeechRecognition Methods
5.1.3 SpeechRecognition Events

98. Special thanks to Eric Bidelman
http://moreawesomeweb.com
Speech Recognition API Support
?

100. Gestures?

102. Amazing, but too new to know
what, if anything, this technology
will mean for the web.

103. Let’s come back from the future and
look at something much Dumber.Dumber

104. Dumber

105. Dumber

106. -pad remote controlsD

107. -pad remote controlsD

108. -pad remote controlsD

109. http://www.flickr.com/photos/stewc/6669743035/

110. http://www.flickr.com/photos/stewc/6669743035/
TVs browsers that
support d-pad, send
arrow key events.

111. http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/

112. If then
http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/

114. is undetectable.
This is a recurring theme for input.

115. Sensors and camera
http://www.flickr.com/photos/retrocactus/2170677056

116. Sensors and camera
Camera
http://www.flickr.com/photos/retrocactus/2170677056

117. GPS
http://www.flickr.com/photos/3dking/149450434

118. GPS
GeoLocation
http://www.flickr.com/photos/3dking/149450434

120. Gyroscope & Accelerometer

121. http://www.flickr.com/photos/chrisjagers/4694134078

122. Back to today’s problems.
http://www.flickr.com/photos/chrisjagers/4694134078

124. Hover state No hover state

125. Hover state
Typing easier for many
No hover state
Typing often more difficult

126. Higher precision with mouse means
smaller targets possible
Hover state
Less precise than mouse and requires
larger touch targets
Typing easier for many
No hover state
Typing often more difficult

127. Higher precision with mouse means
smaller targets possible
Hover state
Less precise than mouse and requires
larger touch targets
Typing easier for many
No hover state
Typing often more difficult
Right clicking and “power” tools Single and multi-touch gestures

130. http://www.flickr.com/photos/28096801@N05/5012309802

131. I got this. Detect touch.
http://www.flickr.com/photos/28096801@N05/5012309802

133. Whatever you may think, it currently isn't
possible to reliably detect whether or not the
current device has a touchscreen, from within
the browser.
—Stu Cox, You Can’t Reliably Detect a Touch Screen
http://www.stucox.com/blog/you-cant-detect-a-touchscreen/

134. Chrome would like to turn on touch events by default.
https://code.google.com/p/chromium/issues/detail?id=159527
https://docs.google.com/a/cloudfour.com/presentation/d/1-n1qyzewpagREbzW2zm0wOalq33UhbtbSkWf9mEdly8/edit#slide=id.gc2d80e5b_171

136. Detect a mouse? Not reliably.

139. Surely we can detect a keyboard?

140. Surely we can detect a keyboard?
NOPE

142. Input is dynamic.

143. Input is dynamic.

144. Boris Smus’s experiments responding to input.

145. Boris Smus’s experiments responding to input.

146. http://www.flickr.com/photos/lyza/7382235106
Maybe we need to be more zen about input.

148. After poking at this problem for a few weeks,
my conclusion is: every desktop UI should be
designed for touch now. When any desktop
machine could have a touch interface, we
have to proceed as if they all do.
—Josh Clark
http://globalmoxie.com/blog/desktop-touch-design.shtml

150. http://ie.microsoft.com/testdrive/ieblog/2011/Sep/20_TouchInputforIE10andMetrostyleApps_1.png
http://www.w3.org/TR/pointerevents/
http://blog.webplatform.org/2013/02/pointing-toward-the-future/
New Pointer Events spec normalizes touch and mouse
Pointer Events builds on the DOM event
model to offer a new way to handle input
on the web, enabling developers to build
touch-first experiences that work with
mouse, pen, and other pointing devices as
well…They are also designed from the
ground up to allow modern browsers to
accelerate the touch-surface performance,
leading to a smoother user experience.

151. What about those who won’t let
go of their “power” interfaces?
http://www.flickr.com/photos/ecos/4092571213/

152. http://www.flickr.com/photos/scarygami/5689980135/
One option: give them a choice.

154. Gmail display density settings

155. Th Dream Experience - …
Uploaded 2 years ago
More Of The Dream Exp…
Uploaded 2 years ago
The Dream Experience -…
Uploaded 2 years ago
The Dream Experience …
Uploaded 2 years ago
The Love King Breaks It…
Uploaded 2 years ago
War Paint for Trees
From Lincoln Motor Company
Join Log In Create Watch Upload Search s
[ ]
–VIDEOS
Vimeo Couch Mode

156. Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos
Th Dream Experience - …
Uploaded 2 years ago
More Of The Dream Exp…
Uploaded 2 years ago
The Dream Experience -…
Uploaded 2 years ago
The Dream Experience …
Uploaded 2 years ago
The Love King Breaks It…
Uploaded 2 years ago
PROMOTED
War Paint for Trees
From Lincoln Motor Company
Join Log In Create Watch Upload Search s
[ ]
–VIDEOS
Vimeo Couch Mode

157. Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos
Th Dream Experience - …
Uploaded 2 years ago
More Of The Dream Exp…
Uploaded 2 years ago
The Dream Experience -…
Uploaded 2 years ago
The Dream Experience …
Uploaded 2 years ago
The Love King Breaks It…
Uploaded 2 years ago
PROMOTED
War Paint for Trees
From Lincoln Motor Company
Join Log In Create Watch Upload Search s
[ ]
–VIDEOS
Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos
Th Dream Experience - …
Uploaded 2 years ago
More Of The Dream Exp…
Uploaded 2 years ago
The Dream Experience -…
Uploaded 2 years ago
The Dream Experience …
Uploaded 2 years ago
The Love King Breaks It…
Uploaded 2 years ago
PROMOTED
War Paint for Trees
From Lincoln Motor Company
Join Log In Create Watch Upload Search s
[ ]
–VIDEOS
Vimeo Couch Mode

158. Vimeo Couch Mode

160. The key benefit of this approach:
You’re designing for user need not
for a specific form factor or input.

161. http://www.flickr.com/photos/raver_mikey/504815463

163. Progressive Input?

164. Graph from Chapter 1 of Adaptive Web Design by Aaron Gustafson
http://easy-readers.net/books/adaptive-web-design/

165. Graph from Chapter 1 of Adaptive Web Design by Aaron Gustafson
http://easy-readers.net/books/adaptive-web-design/
Progressive enhancement contains a value judgment

166. Who are we to judge what input is better?
http://www.flickr.com/photos/fensterbme/4783366926

167. We need to learn
to adapt.
http://www.flickr.com/photos/cdm/147947664/

168. So that’s the puzzle that has
been haunting me.
http://www.flickr.com/photos/sravishankar/3460495

169. http://www.flickr.com/photos/wwworks/1384952210

170. Thank You!Special thanks to Luke Wroblewski, Eric Bidelman and Flickr users for
generously sharing their photos under creative commons license.