WebRTC gives us a way to do real-time, peer-to-peer communication on the web. In this talk, we'll go over the current state of WebRTC (both the awesome parts and the parts which need to be improved) as well as what could come in the future. Mostly though, we'll take a look at how to combine WebRTC with other web technologies to create great experiences on the front-end for real-time, p2p web apps.
10. but …
So far I’ve seen very
few web people
talking about it
11. Whats the biggest barrier to WebRTC adoption?*
Lack of Awareness
Not supported by MS
Not supported by Apple
Other
0% 5.5% 11% 16.5% 22%
* WebRTC World Outlook Survey 2014
23. What can you achieve with gUM?
- Motion Detection (Baby Monitors, Security Cam)
- Hotspots
- Gesture Recognition
- Emotion Recognition
- Sound Detection and Visualisation
- Voice / Sound Activated Controls
- Capture video (Kind of) (whammy.js, gifshot)
24. gUM: Current way to do it
navigator.getUserMedia*
*with prefixes
25. gUM: Broken Error Handling
Exceptions are not caught by the current API.
Promises can fix this.
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html
26. gUM: Current Way
var failure = function(reason){
log("Failed to show camera: " + reason.message);
}
!
navigator.getUserMedia(constraints, function(stream) {
try {
$video.src = window.URL.createObjectURL(stream);
$video.play();
} catch(e) { failure(e); } //failure must tolerate non-MediaStreamError
}, failure);
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html
27. gUM: In the future ?
navigator.mediaDevices.getUserMedia
28. gUM: In the future ? Promises Proposal
navigator.mediaDevices.getUserMedia(constraints)
.then( function(stream) {
videoElement.src = window.URL.createObjectURL(stream);
videoElement.play();
}).catch(function (reason){
log(“Failed to show camera: ” +reason.message);
});
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html
29. Debate
This is still up for debate.
Some are eager to just ship v1.0 and add support for
promises in gUM at a later stage.
!
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Oct/0003.html
30. gUM: Other concerns
Impossible for a gUM app to
use the device flash for low-light
!
Use (auto) zoom
Use (auto) focus
31. gUM: Other concerns
“If you look further into this API then it
doesn’t offer any advanced features
that you expect in a good camera app:
focus, flash, zoom `
etc. It is clear that
…(getUserMedia).. has been designed
for use in P2P applications and not as a
dedicated camera experience.”
- Paul Kinlan
32. Mediastream Image-Capture
!
Will provide a deeper level of control
for taking photos
!
w3.org/TR/image-capture/
33. Mediastream Image-Capture
White Balance
ISO
Red-Eye Reduction
Exposure
Brightness
Contrast
`
Saturation
Sharpness
Zoom
takePhoto()
53. TURN
Basically relays stuff from one
party to the other.
In this case, connection won’t
be p2p, but at least the
connection will be made!
54. TURN
Basically relays stuff from one
party to the other.
In this case, connection won’t
be p2p, but at least the
connection will be made!
Generally speaking, a TURN server will
include a STUN server too.
55. Free ICE Project
github.com/DamonOehlman/freeice
(List of free STUN/TURN servers to use in your projects)
58. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Creates Offer
local description
59. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Offer
local description
60. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Offer
remote description
local description
61. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Creates and
sends Answer
local description
local description
remote description
62. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Answer
local description
local description
remote description
63. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Answer
local description
local description
remote description
remote description
64. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Connectivity
Checks
local description
local description
remote description
remote description
65. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
P2P Connection Established
72. Data Channels
Once RTCPeerConnection is
established, we can use Data Channels
to transfer data P2P.
Data can be text or binary data
73. Whats the advantage?
Data Channels provide a
* High Performance
* P2P
* Low Latency
Connection to others
74. Working with Data
datachannel
.onopen // Data Channel opened
.onclose // Data Channel closed
.onerror // Error occurred
.onmessage // New message arrived
.send() // Send a message
77. notable options
ordered
send data packets in ordered fashion or not
!
maxRetransmitTime
Max time to try to send a failed message
!
maxRetransmits
Max number of tries to deliver failed message before it
gives up
84. ORTC
- WebRTC 1.1 (ORTC)
- SDP exchange not required
- Advanced stuff like simulcast
- More low level than WebRTC
- Shim for WebRTC to work on top of ORTC
- Important goal is compatibility. Existing
apps should not be affected