1. WebRTC
A front-end perspective
Opera Software
Fronteers
2014
@shwetank

2. About me
Shwetank Dixit
Opera Software
Fronteers
2014
@shwetank
Developer Relations, Opera Software

3. http://www.thehindu.com/news/national/other-states/using-pcos-in-mumbai-show-id-proof/article4241422.ece

4. Source: https://www.flickr.com/photos/rambow/858719954

5. Source: https://www.flickr.com/photos/rambow/858717314 Source: https://www.flickr.com/photos/rambow/858719954

8. What is
WebRTC?
Real Time
Peer-to-Peer
Communication

10. but …
So far I’ve seen very
few web people
talking about it

11. Whats the biggest barrier to WebRTC adoption?*
Lack of Awareness
Not supported by MS
Not supported by Apple
Other
0% 5.5% 11% 16.5% 22%
* WebRTC World Outlook Survey 2014

12. WebRTC is for
but … Web Developers

13. Mediastream

14. Mediastream
RTCPeerConnection

15. Mediastream
RTCPeerConnection
RTCDataChannel

16. mediaStream
+
` `
Device Camera Microphone

17. getUserMedia
+
` `
Device Camera Microphone

18. Sqwiggle

19. gUM + CSS(Filters, Masks, Blend Modes)
Create interesting visual effects from
camera output in real-time using CSS

20. gUM + Web Audio
Sound powered actions
and navigation

22. World’s first webcam
http://en.wikipedia.org/wiki/Trojan_Room_coffee_pot

23. What can you achieve with gUM?
- Motion Detection (Baby Monitors, Security Cam)
- Hotspots
- Gesture Recognition
- Emotion Recognition
- Sound Detection and Visualisation
- Voice / Sound Activated Controls
- Capture video (Kind of) (whammy.js, gifshot)

24. gUM: Current way to do it
navigator.getUserMedia*
*with prefixes

25. gUM: Broken Error Handling
Exceptions are not caught by the current API.
Promises can fix this.
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html

26. gUM: Current Way
var failure = function(reason){
log("Failed to show camera: " + reason.message);
}
!
navigator.getUserMedia(constraints, function(stream) {
try {
$video.src = window.URL.createObjectURL(stream);
$video.play();
} catch(e) { failure(e); } //failure must tolerate non-MediaStreamError
}, failure);
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html

27. gUM: In the future ?
navigator.mediaDevices.getUserMedia

28. gUM: In the future ? Promises Proposal
navigator.mediaDevices.getUserMedia(constraints)
.then( function(stream) {
videoElement.src = window.URL.createObjectURL(stream);
videoElement.play();
}).catch(function (reason){
log(“Failed to show camera: ” +reason.message);
});
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Sep/0171.html

29. Debate
This is still up for debate.
Some are eager to just ship v1.0 and add support for
promises in gUM at a later stage.
!
Based on: http://lists.w3.org/Archives/Public/public-media-capture/2014Oct/0003.html

30. gUM: Other concerns
Impossible for a gUM app to
use the device flash for low-light
!
Use (auto) zoom
Use (auto) focus

31. gUM: Other concerns
“If you look further into this API then it
doesn’t offer any advanced features
that you expect in a good camera app:
focus, flash, zoom `
etc. It is clear that
…(getUserMedia).. has been designed
for use in P2P applications and not as a
dedicated camera experience.”
- Paul Kinlan

32. Mediastream Image-Capture
!
Will provide a deeper level of control
for taking photos
!
w3.org/TR/image-capture/

33. Mediastream Image-Capture
White Balance
ISO
Red-Eye Reduction
Exposure
Brightness
Contrast
`
Saturation
Sharpness
Zoom
takePhoto()

34. Mediastream Image-Capture
Focus? Flash? ¯_(ツ)_/¯

35. Getting permission is not easy
Some users don’t seem to
notice the permissions dialog
in certain browsers

36. Getting permission is not easy
Some users don’t seem to
notice the permissions dialog
in certain browsers

37. Some users don’t seem to
notice the permissions dialog
in certain browsers

38. Some users don’t seem to
notice the permissions dialog
in certain browsers

39. Some users don’t seem to
notice the permissions dialog
in certain browsers

40. Getting permission is not easy
Permissions dialogs are different on
different platforms and browsers …
`
especially mobile

42. HTTPS helps …
If the app is on HTTPS, then the
browser will remember the last
granted permission.

43. RTCPeerConnection

46. webrtcglossary.com

47. RTCPeerConnection

48. ICE - Interactive Connectivity Establishment
* May or may not wax a chump like a candle.

49. STUN
Whats my
external IP Address?
Your external
IP Address is x.x.x.x

50. TURN around…
Total eclipse of the (p2p) call …

51. TURN
Basically relays stuff from one
party to the other.

52. Big Fat TURN Server

53. TURN
Basically relays stuff from one
party to the other.
In this case, connection won’t
be p2p, but at least the
connection will be made!

54. TURN
Basically relays stuff from one
party to the other.
In this case, connection won’t
be p2p, but at least the
connection will be made!
Generally speaking, a TURN server will
include a STUN server too.

55. Free ICE Project
github.com/DamonOehlman/freeice
(List of free STUN/TURN servers to use in your projects)

56. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon

57. Session Description (SDP)
v=0
o=- 680121471469462884 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS GUKF430Khp9jEQiPrdYe0LbTAALiNAKAIfl2
Audio Lines
m=audio 54278 RTP/SAVPF 111 103 104 0 8 106 105 13 126
c=IN IP4 180.6.6.6
a=rtcp:54278 IN IP4 180.6.6.6
!
ICE Candidates
a=candidate:4022866446 1 udp 2113937151 192.168.0.197 36768 typ host generation 0
a=candidate:4022866446 2 udp 2113937151 192.168.0.197 36768 typ host generation 0
a=candidate:2706108158 1 tcp 1509957375 192.168.0.197 0 typ host generation 0
a=candidate:2706108158 2 tcp 1509957375 192.168.0.197 0 typ host generation 0
a=candidate:1853887674 1 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0
a=candidate:1853887674 2 udp 1845501695 46.2.2.2 36768 typ srflx raddr 192.168.0.197 rport 36768 generation 0
a=candidate:2157334355 1 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0
a=candidate:2157334355 2 udp 33562367 180.6.6.6 54278 typ relay raddr 46.2.2.2 rport 38135 generation 0

58. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Creates Offer
local description

59. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Offer
local description

60. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Offer
remote description
local description

61. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Creates and
sends Answer
local description
local description
remote description

62. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Answer
local description
local description
remote description

63. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Receives Answer
local description
local description
remote description
remote description

64. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
Connectivity
Checks
local description
local description
remote description
remote description

65. The WebRTC Signalling Process
WebSocket || XHR || SIP || Carrier Pigeon
P2P Connection Established

66. Libraries

70. Bandwidth Numbers
!
720p : 1-2 Mbps // VP8 at 30fps
360p : 0.5 - 1 Mbps // VP8 at 30fps
180p : 0.1 - 0.5 Mbps // VP8 at 30fps
Audio : .06 - 0.51 Mbps // Opus

71. Debugging
opera://webrtc-internals
chrome://webrtc-internals
about:webrtc
!
The getStats() method
!
callstats.io
!

72. Data Channels
Once RTCPeerConnection is
established, we can use Data Channels
to transfer data P2P.
Data can be text or binary data

73. Whats the advantage?
Data Channels provide a
* High Performance
* P2P
* Low Latency
Connection to others

74. Working with Data
datachannel
.onopen // Data Channel opened
.onclose // Data Channel closed
.onerror // Error occurred
.onmessage // New message arrived
.send() // Send a message

75. DataChannels can work
exactly like WebSockets if
you want, but they don’t
have to …

76. SCTP // Transport Protocol
DTLS // Encryption
UDP

77. notable options
ordered
send data packets in ordered fashion or not
!
maxRetransmitTime
Max time to try to send a failed message
!
maxRetransmits
Max number of tries to deliver failed message before it
gives up

78. In the wild
Peerflix
PeerCDN
WebTorrent

79. In the wild
Peerflix
@mafintosh
!
PeerCDN
WebTorrent
@feross

80. IsWebTRwToCR MeaoddyeYest.com

84. ORTC
- WebRTC 1.1 (ORTC)
- SDP exchange not required
- Advanced stuff like simulcast
- More low level than WebRTC
- Shim for WebRTC to work on top of ORTC
- Important goal is compatibility. Existing
apps should not be affected

85. Communication

86. Opera Software
Cheers!
@shwetank
Fronteers
2014
shwetankd@opera.com