This document summarizes a webinar about WebRTC standards hosted by Jan-Ivar Bruaroey of Mozilla. The webinar discussed the getDisplayMedia specification for screen sharing, constraints for downscaling shared screens, and security considerations for sharing full screens and browser tabs. It also covered the history of screen sharing in browsers, implementation status across browsers, and questions about supporting audio with screen sharing and handling requests from iframes.

1. WebRTC Standards Webinar & Q&A
Amir Zmora
TheNewDialTone
Jan-Ivar Bruaroey
Mozilla

2. Watch video recording of this session
http://ccst.io/e/webrtcstandards28

3. Jan-Ivar Bruaroey
jib on irc.mozilla.org #media
@jibrewery on Twitter
On the Firefox Media team, working to implement WebRTC:
getUserMedia(), getDisplayMedia(), RTCPeerConnection().
W3C editor of mediacapture-main and webrtc-pc.
Contributor to mediacapture-screen-share and other specs.
I’m currently implementing getDisplayMedia() in Firefox, and
contributing to finish off its specification.

4. Session sponsored by
WebRTC.ventures is a custom design and development shop dedicated to building WebRTC based applications
for web and mobile. We have built end-to-end broadcast solutions for events and entertainment clients,
telehealth solutions for multiple clients, live support tools, as well as communication tools for a variety of other
applications. WebRTC.ventures is a recognized development partner of TokBox and has also built native
WebRTC solutions

5. We use CrowdCast….It’s WebRTC

6. getDisplayMedia 1.0
Standard Screen-sharing

7. Outline
1. Introduction to getDisplayMedia()
2. Constraints for downscaling
3. Finalizing the spec.
4. Security concerns. Full-screen/browser tab-sharing is
“scary”.
5. Permissions
6. Iframe permissions (Feature Policy)

8. History
Since 2013-14,Chrome and Firefox have supported screen capture through incompatible
proprietary and non-standard extensions of getUserMedia().
Firefox with the mediaSource constraint (dropped white-list in 52).
Chrome requires an extension be installed; with chromeMediaSource constraint.
Progress lingered on standard specification, until recently.
Chrome to drop inline installation of extensions soon; new friction injects urgency.

9. W3C Screen Capture specification
Spec doc: mediacapture-screen-share.
Standard way to let websites request the end-user to share their display of choice.
Nearing completion. Implementations available:
• Edge: Yes (early 2018 draft)
• Chrome: In development (yes behind pref in Canary, early 2018 draft)
• Safari: In development
• Firefox: In development. Targeting 64 or 65.
• Adapter.js: Yes (early 2018 draft), w/setup in Firefox/Chrome w/Chrome plugin.

10. Screen-sharing with getDisplayMedia()
Early 2018:
video.srcObject = await navigator.getDisplayMedia({video: true});

11. Downscaling with track.applyConstraints()
Early 2018: constraints on gDM forbidden. Mustn’t influence user-selection. Instead:
const stream = video.srcObject = await navigator.getDisplayMedia({video: true});
// Got it! Now downscale to not be huge
await stream.getVideoTracks()[0].applyConstraints({width: 320, height: 200});

12. Downscaling with getDisplayMedia()
Late 2018: constraints allowed, but browsers MUST apply only after user selection.
stream = await navigator.getDisplayMedia({video: {width: 320, height: 200}});
// Got it! And it’s downscaled already.
Simpler. But what about OverconstrainedError? Would stink post-prompt (more later).

13. Downscaling with getDisplayMedia()
Of course you can still use applyConstraints and the rest of the API:
stream = await navigator.getDisplayMedia({video: {width: 320, height: 200}});
const [track] = stream.getVideoTracks();
const halve = async () => {
const {width, height} = track.getSettings();
await track.applyConstraints({width: width / 2, height: height / 2});
}
halve();

14. Constraints in getDisplayMedia() 1.0
Some differences from camera. Constraints are for downscaling, not discovery.
Therefore, the following constraint keywords are disallowed:
await navigator.getDisplayMedia({video: {width: {min: 320}}); // TypeError
await navigator.getDisplayMedia({video: {width: {exact: 320}}); // TypeError
await navigator.getDisplayMedia({video: {advanced: [{width: 320}]}); // TypeError
But the ideal and max constraints are allowed (more on this later).
This eliminates the risk of post-prompt OverconstrainedError.
Again, getDisplayMedia() is not for discovery.

15. Captured windows may have surprising aspect.
await gDM({video: {width: 320, height: 200}});
Downscaling always preserves aspect, never crops
Best effort based on fitness distance.
There’s some question whether fitness distance alone
gives good results.
Firefox uses-area-based approximation when aspects
differ. Allowed to go out of bounds.
ideal

16. Why? End-user might resize window extremely tall or wide during live capture.
await gDM({video: {width: {ideal:320, max:320}, height: {ideal:200, max:340}}});
Use max constraint to define outer bounds
Having two forms of constraints allows room for aspect
changes within limits.
If end-user makes window any taller, it’s always
downscaled to fit within outer bounds.
ideal
max
User resizes live. Video stays inside bounds

17. How to detect Retina displays
These render at 2x size. Unexpected? Detectable? The plan is to downscale these to ½ by default to
render at the expected same scale. This also makes it detectable:
stream = await navigator.getDisplayMedia({video: true}); // unconstrained
const [track] = stream.getVideoTracks();
const isRetina = track.getSettings().resizeMode != ‘none’; // already downscaled?
// I have 1 gigabit upload, no defaults for me, I want the whole enchilada
await track.applyConstraints({width: 9999, height: 9999}); // no worry no upscale

18. How to detect Retina displays (alternative)
Another approach that could someday work with other constraints applied:
stream = await navigator.getDisplayMedia({video: myConstraints}); // constrained
const [track] = stream.getVideoTracks();
const {width, height} = track.getSettings();
const cap = track.getCapabilities();
const isRetina = width < cap.width.max || height < cap.height.max;
await track.applyConstraints({width: cap.width.max, height: cap.height.max});
But this depends on expectations of getCapabilities() for windows that change size.

19. Adapter.js polyfill
Works in Firefox (and Edge of course): https://jsfiddle.net/jib1/q75yb8pf/
adapter.browserShim.shimGetDisplayMedia(window, ‘window’); // or ‘screen’
const stream = video.srcObject = await navigator.getDisplayMedia({video: true});
We will combine “screen” and “window” choices in Firefox Nightly soon.
Constraint support demo: https://jsfiddle.net/jib1/dkt5wexz/
Works with Firefox and Canary (Edge does not allow constraints on getDisplayMedia)
Shim for Chrome plugin is more involved. Google “adapter.js getDisplayMedia”.

20. Finalizing the spec: How to ask for audio?
This isn’t in the spec yet. But Chrome apparently has this ability, but only for browser tabs. Maybe:
await navigator.getDisplayMedia({video: true, audio: true});
Should this fail immediately with OverconstrainedError if audio is unsupported, or just return a
video track with no audio?
Some sources may have audio, others may not. How to handle that? Silence?
What about:
await navigator.getDisplayMedia({audio: true});
?

21. Security concerns
Full-screen/browser sharing is scary!
Not just passive threats.
If a web surface under site control is
captured, that website has keys to the car,
and can iframe-navigate as the logged-in
user effectively.
Sidesteps cross-origin protections.
Firefox warns, but hard to explain !
Google “share screen trust” for more

22. SecureContext
getDisplayMedia() will be https only, pending w3c/mediacapture-screen-share#77
if (!(‘getDisplayMedia’ in navigator)) {
// http or unsupported in browser
return;
}

23. Persistent “granted” permissions are forbidden.
Persistent “denied” permissions are allowed.
Prompts user before every access.
Website cannot narrow user choice,
(to avoid steering them toward scary sources).
Spec recommends elevated permission for scary sources.
Question: How to prompt for video + audio?
Permissions

24. How to tackle requests from iframes?
Users don’t read URLs in prompts! Too hard.
wicg.github.io/feature-policy to the rescue.
Says: All grants to top-level domain.
It’s up to top-level domain to delegate to iframes.
Similar to camera and microphone:
<iframe allow="camera; microphone; display">
Disallow by default. navigator.getDisplayMedia remains but throws SecurityError
Should “display” cover both video and related audio?
Iframe permissions (Feature Policy) Cross-origin domain!
“screen” or
“display”?

25. Questions? Use-cases?
We’re interested in hearing of screen-sharing use-cases we haven’t thought of.
For instance, appear.in uses screen-sharing to record a WebRTC meeting.
Audio use-cases? Any not covered by web-audio? Audio-only?
Do security measures seem sufficient? Expect any different in Private Browsing?

26. ?

27. Session sponsored by

28. WebRTC Standards Webinar & Q&A
Amir Zmora
TheNewDialTone
Jan-Ivar Bruaroey
Mozilla