Bug 198040 – getDisplayMedia fails when called from a promise created during a user gesture

• Component: WebRTC
• Status: NEW

Created attachment 370251 [details]
getDisplayMedia example

Hi,

Since Safari Tech Preview 82 

> getDisplayMedia must be called from a user gesture handler.

If said handler has an async call to say getUserMedia or enumerateDevices, before
invoking getDisplayMedia, the call will always fail with the above message.

```javascript
//  my user gesture handler is called after a button is clicked
async function myUserGestureHandler() {
  // both will cause a failure
  // const camera = await navigator.mediaDevices.getUserMedia({ video: true });
  const devices = await navigator.mediaDevices.enumerateDevices();

  // calling getUserMedia here works
  const screen = await navigator.mediaDevices.getDisplayMedia({ video: true });
}
```

- Invoking getUserMedia works.
- Recent Firefox and Chrome versions work too.
- Worked in Safari Tech Preview 81.

I've attached an example page demonstrating the issue.

[Radar WebKit Bug Importer]

<rdar://problem/50950989>

[youenn fablet]

Hi Philipp,

I guess a workaround is to do:

const gumPromise = navigator.mediaDevices.getUserMedia({ video: true });
const screenPromise = await anavigator.mediaDevices.getDisplayMedia({ video: true });
const camera = await gumPromise;

Would that work for you?
Or do you want to NOT request getDisplayMedia access if getUserMedia is rejected?

I am also not sure why you want to do enumerateDevices there.


FWIW, it will, in most cases, trigger two prompts to the user in a very short amount of time. I wonder whether this is a good application flow.

Hi! Thanks for the quick response.

Let's say we want to build a screen sharing app where we mix the
camera and the screen stream.
We'd have a button "share screen" and invoke the two .getMedia
functions in a handler.
Is that handler function not a "user gesture handler"?

My main concern is that "user gesture handler" is ambiguous.
Can you clarify what a user gesture handler is and why the example
code I posted,
doesn't qualify as such?

If I invoke a different Promise instead of getUserMedia or enumarteDevices, e.g:

const examplePromise = () => {
  return new Promise(res => window.setTimeout(() => res(), 500));
};

A subsequent call to getDisplayMedia works just fine, but increasing
the delay to 1s causes the error message too.

The work-around is viable, but I'd it's still consider it a "work-around".
It's not clear why you'd have to invoke gDM before gUM, right?
I'd probably have to add a comment "order is important here!" so I
won't forget it in 6 months :).

(As far as enumarteDevices goes, that was just an example).

Hope that clarifies my issue.

[youenn fablet]

> A subsequent call to getDisplayMedia works just fine, but increasing
> the delay to 1s causes the error message too.

We currently bound the user gesture duration to 1s.

> The work-around is viable, but I'd it's still consider it a "work-around".
> It's not clear why you'd have to invoke gDM before gUM, right?

My example is invoking gum first, then gdm.
It is not waiting for gum response to call gdm response.

The questions might be:
1. Can your application live with just screen, just camera or require both?
2. Do you want to ask the user for gdm in case gum is rejected?

The current design does not allow you to not ask gdm if gum is rejected without another user gesture.

Note though that two prompts that are happening shortly one after the other might be confusing to the user.
A more user friendly flow might be to enter a room and at that time trigger gum or gdm. Then have the user interact with the UI to enable the other one.

> We currently bound the user gesture duration to 1s.

Thanks for the clarification.

> My example is invoking gum first, then gdm.
> It is not waiting for gum response to call gdm response.

My bad. The point is that the order is important,
because of the user gesture restrictions.

> 1. Can your application live with just screen, just camera or require both?

For this application I'd require both.

> 2. Do you want to ask the user for gdm in case gum is rejected?

No.

You're absolutely right, the current flow could be improved in that way.

I think a clarification for the behaviour might be good.

Maybe something like:

> getDisplayMedia must be called from a user gesture handler. Learn more at:
> https://bugs.webkit.org/show_bug.cgi?id=198040 or whatever the link might be.