Bug 259465 – No permission prompt for getUserMedia

• Component: WebRTC
• Status: RESOLVED WORKSFORME

[bearman]

Created attachment 467108 [details]
MCVe WKWebView implementation

Foreword: I've reviewed almost every issue in the webkit relating to prompts and getUserMedia, most are seemingly irrelevant with dead threads claiming 'fixed' but this issue persists.

Platform: 
- MacOS 12.6.8, 
- Safari 16.6

This is a typical usage of `getUserMedia` (for permissions only)
```
navigator.mediaDevices.getUserMedia({audio: true})
    .then((e) => alert("Permissions granted"))
    .catch((e) => alert("Permissions Denied"))
```

Adding this to an html <script/> tag, evaluating it during runtime as a function or calling it in web inspector console all provide the same results:

- On websites like google.com, bing.com, etc the above succeeds as expected: shows prompt and grants mic/camera access with indicator in status bar, etc.

- When accessing site via WKWebView (in iOS or macOS Swift application), this repeatably and consistently fails. However, asking for native microphone permissions succeeds (i.e. via AVCaptureDevice).

By failing (calling the following in `WKWebView`):

- `navigator.mediaDevices` is present
- `navigator.mediaDevices.enumerateDevices()` resolves with a single MediaDeviceInfo object: `[{deviceId: "", kind: "audioinput", label: "", groupId: ""}]`
- `navigator.mediaDevices.getUserMedia({audio: true, video: true})` promise never resolves but remains pending forever.

See more information here with full write-up (https://developer.apple.com/forums/thread/734363)

Find attached a MVCe Webkit (WKWebView) sample application to demonstrate this (bug?)

[bearman]

MCVE: https://github.com/cybex-dev/WKWebView

[bearman]

I've been able to narrow the issue down slightly.

With Hardened Runtime Permissions (macOS), with the capability: missing, or present but `Audio Input` and/or `Camera` permissions not added `navigator.mediaDevices` will be undefined.


With `Audio Input` and/or `Camera` permissions added (checkboxes), the following outputs can be found in Safari debug terminal:

`getUserMedia` - defined, promise hangs, no prompt for permissions
`enumerateDevices` - provides XCode Console output (native) of :

*Note: date & time omitted, shortened output due to duplicates*
```
WKWebView[60215:1070748] [plugin] AddInstanceForFactory: No factory registered for id <CFUUID 0x600000266820> F8BB1C28-BAE8-11D6-9C31-00039315CD46
WKWebView[60215:1070748]   saved enable noise cancellation setting is the same as the default (=1)
WKWebView[60215:1070748] [plugin] AddInstanceForFactory: No factory registered for id <CFUUID 0x600000271760> 30010C1C-93BF-11D8-8B5B-000A95AF9C6A
WKWebView[60215:1070748] [plugin] AddInstanceForFactory: No factory registered for id <CFUUID 0x600000271760> 30010C1C-93BF-11D8-8B5B-000A95AF9C6A
WKWebView[60215:1070703]  HALC_ShellObject::GetPropertyData: call to the proxy failed, Error: 1852797029 (nope)
WKWebView[60215:1070703]  HALPlugIn::ObjectGetPropertyData: got an error from the plug-in routine, Error: 1852797029 (nope)
...
```

[Eric Carlson]

Created attachment 467124 [details]
Modified test app

In your test app, `enumerateDevices` succeeds but you forgot to call `overrideLogging` so you don't inject the JS `log` function so the app isn't notified and getDevices gets a script error. You could have seen this by examining your app with the web inspector.

The call to `getUserMedia` never prompts because the webview is not visible when it is called. Exactly the same thing happens in Safari if `getUserMedia` is called from a background tab. In this situation, the call is made once the view becomes visible. 

I have attached a slightly modified version of your test app that parents the webview and injects your `log` script. It also fixes a bug that made the app create three instances of the webview.

[bearman]

(In reply to Eric Carlson from comment #3)
> Created attachment 467124 [details]
> Modified test app
> 
> In your test app, `enumerateDevices` succeeds but you forgot to call
> `overrideLogging` so you don't inject the JS `log` function so the app isn't
> notified and getDevices gets a script error. You could have seen this by
> examining your app with the web inspector.
> 
> The call to `getUserMedia` never prompts because the webview is not visible
> when it is called. Exactly the same thing happens in Safari if
> `getUserMedia` is called from a background tab. In this situation, the call
> is made once the view becomes visible. 
> 
> I have attached a slightly modified version of your test app that parents
> the webview and injects your `log` script. It also fixes a bug that made the
> app create three instances of the webview.

Absolutely wonderful!

Regarding `enumerateDevices`, calling this in the web inspector resolved to
```
WKWebView[60215:1070703]  HALC_ShellObject::GetPropertyData: call to the proxy failed, Error: 1852797029 (nope)
WKWebView[60215:1070703]  HALPlugIn::ObjectGetPropertyData: got an error from the plug-in routine, Error: 1852797029 (nope)
```

I intended to remove the logging override to reduce the complexity, but seems it was useful afterall.

Regarding the main issue that is GUM, I tested a WKWebView implementation on iOS simulator and native mac without success (i.e. a visible webview) however no prompt was visible. I'll take a look at report back anything that looks odd.

@Eric: Many thanks for your time and providing a working solution. The goal is to make use of the WKfeatures that JSC doesn't provide out of the box. 

Do you have any suggestions or recommendations for granting GUM on a headless WebView?

[bearman]

For additional context:
Flutter package, using a JS library for VOIP functionality (until native functionality is implemented)

[Eric Carlson]

> Do you have any suggestions or recommendations for granting GUM on a headless WebView?

From WebKit's perspective the Document has to be visible to get a prompt, see https://github.com/WebKit/WebKit/blob/c646f2370d230da0bf274514b07f0b6f8d624309/Source/WebCore/Modules/mediastream/MediaDevices.cpp#L169.

It may be enough to have a very small WKWebView, as long as it is parented.

[bearman]

(In reply to Eric Carlson from comment #6)
> > Do you have any suggestions or recommendations for granting GUM on a headless WebView?
> 
> From WebKit's perspective the Document has to be visible to get a prompt,
> see
> https://github.com/WebKit/WebKit/blob/
> c646f2370d230da0bf274514b07f0b6f8d624309/Source/WebCore/Modules/mediastream/
> MediaDevices.cpp#L169.
> 
> It may be enough to have a very small WKWebView, as long as it is parented.

Hmm, that should work.

Once again, many thanks for your help.