getUserMedia does not work in WKWebView-based browsers like Chrome, Firefox.

• Component: Media
• Status: RESOLVED CONFIGURATION CHANGED

[Erik Hodge]

getUserMedia does not work in iOS WKWebView-based browsers like Chrome, Firefox, works in iOS Safari.

Bug 185448 has been resolved, but it only addresses apps launched from the iOS home screen:
https://bugs.webkit.org/show_bug.cgi?id=185448
A comment in 185448 says it is confirmed fixed in iOS 13.4 beta 1, and that getUserMedia still doesn't work in Chrome in that build

[gabrielstuff]

Hello,

I confirm that this is utterly needed. WKWebView-based browsers like Chrome, Firefox, Brave, Phonegap App, Capacitor, etc... is a must !

[Peter]

Agreed this opens up whole new possibilities.

Please fix! 

I was super excited when I thought it was already addressed with https://bugs.webkit.org/show_bug.cgi?id=185448 and then equally as bummed when I realized that would not address WKWebView-based browsers. Truly a rollercoaster of emotions.

Thanks for listening.

[Alejandro Silva]

I thought iOS beta 13.4 fixed WKWebView to have getUserMedia, it would be cool to have it to avoid the use of SFSafariViewController inside an app, just to start a WebRTC flow

[Matt T]

I can unfortunately verify in 13.4 beta 4 that navigator.getUserMedia is still not there for WKWebView :( Apple please, this blocks so many apps!

[KeiroMidori]

It indeed is very much needed! Please Apple make it happen.

[Peter]

Is it reasonable to expect this will be fixed, since Bug 185448 was finally resolved, and this is closely related?

[Eric]

Hi.

Just want to report that due to THIS single bug, my company advises all our clients to buy Samsung tablets instead of iPads.

This bug accounts for probably hundreds of Android tablet purchases for this month.

If you want to stay relevant in the browser market during these days of stay-at-home, this is a MUST FIX.

[Simon]

Please fix this one. 

It is super annoying that I cannot use getusermedia in WKwebview Apps. Please allow to use this function!

[b0sh]

Looking forward to this fix :)

[Simon]

(In reply to Eric from comment #8)
> Hi.
> 
> Just want to report that due to THIS single bug, my company advises all our
> clients to buy Samsung tablets instead of iPads.
> 
> This bug accounts for probably hundreds of Android tablet purchases for this
> month.
> 
> If you want to stay relevant in the browser market during these days of
> stay-at-home, this is a MUST FIX.


We changed to SFSarafiWebView to fix this. You cant customize this view, but at least it works.

[Eric]

(In reply to Simon from comment #11)
> (In reply to Eric from comment #8)
> > Hi.
> > 
> > Just want to report that due to THIS single bug, my company advises all our
> > clients to buy Samsung tablets instead of iPads.
> > 
> > This bug accounts for probably hundreds of Android tablet purchases for this
> > month.
> > 
> > If you want to stay relevant in the browser market during these days of
> > stay-at-home, this is a MUST FIX.
> 
> 
> We changed to SFSarafiWebView to fix this. You cant customize this view, but
> at least it works.

Thank you Simon for your reply.

We use the Cordova framework for our app and SFSafariWebView cannot fit 
in what we do.

Until Apple fixes WkWebViews, we have no other choices than recommend Android.

Please Apple, give the deserved attention your software needs. We just wanna make awesome software for your once excellent platform.

[Frederik Riedel]

It would be great to have the getUserMedia capabilities in WKWebView. Not only for 3rd party web browsers, as mentioned above, but also for 3rd party apps, embedding Web View content that utilizes WebRTC.

[Peter]

Yes, much more so than support in Chrome
or Firefox is the need for this in our apps.

[Peter]

In the last two weeks alone I’ve turned down four potential huge deals for integrations of my web app in other people’s mobile apps... For no other reason than this bug.

Please, please get on this.

[Mickael]

I have a WKWebView in my macOS app and I am trying to enable the userMedia. To do this I have to use private API.

I have successfully enabled userMedia in the webview by creating a category in Objective-C:

```
@interface WKPreferences (MyPreferences)
- (void)_setMediaDevicesEnabled:(BOOL)enabled;
- (void)_setMediaStreamEnabled:(BOOL)enabled;
- (void)_setWebRTCLegacyAPIEnabled:(BOOL)enabled;
- (void)_setAVFoundationEnabled:(BOOL)enabled;
- (void)_setMediaSourceEnabled:(BOOL)enabled;

- (void)setMediaDevicesEnabled:(BOOL)enabled;
@end
```
And:
```
@implementation WKPreferences (MyPreferences)

- (void)setMediaDevicesEnabled:(BOOL)enabled {
    [self _setAVFoundationEnabled:enabled];
    [self _setMediaStreamEnabled:enabled];
    [self _setMediaSourceEnabled:enabled];
    [self _setWebRTCLegacyAPIEnabled:enabled];
    [self _setMediaDevicesEnabled:enabled];
}

@end
```
Using in Swift code:

```
webView.configuration.preferences.setMediaDevicesEnabled(true)
```

This code makes webviews appear their icons to record the voice or launch video. In reality just it was enough to enable `(void) _setMediaDevicesEnabled: (BOOL)enabled` but since it does not work entirely so I have to add all the others.

The problem is that when I click in the webview on the record button or start the video, I get this error:

```
[plugin] AddInstanceForFactory: No factory registered for id <CFUUID 0x6000003b14c0> F8BB1C28-BAE8-11D1-9C31-00032314CD45 HALC_ShellDriverPlugIn::Open: Can't get a pointer to the Open routine [] CMIO_DAL_PlugInManagement.cpp:191:Initialize Missing device-camera entitlement
```

I guess I need to implement delegates to be able to request permission and launch the camera or microphone. I found delegates who could be the right ones in the `WKUIDelegatePrivate`

```

- (void)_webView:(WKWebView *)webView requestUserMediaAuthorizationForDevices:(_WKCaptureDevices)devices url:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL decisionHandler:(void (^)(BOOL authorized))decisionHandler;
- (void)_webView:(WKWebView *)webView checkUserMediaPermissionForURL:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL frameIdentifier:(NSUInteger)frameIdentifier decisionHandler:(void (^)(NSString *salt, BOOL authorized))decisionHandler;
- (void)_webView:(WKWebView *)webView mediaCaptureStateDidChange:(_WKMediaCaptureState)state;
- (void)_webView:(WKWebView *)arg1 requestMediaCaptureAuthorization:(unsigned long long)arg2 decisionHandler:(void (^)(BOOL))arg3;
```

So I tried to implement them but the delegates are NEVER CALLED. This is my problem: how can I initialize WKUIDelegatePrivate?

The code I tried:

```
@protocol WKUIDelegatePrivate <WKUIDelegate>

- (void)_webView:(WKWebView *)webView requestUserMediaAuthorizationForDevices:(_WKCaptureDevices)devices url:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL decisionHandler:(void (^)(BOOL authorized))decisionHandler;
- (void)_webView:(WKWebView *)webView checkUserMediaPermissionForURL:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL frameIdentifier:(NSUInteger)frameIdentifier decisionHandler:(void (^)(NSString *salt, BOOL authorized))decisionHandler;
- (void)_webView:(WKWebView *)webView mediaCaptureStateDidChange:(_WKMediaCaptureState)state;
- (void)_webView:(WKWebView *)arg1 requestMediaCaptureAuthorization:(unsigned long long)arg2 decisionHandler:(void (^)(BOOL))arg3;

- (void)registerDelegate:(WKWebView *)webView;
@end


@interface UserMediaProvider : NSObject <WKUIDelegatePrivate>
@end
```

And:
```
@implementation UserMediaProvider

- (id)init {
    self = [super init];
    if (self) {
    }

    return self;
}

- (void)_webView:(WKWebView *)webView requestUserMediaAuthorizationForDevices:(_WKCaptureDevices)devices url:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL decisionHandler:(void (^)(BOOL authorized))decisionHandler {
    NSLog(@"AAAAAAAAA");
}

- (void)_webView:(WKWebView *)webView checkUserMediaPermissionForURL:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL frameIdentifier:(NSUInteger)frameIdentifier decisionHandler:(void (^)(NSString *salt, BOOL authorized))decisionHandler {
    NSLog(@"BBBBBB");
}

- (void)_webView:(WKWebView *)webView mediaCaptureStateDidChange:(_WKMediaCaptureState)state {
    NSLog(@"CCCCCC");
}

- (void)_webView:(WKWebView *)arg1 requestMediaCaptureAuthorization:(unsigned long long)arg2 decisionHandler:(void (^)(BOOL))arg3{
    NSLog(@"PPPPPPP");
}


- (void)registerDelegate:(WKWebView *)webView {
    webView.UIDelegate = self;
}

@end
```

Using from Swift code:

```
webview.registerDelegate(self)
```

I see WKWebView.mm for example that uiDelegatePrivate is cast to UIDelegate when used:

```
id<WKUIDelegatePrivate> uiDelegatePrivate = static_cast<id <WKUIDelegatePrivate>>([self UIDelegate]);
```

Also I can see in UIDelegate.mm That the delegate is called via uidelegate
```

{
    auto delegate = m_uiDelegate.m_delegate.get();
    if (!delegate || !m_uiDelegate.m_delegateMethods.webViewRequestUserMediaAuthorizationForDevicesURLMainFrameURLDecisionHandler) {
        request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::UserMediaDisabled); // THIS
        return true;
    }

    bool requiresAudio = request.requiresAudio();
    bool requiresVideo = request.requiresVideo();
    if (!requiresAudio && !requiresVideo) {
        request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::NoConstraints);
        return true;
    }
...
```

How can I enable these delegate methods to be called?

I specify that it is for internal application of my company so I do not want the upload to the store

[geckse]

This bug currently affects heavily our iOS users of our WKWebkit based app. We've implemented a video-capture-feature the getUserMedia-Way and found a workaround (get camera stream trough the cordova-plugin-iosrtc plugin and render the stream to a canvas and capture this canvas with ffmpeg.) but it significantly impacts the performance and thus experience of our users. I don't know what else to tell them but that iOS simply does not support this feature, at the moment..

[Sudheer]

We do rely on getUserMedia to share camera across all platforms and this is seems to limiting our feature capability on IOS native app with webview. It will be nice to address this as it opens up new possibilities and features.

+1

This also breaks WebRTC voice.

[Matt A]

I can also confirm that this is still not fixed in iOS 14 Beta 1. I have reported it via Apple Developer Bug reporting tool, so hopefully another angle will help us. This is particularly worrying now since users can chose to used 3rd party apps so opening links via an invite in mail or SMS will get our users stuck.

Apple also does not have support for redirecting the user back to Safari which makes this a total mess. 

PLEASE WebKit team, solve this in iOS 14.

[Peter]

With Apple preparing to allow users to change their default browser in iOS 14, it is unreasonable for this bug to still exist beyond that. This is one of the few major bugs that makes third party browsers, and WebView in general, into second class citizens.

[Matt A]

Hi All, I have been able to speak directly to Apple Engineering via phone (due to the company I work for being involved in their enterprise support developer program) and they have acknowledged the issue however ... I was apparently the first to report this issue to them under their WebKit team. It will be moved up in priority if everyone can lodge a ticket about this issue via https://feedbackassistant.apple.com/ 

Please ensure:
- you specify the bug is on iOS 14 as that is their priority at the moment
- Specify it does not work on 3rd party browsers using WKWebView (i.e. Chrome).

More numbers means we have a better chance of this getting fixed especially as they are looking into iOS 14 bugs more now.

[Matt A]

Also make sure you specify it to be sent to the "WebKit" team, its the last option on the list of departments.

Thanks a lot Matt A, have added a ticket as well.

Their answer is surprising though given this thread: https://bugs.webkit.org/show_bug.cgi?id=185448

It's clearly been worked on in the narrower context of the home screen.

Any chance you could speak to Apple Engineering again and make sure the Webkit team is aware of both threads? Maybe it was somehow a different team that did the home screen work

[Eric]

(In reply to Matt A from comment #24)

Thank you Matt for the suggestion! I did it 👍
To Others: please do it as well to weigh in!

Howto: 
1. You need to use Safari for https://feedbackassistant.apple.com/
  (Chrome won't work, you get a 403)
2. You need to login with your Apple developer account to submit the feedback.
3. Mention iOS 14 in the title
4. Select "WebKit" as area

Here is what I posted in the description:

----


Bug https://bugs.webkit.org/show_bug.cgi?id=208667

This bug is a HUGE problem for several reasons:

1. It prevents ALL Cordova apps and apps relying on WkWebViews to use the WebRTC standard.
   Just because of this bug, we are forced to recommend all our clients to use Android phones & tablets, that's too bad.

2. It prevents ALL other browsers on iOS to offer video-conferencing, while Safari can
   => it's a nasty anti-competitive behaviour that will for sure be scrutinized by US House Antitrust Committee & EU Commission,
   and Apple should not accumulate evidence of evil conduct.

3. More generally speaking, fixing this bug would prove that Apple is committed to W3C standards for WebKit,
   and that WebKit is not a second class citizen in Apple ecosystem.
   Being faster than Chrome on rendering while lagging on standards is pointless.

Good luck for the fix!

[Peter]

Added my own feedback with Apple as well. Thank you.

This is really a missing feature. Lots of creativities are blocked by this missing feature. Hope, it gets addressed in release.

I'm also experiencing this issue in the context of a WKWebView embedded into my own app.

This specific bug is holding us off from promoting apple products to go along with our services. It not only is in the way of simple picture capture, it furthermore prohibits audio recording, video recording, qr (or other code) scans and any other application that would require audio and/or video stream data within the WKWebView.

This bug in itself is a deal-breaker for many when it comes to the choice of an ecosystem.

[uhurusurfa]

Definitely a show stopper for supporting Apple devices where users prefer to use Firefox or Chrome and definitely results in promoting non-apple devices as the preferred device for our platform.

[Jeffrey Alan Scudder]

Lots of people are downloading my new game in the App Store right now, and I can't believe that I have to tell them to go use the website instead if they want to access the beautiful Camera feature I have just added. Please fix this!

[Nick]

+1

[uhurusurfa]

Pretty fundamental piece of functionality missing for a modern app.

[Alexander Borsuk]

Please fix this bug, or provide a usable workaround, at least for WebRTC. WebRTC cam/mic access is required for any app which uses WebRTC.

[Yossi]

+1

[Mayowa Daniel]

Why dooesn't this work yet?

Browser manufacturers are forced to use WebKit and WebKit disables features that work in Safari? If the antitrust authorities weren't so incompetent then this would be a case for them.

So please fix it in the spirit of fair competition and finally let our apps run on your devices.

(In reply to friedrich.waack from comment #39)
> Browser manufacturers are forced to use WebKit and WebKit disables features
> that work in Safari? If the antitrust authorities weren't so incompetent
> then this would be a case for them.
> 
> So please fix it in the spirit of fair competition and finally let our apps
> run on your devices.

Completely agree.  Yet another example of Apple's anti-trust violation.  We should get visibility of this example to law makers.

They pretend to give users a choice, but then cripple the alternatives.

[Peter]

How is it possible that this bug still exists post iOS 14 where users are now allowed to change their default browser?

[Eric]

Overall, the SILENCE from APPLE about this CRITICAL issue shows it's an *intentional* anti-competitive practice.

There are no technical excuses that can explain why it would work in Safari and not in WkWebView.

It's a HUGE SHAME that should be sanctioned by authorities.

Please spread the word as widely as possible.

[Gabriele Nosso]

Is there any progress?
We absolutely need this for our app! (35k users)

[Erik Hodge]

Matt A: could you speak to Apple Engineering again and verify that the Webkit team has seen this thread and has seen our requests made through feedbackassistant.apple.com?  I just verified that this bug isn't fixed in iOS 14.1.

[Michal]

Hey @Apple,

please do something with this issue.
It's really shame that this problem is here so long.
Many WEB developers are stopped.
Don't be like Microsoft and their IE case, pls.


Reported via Feedback tool, 
thanks Eric for tutorial.

What, this is STILL not available in WKWebView? :D Seriously?

[Peter]

@Eric Carlson

Any reason why this is still outstanding?

[Bernard Baker]

Looks like this is fixed!

Tested on iOS iPhone 6, using latest Google Chrome.

[Bernard Baker]

(In reply to Bernard Baker from comment #49)
> Looks like this is fixed!
> 
> Tested on iOS iPhone 6, using latest Google Chrome.

But this test was for a Web based application.

[Erik Hodge]

Bernard: could you describe exactly what you did test that works?

[Bernard Baker]

Erik I tested the Mediarecorder API on Google Chrome on iOS iPhone 6.

[Frederik Riedel]

(In reply to Bernard Baker from comment #52)
> Erik I tested the Mediarecorder API on Google Chrome on iOS iPhone 6.

I just wanted to mention that the iPhone 6 does not support the latest iOS version (iOS 14). Latest iOS available for iPhone 6 is iOS 12.4.9.

[Peter]

Nope. Tested getUserMedia in Chrome on iOS 14.2 and it fails like it always has. Nothing has changed.

It is pretty ridiculous that this still isn't possible, considering iOS 14 allows you to change the default browser. How can you allow users to change the default browser when critical functionality is blocked from functioning in any third party browser? That's very, very anti-user and anti-developer at the same time.

[Bernard Baker]

Apologies if I got mixed up. I'm using an iPhone 6 with iOS 12.x.

I've programmed an API that implements the MediaRecorder API and I'm able to record video and audio in a web application that uses my API on Google Chrome on the iOS iPhone 6.

[Michal]

(In reply to Bernard Baker from comment #55)
> Apologies if I got mixed up. I'm using an iPhone 6 with iOS 12.x.
> 
> I've programmed an API that implements the MediaRecorder API and I'm able to
> record video and audio in a web application that uses my API on Google
> Chrome on the iOS iPhone 6.

@Bernard Baker which will not work for most of users with up-to-date system. I was quite happy if I saw activity here.

Sadly, nothing new.

[gabrielstuff]

I was also very happy to read that first. Please let's not forget the main interest of this thread :

Making getUserMedia work in a WKWebView. Wich means something that works in Chrome, Brave Browser, Firefox, Cordova or Capacitor App.

What works is Safari view so for now, getUserMedia is ok in :

- Safari
- Adding to home screen as WebApp

This simple glitch allow to test and see if WKWebview is ok : 

https://pwa-cam.glitch.me/

Just click the only button : "Open camera"

If it works, then you see your face or whatever you are aiming at with the camera.

If the bug is still active you see : 

"getUserMedia error : TypeError"


Thank you all.

Teamwork Makes the Dreamwork

[Bernard Baker]

@gabrielstuff I followed your instructions and it failed on Google Chrome on iOS iPhone 6.

However, I opened a web-app that I'm working on, and my implementation of the MediaRecorder API passed and opened the camera which is overlaid upon the web-app UI.

[Bernard Baker]

@gabrielstuff is correct. The bug is still there. My implementation falls back to the file picker API of iOS.

[Thomas Steiner]

I have just successfully tested this to work on iOS 14.3 (18C5054c): https://twitter.com/tomayac/status/1329339360685264896.

[geckse]

I tested it with an App, which (seems to load) the page in a WKWebView and it actually worked with iOS 14.3 Beta 2. But I still don't trust that app.. or still can't believe it?!

But it opens that full-screen capturing UI which @Bernard Baker mentioned. Maybe that goes away when we add the "playsinline" to the <video>-tag?

https://twitter.com/geckse/status/1329541371041026048

[geckse]

I managed to get webRTC with Video to run with an Ionic / Capacitor App!

Here is the example:
https://github.com/geckse/ionic-capacitor-webrtc-camera-example

So far it worked good. It prompts for camera and microphone permissions and worked after granting these like a charm. Even the overlapping Camera-UI-Popup mentioned by @Bernard Baker disappears, when you utilize the "playinline" attribute on the <video>-element. 

The only thing which makes it currently unuseable in that usecase with Ionic is that it seems to require to be run on "https://", even for localhost. With a custom protocol, which It often used by Hybrid-Frameworks, like Ionic ("ionic://") or Capacitor ("capacitor://") didn't work so far for me.. but maybe that's a problem for these frameworks..?

[Keg]

Working with Cordova and the Construct 3 HTML5 game framework on IOS, it looks like there has been improvement, but as geckse mentions, there is still an issue with a Cordova app getting access to getUserMedia, perhaps due to not using https for the corodova app?

It looks like this may be a security issue based on this link:
https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia

"Note: If the current document isn't loaded securely, navigator.mediaDevices will be undefined, and you cannot use getUserMedia(). See Security for more information on this and other security issues related to using getUserMedia()."

After adding NSCameraUsageDescription key and an appropriate string to the project info.plist, calling navigator["mediaDevices"]["getUserMedia"] is now available (previously before ios14 and adding the key it is undefined.)

However, after the camera permission pop up appears and is agreed to, access fails with this in debug from Xcode:

"2020-11-25 12:21:38.293062-0800 Camera fx[1629:786395] WARN: Error requesting camera:  NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission."

(I did not deny permission and checking under Privacy Settings for camera, the test app is marked as allowing camera permissions.)

I have looked at ways to have Cordova serve locally via https, but I have not yet found a solution (geckse details an interesting work around in the above git repo, which serves the entire app from an external https server).

[gabrielstuff]

It looks like now that this bug is fixed. We are stuck with this one : https://bugs.webkit.org/show_bug.cgi?id=138169

Without possibility to declare our custom protocole as secure, or to simply make the wkwebview trust our protocole, we are stuck.

Created attachment 415730 [details]
Confirm that getUserMedia() works with iOS 14 Beta 2 and Beta 3 using a WKWebView

Comment on attachment 415730 [details]
Confirm that getUserMedia() works with iOS 14 Beta 2 and Beta 3 using a WKWebView

I can confirm that getUserMedia() works with iOS 14 Beta 2 and Beta 3 using a WKWebView.
Unfortunately there is another annoying issue with "the emerging life" of getUserMedia().
When executing getUserMedia() two similar prompts for user permission appear for:
a) using the camera and (getUserMedia({ video: true }))
b) using the microphone (getUserMedia({ audio: true }))
Since this makes absolutely no sense for an enduser I consider this a bug.
I attached the HTML and screenshots that show the problem.
I use webView in Swift 5 as follows:
```
class ViewController: UIViewController, WKNavigationDelegate, UIScrollViewDelegate, WKUIDelegate {
    @IBOutlet var webView: WKWebView!
    
    override func loadView() {
        isLoading = true
        NotificationCenter.default.addObserver(self, selector: #selector(onOpenDeepLink(_:)), name: .openDeepLink, object: nil)

        super.loadView()
        let config = WKWebViewConfiguration()
        config.preferences.setValue(true, forKey: "allowFileAccessFromFileURLs")
        config.allowsInlineMediaPlayback = true
        navigatorGeolocation.setUserContentController(webViewConfiguration: config)
        webView = WKWebView(frame: .zero , configuration: config)
        webView.navigationDelegate = self
        webView.uiDelegate = self
        webView.scrollView.delegate = self
        webView.backgroundColor = UIColor(red:0.855, green:0.855, blue:0.855, alpha:1)
        webView.scrollView.backgroundColor = UIColor(red:0.855, green:0.855, blue:0.855, alpha:1)
        navigatorGeolocation.setWebView(webView: webView)
        view = webView
    }
    
    override func viewDidLoad() {
        super.viewDidLoad()
        url = URL(string: "https://app.buildagil.ch/assets/www/camera.html")!
    }
    
    func webView(_ webView: WKWebView, didFinish navigation: WKNavigation!) {
        guard #available(iOS 14.2, *) else {
            webView.evaluateJavaScript("delete navigator.__proto__.mediaDevices")
            return
        }
    }
}


``` 

Note: this behaviour is basically the same as we already know from the navigator.geolocation.getCurrentPosition() API. This problem can be solved by following the solution described here: https://stackoverflow.com/questions/39665367/how-to-prevent-wkwebview-to-repeatedly-ask-for-permission-to-access-location
There seems to be the same underlying problem causing this behavior with WKWebView. As it seems there is a problem with managing/mapping the permissions granted to the app (1st prompt) itself and the embedded webView (2nd prompt).

Comment on attachment 415730 [details]
Confirm that getUserMedia() works with iOS 14 Beta 2 and Beta 3 using a WKWebView

Unfortunately there is another annoying issue with "the emerging life" of getUserMedia().
When executing getUserMedia() two similar prompts for user permission appear for:
a) using the camera and (getUserMedia({ video: true }))
b) using the microphone (getUserMedia({ audio: true }))
Since this makes absolutely no sense for an enduser I consider this a bug.
I attached the HTML and screenshots that show the problem.
I use webView in Swift 5 as follows:
```
class ViewController: UIViewController, WKNavigationDelegate, UIScrollViewDelegate, WKUIDelegate {
    @IBOutlet var webView: WKWebView!
    
    override func loadView() {
        isLoading = true
        NotificationCenter.default.addObserver(self, selector: #selector(onOpenDeepLink(_:)), name: .openDeepLink, object: nil)

        super.loadView()
        let config = WKWebViewConfiguration()
        config.preferences.setValue(true, forKey: "allowFileAccessFromFileURLs")
        config.allowsInlineMediaPlayback = true
        navigatorGeolocation.setUserContentController(webViewConfiguration: config)
        webView = WKWebView(frame: .zero , configuration: config)
        webView.navigationDelegate = self
        webView.uiDelegate = self
        webView.scrollView.delegate = self
        webView.backgroundColor = UIColor(red:0.855, green:0.855, blue:0.855, alpha:1)
        webView.scrollView.backgroundColor = UIColor(red:0.855, green:0.855, blue:0.855, alpha:1)
        navigatorGeolocation.setWebView(webView: webView)
        view = webView
    }
    
    override func viewDidLoad() {
        super.viewDidLoad()
        url = URL(string: "https://app.buildagil.ch/assets/www/camera.html")!
    }
    
    func webView(_ webView: WKWebView, didFinish navigation: WKNavigation!) {
        guard #available(iOS 14.2, *) else {
            webView.evaluateJavaScript("delete navigator.__proto__.mediaDevices")
            return
        }
    }
}


``` 

Note: this behaviour is basically the same as we already know from the navigator.geolocation.getCurrentPosition() API. This problem can be solved by following the solution described here: https://stackoverflow.com/questions/39665367/how-to-prevent-wkwebview-to-repeatedly-ask-for-permission-to-access-location
There seems to be the same underlying problem causing this behavior with WKWebView. As it seems there is a problem with managing/mapping the permissions granted to the app (1st prompt) itself and the embedded webView (2nd prompt).

Comment on attachment 415730 [details]
Confirm that getUserMedia() works with iOS 14 Beta 2 and Beta 3 using a WKWebView

Unfortunately there is another annoying issue with "the emerging life" of getUserMedia().
When executing getUserMedia() two similar prompts for user permission appear for:
a) using the camera and (getUserMedia({ video: true }))
b) using the microphone (getUserMedia({ audio: true }))
Since this makes absolutely no sense for an enduser I consider this a bug.
I attached the HTML and screenshots that show the problem.
I use webView in Swift 5 as follows:
```
class ViewController: UIViewController, WKNavigationDelegate, UIScrollViewDelegate, WKUIDelegate {
    @IBOutlet var webView: WKWebView!
    
    override func loadView() {
        super.loadView()
        let config = WKWebViewConfiguration()
        config.allowsInlineMediaPlayback = true
        webView = WKWebView(frame: .zero , configuration: config)
        webView.navigationDelegate = self
        webView.uiDelegate = self
        webView.scrollView.delegate = self
        view = webView
    }
    
    override func viewDidLoad() {
        super.viewDidLoad()
        url = URL(string: "https://app.buildagil.ch/assets/www/camera.html")!
    }
    
    func webView(_ webView: WKWebView, didFinish navigation: WKNavigation!) {
        guard #available(iOS 14.2, *) else {
            webView.evaluateJavaScript("delete navigator.__proto__.mediaDevices")
            return
        }
    }
}


``` 

Note: this behaviour is basically the same as we already know from the navigator.geolocation.getCurrentPosition() API. This problem can be solved by following the solution described here: https://stackoverflow.com/questions/39665367/how-to-prevent-wkwebview-to-repeatedly-ask-for-permission-to-access-location
There seems to be the same underlying problem causing this behavior with WKWebView. As it seems there is a problem with managing/mapping the permissions granted to the app (1st prompt) itself and the embedded webView (2nd prompt).

Comment on attachment 415730 [details]
Confirm that getUserMedia() works with iOS 14 Beta 2 and Beta 3 using a WKWebView

Unfortunately there is another annoying issue with "the emerging life" of getUserMedia().
When executing getUserMedia() two similar prompts for user permission appear for:
a) using the camera and (getUserMedia({ video: true }))
b) using the microphone (getUserMedia({ audio: true }))
Since this makes absolutely no sense for an enduser I consider this a bug.
I attached the HTML and screenshots that show the problem.
I use webView in Swift 5 as follows:
```
class ViewController: UIViewController, WKNavigationDelegate, UIScrollViewDelegate, WKUIDelegate {
    @IBOutlet var webView: WKWebView!
    
    override func loadView() {
        super.loadView()
        let config = WKWebViewConfiguration()
        config.allowsInlineMediaPlayback = true
        webView = WKWebView(frame: .zero , configuration: config)
        webView.navigationDelegate = self
        webView.uiDelegate = self
        webView.scrollView.delegate = self
        view = webView
    }
    
    override func viewDidLoad() {
        super.viewDidLoad()
        url = URL(string: "https://app.buildagil.ch/assets/www/camera.html")!
    }
    
    func webView(_ webView: WKWebView, didFinish navigation: WKNavigation!) {
        guard #available(iOS 14.2, *) else {
            webView.evaluateJavaScript("delete navigator.__proto__.mediaDevices")
            return
        }
    }
}
``` 

Note: this behaviour is basically the same as we already know from the navigator.geolocation.getCurrentPosition() API. This problem can be solved by following the solution described here: https://stackoverflow.com/questions/39665367/how-to-prevent-wkwebview-to-repeatedly-ask-for-permission-to-access-location
There seems to be the same underlying problem causing this behavior with WKWebView. As it seems there is a problem with managing/mapping the permissions granted to the app (1st prompt) itself and the embedded webView (2nd prompt).

I also opsted a question here: https://developer.apple.com/forums/thread/669011

[youenn fablet]

@[e-mail], the two prompts should only happen the first time.
After that point, there should be only one prompt.
The first prompt is an OS prompt asking whether the application can access camera.
The second prompt is asking whether the page with the given domain can access camera.

We might add in the future API to control the second prompt.
The first prompt will remain as is but is a one time prompt.

In the meantime, you can control when the first prompt appears by using https://developer.apple.com/documentation/avfoundation/avcapturedevice/1624584-requestaccessformediatype.

[Michal]

Confirmed, yesterday update and Camera working in #PWA in Chrome browser on iOS 14.3. Let's update your bricks!

Big thanks @Apple!

Few next things which I need in PWA: receiving notifications on background, without running browser, contacts sync permission with PWA. These things already working on Android and not on iOS. Is there already existing issues for these?

[Sam Gabriel]

@Apple First great job on adding the getUserMedia API to the wkwebview. However as many other posters have mentioned, we are unable to use the API due to the requirement that the page is loaded with https and with no ability to override this requirement. When working with ionic or cordova apps that is not an option.
 
I understand security is at the center of Apple philosophy. However as a developer, we are left with no choice but to use other libraries that are not updated as often as Apple would update the OS. Many of these older versions of those libraries might contain gaping security issues that will not be updated in time, leaving the user more vulnerable. 

Using a cordova or other web frameworks should be no different that using a native app. And as a matter of fact the mere fix to apple's implementation is to package and ship libwebrtc with the app and use some URL handling to get around that one issue. Ultimately building a less performant, less tested and less secure applications. 

Thank you

[youenn fablet]

(In reply to Sam Gabriel from comment #72)
> @Apple First great job on adding the getUserMedia API to the wkwebview.
> However as many other posters have mentioned, we are unable to use the API
> due to the requirement that the page is loaded with https and with no
> ability to override this requirement. When working with ionic or cordova
> apps that is not an option.
>  
> I understand security is at the center of Apple philosophy. However as a
> developer, we are left with no choice but to use other libraries that are
> not updated as often as Apple would update the OS. Many of these older
> versions of those libraries might contain gaping security issues that will
> not be updated in time, leaving the user more vulnerable. 
> 
> Using a cordova or other web frameworks should be no different that using a
> native app. And as a matter of fact the mere fix to apple's implementation
> is to package and ship libwebrtc with the app and use some URL handling to
> get around that one issue. Ultimately building a less performant, less
> tested and less secure applications. 
> 
> Thank you

I am not familiar with ionic or Cordova.
I did a quick check with minibrowser in the simulator and loading http://localhost as the main page and calling getUserMedia does work there.

Is it possible that the main page is not http://localhost and an iframe loading from http://localhost is used?

[Ashley Gullen]

I think the problem is if you use a custom URL scheme, e.g. app://localhost - then getUserMedia still does not work.

I tested this myself and I also still can't get it to work on the app: scheme. Calling getUserMedia does show permission prompts, but you still get a NotAllowedError even if you approve them. Modern hybrid apps e.g. Cordova use a custom scheme to serve local content so it needs to be supported with that as well.

[youenn fablet]

(In reply to Ashley Gullen from comment #74)
> I think the problem is if you use a custom URL scheme, e.g. app://localhost
> - then getUserMedia still does not work.
> 
> I tested this myself and I also still can't get it to work on the app:
> scheme. Calling getUserMedia does show permission prompts, but you still get
> a NotAllowedError even if you approve them. Modern hybrid apps e.g. Cordova
> use a custom scheme to serve local content so it needs to be supported with
> that as well.

Right, I think the ask is to allow exposing getUserMedia in custom schemes.

It is odd that you get a permission prompt but end up with NotAllowedError.
That might be a separate bug.

[Sam Gabriel]

(In reply to youenn fablet from comment #73)
> (In reply to Sam Gabriel from comment #72)
> > @Apple First great job on adding the getUserMedia API to the wkwebview.
> > However as many other posters have mentioned, we are unable to use the API
> > due to the requirement that the page is loaded with https and with no
> > ability to override this requirement. When working with ionic or cordova
> > apps that is not an option.
> >  
> > I understand security is at the center of Apple philosophy. However as a
> > developer, we are left with no choice but to use other libraries that are
> > not updated as often as Apple would update the OS. Many of these older
> > versions of those libraries might contain gaping security issues that will
> > not be updated in time, leaving the user more vulnerable. 
> > 
> > Using a cordova or other web frameworks should be no different that using a
> > native app. And as a matter of fact the mere fix to apple's implementation
> > is to package and ship libwebrtc with the app and use some URL handling to
> > get around that one issue. Ultimately building a less performant, less
> > tested and less secure applications. 
> > 
> > Thank you
> 
> I am not familiar with ionic or Cordova.
> I did a quick check with minibrowser in the simulator and loading
> http://localhost as the main page and calling getUserMedia does work there.
> 
> Is it possible that the main page is not http://localhost and an iframe
> loading from http://localhost is used?

It is a custom URLScheme it is not an iframe issue. I believe it is ionic for the wkwebview. https://github.com/ionic-team/cordova-plugin-ionic-webview#configuration

[youenn fablet]

OK, let's use bug 220184 for custom schemes and getUserMedia.
I am closing this bug here since WKWebView applications can now have access to getUserMedia.

[Imre Farkas]

This probem stil present in iOS 15