Bug 259304 – In WKWebView, getUserMedia API calls fail with the message "The operation was aborted." in pages loaded at capacitor://localhost:8080

• Status: NEW
• Component: WebRTC

[ShivaSantoshT]

Overview

Our iOS mobile application is developed using the technology stack Angular+Ionic+Capacitor. In our iOS App, the webpages are loaded at capacitor://localhost:8080 in WKWebView. We are attempting to access the camera on our pages using the web API getUserMedia() and the API is failing with Exception AbortError. We have given Camera Permission in our info.plist file and we have verified that our pages are in a secure context. Window.isSecureContext flag is true for our pages.

Sample Code

homepage.html code-
<video autoplay playsinline id="videoElement"></video>

homepage.ts code-

import { AfterViewInit, Component } from '@angular/core';

@Component({
  selector: 'app-home',
  templateUrl: 'home.page.html',
  styleUrls: ['home.page.scss'],
})
export class HomePage implements AfterViewInit{

  constructor() {}

  ngAfterViewInit(): void {
    let video: any = document.querySelector("#videoElement");
    navigator.mediaDevices.getUserMedia({ video: { facingMode: "environment" }, audio:false })
    .then((stream) => {
      video.srcObject = stream;
      video.play();
    })
    .catch( (e) => {
      // eslint-disable-next-line no-console
      console.log("Something went wrong!" + e.message);
    });
  }

}

Output-
In WKWebView, getUserMedia API calls fail with the message "The operation was aborted." in pages loaded at capacitor://localhost:8080

Expected Result-
getUserMedia should resolve with stream object.

Additional Information-

We are able to successfully access the camera on other platforms like Android.

We are able to successfully access the camera when the application started locally and is accessed in Safari.

We are able to access the camera in WKWebView on the latest iOS device when we change our hostname from capacitor://localhost:8080 to capacitor://localhost. But the question is, why the getUserMedia API is not working in capacitor://localhost:8080 which is a secure context?

[Radar WebKit Bug Importer]

<rdar://problem/112828705>

[bearman]

@ShivaSantoshT

Have you been able to get audio working (and getting `getUserMedia()` permission prompt to show)?

[ShivaSantoshT]

Hi Team, 

In iOS 16 versions, we are not getting permission prompts for the Camera or Microphone when accessing the getUserMedia API on a page with the domain capacitor://localhost:8080.

I have added both Camera and Microphone permissions in the info.plist file of the application.

Privacy - Microphone Usage Description
Privacy - Camera Usage Description

For the below code, we are encountering an abort error with code-20, name-AbortError, and the message 'The operation was aborted.'

Html file
<video autoplay playsinline id="videoElement"></video>

homepage.ts code-

import { AfterViewInit, Component } from '@angular/core';

@Component({
  selector: 'app-home',
  templateUrl: 'home.page.html',
  styleUrls: ['home.page.scss'],
})
export class HomePage implements AfterViewInit{

  constructor() {}

  ngAfterViewInit(): void {
    let video: any = document.querySelector("#videoElement");
    navigator.mediaDevices.getUserMedia({ video: true, audio:true })
    .then((stream) => {
      video.srcObject = stream;
      video.play();
    })
    .catch( (e) => {
      // eslint-disable-next-line no-console
      console.log("Something went wrong!" + e.message);
    });
  }
}

Please find below the link for the sample code published in the public Git repo-
https://github.com/TShivaSantosh/ionic-sample-app-camera-issue

Steps to reproduce-
Download the project from the link above.
Run 'npm install' inside the folder.
Run the sample app on an iOS 16 device.
Launch the application.
Camera and Microphone permissions should be requested, and the preview should be displayed inside the video element on the HTML page.

Additional information

The issue mentioned above is occurring on iOS 16. However, when I checked the getUserMedia API (using the above code) on a page with the URL capacitor://localhost:8080, it worked without any problems on iOS 17 beta. Can we assume that the above API will continue to work in the iOS 17 release?

[bearman]

(In reply to ShivaSantoshT from comment #3)
> Hi Team, 
> 
> In iOS 16 versions, we are not getting permission prompts for the Camera or
> Microphone when accessing the getUserMedia API on a page with the domain
> capacitor://localhost:8080.
> 
> I have added both Camera and Microphone permissions in the info.plist file
> of the application.
> 
> Privacy - Microphone Usage Description
> Privacy - Camera Usage Description
> 
> For the below code, we are encountering an abort error with code-20,
> name-AbortError, and the message 'The operation was aborted.'
> 
> Html file
> <video autoplay playsinline id="videoElement"></video>
> 
> homepage.ts code-
> 
> import { AfterViewInit, Component } from '@angular/core';
> 
> @Component({
>   selector: 'app-home',
>   templateUrl: 'home.page.html',
>   styleUrls: ['home.page.scss'],
> })
> export class HomePage implements AfterViewInit{
> 
>   constructor() {}
> 
>   ngAfterViewInit(): void {
>     let video: any = document.querySelector("#videoElement");
>     navigator.mediaDevices.getUserMedia({ video: true, audio:true })
>     .then((stream) => {
>       video.srcObject = stream;
>       video.play();
>     })
>     .catch( (e) => {
>       // eslint-disable-next-line no-console
>       console.log("Something went wrong!" + e.message);
>     });
>   }
> }
> 
> Please find below the link for the sample code published in the public Git
> repo-
> https://github.com/TShivaSantosh/ionic-sample-app-camera-issue
> 
> Steps to reproduce-
> Download the project from the link above.
> Run 'npm install' inside the folder.
> Run the sample app on an iOS 16 device.
> Launch the application.
> Camera and Microphone permissions should be requested, and the preview
> should be displayed inside the video element on the HTML page.
> 
> Additional information
> 
> The issue mentioned above is occurring on iOS 16. However, when I checked
> the getUserMedia API (using the above code) on a page with the URL
> capacitor://localhost:8080, it worked without any problems on iOS 17 beta.
> Can we assume that the above API will continue to work in the iOS 17 release?

Thanks for the additional information, I'm experiencing a very similar issue but with Swift & native macOS build.

If/when possible, please check the following:

1. `navigator.mediaDevices` is defined (including adding the appropriate `Audio-Input`, `Camera` Hardened Runtime permissions and associated plist entries)

You can also check this via Safari developer console, by evaluating javascript when the app is opened in debug.

2. (assuming 1 passes), please evaluate `navigator.mediaDevices.enumerateDevices()` and post the output here (omit any repetitions as it could be lengthy)

With hope, my issue is repeatable (even though we're building to different platforms).

[bearman]

Link to my issue for additional context, and possibly related root cause?

https://bugs.webkit.org/show_bug.cgi?id=259465