Bug 233419 - WKWebView microphone access not working in background
Summary: WKWebView microphone access not working in background
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKit API (show other bugs)
Version: Other
Hardware: iPhone / iPad iOS 15
: P2 Blocker
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-11-22 05:22 PST by Franz
Modified: 2022-06-13 22:45 PDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Franz 2021-11-22 05:22:37 PST
What's happening: We're loading a WebRTC-based mobile-optimized chat web app into WKWebView which allows users to toggle on their microphone and talk to others in a virtual room. The WKWebView gets added as a subview to the root view controller's root view. We also want camera access, but I'm focusing on getting the microphone to work correctly for now.

However, upon sending the app to the background, all audio as well as the microphone stops working.
Expected behavior: Audio and microphone continue to work as the user backgrounds the app.

Under "Signing & Capabilities", we're specifying these Background Modes:
+ Audio, AirPlay, and Picture in Picture
+ Voice over IP

The .entitlements file specifies:
  com.apple.security.network.client = 1
  com.apple.security.device.camera = YES
  com.apple.security.device.audio-input = YES

Side note: Editing the capabilities does not seem to change the .entitlements file in consistent ways, so I'm not sure this is a potential part of the problem.

While running the app on a device in Debug, the console shows these suspicious logs:

[assertion] Error acquiring assertion: <Error Domain=RBSServiceErrorDomain Code=1 "(originator doesn't have entitlement com.apple.runningboard.assertions.webkit AND originator doesn't have entitlement com.apple.multitasking.systemappassertions)" UserInfo={NSLocalizedFailureReason=(originator doesn't have entitlement com.apple.runningboard.assertions.webkit AND originator doesn't have entitlement com.apple.multitasking.systemappassertions)}>

[ProcessSuspension] 0x102cbca50 - ProcessAssertion: Failed to acquire RBS assertion 'WebKit Media Playback' for process with PID=476, error: Error Domain=RBSServiceErrorDomain Code=1 "(originator doesn't have entitlement com.apple.runningboard.assertions.webkit AND originator doesn't have entitlement com.apple.multitasking.systemappassertions)" UserInfo={NSLocalizedFailureReason=(originator doesn't have entitlement com.apple.runningboard.assertions.webkit AND originator doesn't have entitlement com.apple.multitasking.systemappassertions)}

The mentioned entitlements com.apple.runningboard.assertions.webkit and com.apple.multitasking.systemappassertions seem to be private / undocumented. I've tried adding them anyways just to see whether that helps, and uninstalled the app to get a fresh install. It didn't work. So I attempted submitting a release build to TestFlight, but App Store Connect rejected the build due to unsupported use of these entitlements.

I also tried keeping the WKWebView alive by repeatedly injecting JavaScript evaluating "1*1" every 0.2 seconds, as some suggested. This didn't help either.

Is there any way to keep WebRTC and the microphone alive while a WKWebView-based app enters the background?

I consider it a blocking bug, as it not being able to background the app while talking to others makes it useless in productivity contexts (which is the app's focus).

If I've missed something, I'd be glad to hear about it.

Thanks
Comment 1 Franz 2021-11-22 07:53:08 PST
Observed system behavior:

+ User's microphone & camera works (using webRTC).

+ User's microphone & camera doesn't work in the background or when locking the screen.

+ User's microphone & camera resumes working when re-entering the foreground.

+ Incoming webRTC audio (other people talking) keeps working when entering the background or locking the screen.

+ Incoming audio requires one user interaction so it can play, and persists across entering background & relaunching (note: This is bad UX, it should resume working without user interaction).

+ Other remote user turns on their microphone: Requires user interaction so you can hear it (bad UX).

+ Other remote user turns their microphone off and on again very fast: No user interaction required again, can still hear it.

+ Other remote user turns their microphone off and turns it on after a few seconds: User interaction required to hear it (bad UX).

+ Incoming remote video does not require user interaction to become visible (nice!).

+ User goes offline for everyone else when locking the screen or entering the background, unless someone else has their microphone on, resulting in an incoming audio stream. That seems to keep the session/process alive. Upon failure I can observe Apollo/GraphQL load errors, indicating that WKWebView may have lost internet connectivity (not clear this is the case).
Comment 2 Radar WebKit Bug Importer 2021-11-26 00:19:44 PST
<rdar://problem/85761617>
Comment 3 youenn fablet 2021-11-26 00:20:28 PST
<rdar://85675726>