Bug 204681 – Audio track goes to readyState ended when Safari is backgrounded for more than 30 seconds

• Status: NEW
• Component: WebRTC

[Dag-Inge Aas]

Created attachment 384455 [details]
Screenshot showing the issue on an iPhone X running iOS 13.2.3

Summary:
If a web page capturing audio is backgrounded for more than 30 seconds the audio track's readyState goes to "ended". This does not affect the video track. The audio track is now useless until you refresh the page or call getUserMedia again. Sometimes the video track will end as well, there seems to be some kind of difference in the background kill rate for these two resources.

Steps to reproduce:
1. Go to https://codepen.io/daginge/full/XWWvBbe on an iPhone running iOS 13.2.3
2. Tap: "Start"
3. Observe that video is now working, and the system logs as expected. You should also be able to hear yourself.
4. Go to the home screen, and stay there for at least 30 seconds, sometimes 5-6 seconds more.
5. Open Safari again
6. Observe that the audio track's readyState is now "ended".

Sometimes both video and audio will end. Calling getUserMedia again fixes the issue.

Expected behaviour:
If a page is capturing, don't background/clean up the Safari process so that it looses media access. I would expect once media access is granted that the device does not go to sleep or kill the process until the media access is cleanly shut down.

Impact:
Pages implementing some kind of waiting room functionality, or pages where a user might go to the home screen to do a task, then come back.

Workaround:
It seems using the Page Visibility API to trigger getUserMedia again works. If you are in a call you need to use replaceTrack to get the audio and video flowing again. Refreshing the page also works.

[youenn fablet]

We have some logic to check that the audio capture callback is being called regularly.
It might be that this does not happen when in the background.
After a few seconds, audio capture is considered as failing and the track will be ended.

[Dag-Inge Aas]

After more extensive testing and metrics in the wild I have found that while the track does go to ended sometimes, it's much more likely that it ends up as OS muted, or simply broken. Video and audio are frozen with no change to the MediaStream or MediaTrack objects. Sometimes the tab reloads itself, sometimes it breaks the tab to the point where I can't even use it to navigate to other pages like apple.com. The latter being the worst of all since some users don't know how to force quit safari, and we are unable to provide them with any help.

What do you need from me in terms of debugging information, and how do I get that, to be able to solve this issue?

[Dag-Inge Aas]

Just tested in iOS 13.3 stable. The red pill now remains active even after 30 seconds have passed. This seems to indicate that the OS now correctly keeps Safari alive when capturing.

However, upon returning to the tab after 1 minute (with a red camera icon) the video element is now black, and the audio has stopped working. Both audio and video track have muted = false and readyState = live. Reloading the page sometimes works to regain audio/video input, sometimes not.

On a positive note, when readyState goes to ended, the workaround to re-request getUserMedia seems to work. We do get the occasional NotReadableError as mentioned in https://bugs.webkit.org/show_bug.cgi?id=204106.

I would love to be able to capture better debug data for you youenn, but I don't know how. I hope at least this helps somewhat.

[youenn fablet]

There is also https://bugs.webkit.org/show_bug.cgi?id=208516 which sometimes will keep the microphone capture in muted state.

[youenn fablet]

On a side note, when capturing microphone, the webpage will not get suspended if in the background.
This allows to make sure networking and rendering will continue working.

If a phone call happens, the page microphone capture will be suspended, track will be muted and the web page is now suspendible, which will probably happen.
At that point, networking might start to fail.
When getting back to the page, the web page will get unsuspended but the network connection might be disconnected. It might be needed to restart it, maybe through iceRestart.
The application might be prepared to recover from that scenario.

[Brent Fulgham]

Can anyone confirm if this is still happening in iOS 15 or newer?