Bug 219318 – Black background on video element after an incoming phone call

• Component: WebRTC
• Status: NEW

[German Goldenstein]

Created attachment 414963 [details]
Screenshot - no video

Hello! I see a bug in iOS Safari 14, that shows a black background on a video element after I receive a non-intrusive phone call.


Steps to reproduce it.

1) Join a session in app.rtc from iOS Safari
2) Join the same session from another browser
3) once Safari is publishing video and receiving video from the other peer, stop all the local tracks.
   You can do this by running this line on the console: 
   ```
   document.querySelectorAll('video')[0].srcObject.getTracks().forEach(t => t.stop())
   ```
   For this, you will need to connect your iphone to your mac and use the remote safari console.
   Make sure you only see the remote video and audio.
4) Now call your phone and reject the call.
5) The video now shows a black background. The video and audio tracks are enabled and not muted, and the video element is not paused. See attached screenshot.



Could you please advise if there's a workaround? 
Thanks!!

[Radar WebKit Bug Importer]

<rdar://problem/71788766>

[youenn fablet]

The video element is probably paused by the video call.
You can check this by looking at video.paused.

In appr.tc, the remote video is being autoplayed.
The phone call is suspending playing the remote video.
After the phone call, the video playing does not restart, probably as autoplay does not kick in anymore since there is no local capture.

A potential workaround is to call video.play() as part of a user gesture.
This will remove the autoplay restriction and make sure the video element will restart playing after the phone call.

We could try to improve our autoplay policy.

[German Goldenstein]

Thanks.

Yes, the video is paused in apprtc.


I see 2 issues here:
1) You need to ask a user gesture to be able to call play().
2) If at the moment of the incoming call the local camera is being used (that is ignoring the step #3 of the repro steps), when the call is rejected everything looks good. So, IMHO I think it should always work with the same behavior, that is, after the income call is rejected, the remote video should work without workarounds.

[youenn fablet]

> 1) You need to ask a user gesture to be able to call play().

This is the general rule to play audio.
There is an exception when the camera/mic does not have access.

If you start without camera/mic, the video will need a user gesture.
In that case, after the phone call happens, I believe the video will restart playing.
The issue is probably if the video autoplayed without a user gesture in the first place.

> 2) If at the moment of the incoming call the local camera is being used
> (that is ignoring the step #3 of the repro steps), when the call is rejected
> everything looks good.

Right, this case though is the usual one.
The one you describe (start with camera, then stop camera then phone call) is much more rare.

> So, IMHO I think it should always work with the same
> behavior, that is, after the income call is rejected, the remote video
> should work without workarounds.

That is a possibility we should probably dig in.
I agree it is inconvenient that the video gets paused without the application being able to notice it easily. The intent is probably that when resuming, we get to the same state as previously, and this is not the case here.

[German Goldenstein]

Thanks!

BTW, about the rare case about stoping the local tracks IMHO is not that rare.

It's common to just subscribe a stream without publishing the local stream, for example, a broadcasting app where you only play the remote video without using the local camera.

I needed to stop the tracks like that because the way apprtc is built you always send video from the beginning of the call. 

Should we expect a fix for this issue then?

[youenn fablet]

(In reply to German Goldenstein from comment #5)
> Thanks!
> 
> BTW, about the rare case about stoping the local tracks IMHO is not that
> rare.
> 
> It's common to just subscribe a stream without publishing the local stream,
> for example, a broadcasting app where you only play the remote video without
> using the local camera.

In that case, you have to start playing the video element with a user gesture.
The video will then resume playing automatically after the phone call.

> Should we expect a fix for this issue then?

Need to look at exactly how this would be done and what would be the changes for other potential edge cases.

[youenn fablet]

The workaround as identified in https://bugs.webkit.org/show_bug.cgi?id=209594#c3, is to check for the pause event.
While we try to improve our heuristics, there are some alternatives that might help:
1. Make sure to call play() for each video element as part of click event. This will remove the autoplay restrictions. Suspend/resume should then work fine.

2. Register a pause event handler. When pause is set and play rejects, mute the media elements and call play. This should allow seeing the video tracks. Add some UI to ask users to play audio/unmute audio. User will click, you can then unmute the elements/call play on all elements.

3. For multi party calls where capture is not on, you might want to keep a buffer of html media elements. On click event, call play on all of these media elements. They do not need to be set to a given src. When you have some media stream to attach, set srcObject and you should be good.