Bug 230922 - REGRESSION (iOS 15): Autoplayed video element with mediaStream srcObject freezes
Summary: REGRESSION (iOS 15): Autoplayed video element with mediaStream srcObject freezes
Status: RESOLVED CONFIGURATION CHANGED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: Other
Hardware: iPhone / iPad Other
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-09-28 16:05 PDT by Alec Bargas
Modified: 2022-06-04 02:50 PDT (History)
17 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alec Bargas 2021-09-28 16:05:09 PDT
A video element with the 'autoplay + playsinline + muted' property set with a mediaStream srcObject can freeze. This freeze can either occur before the video starts playing (so the background is seen) or a single frame is frozen. The expectation is that the video should play continuously without freezing.

This behavior can be seen on the https://webrtc.github.io/samples/src/content/capture/canvas-video/ sample page.
The steps to reproduce the bug are:
1. Navigate to page.
2. Drag on the canvas (first teapot if two exist) and observe.
3a. If after dragging, a second teapot (appears if one didn't exist before and) moves, then the issue did not reproduce (move on to 4a).
3b. If after dragging a second teapot did not move, then the issue was reproduced and no need to follow the next steps.
4a. Close the tab, kill Safari, and relaunch Safari.
5a. From history, select the sample page.
6a. Drag on the canvas and observe, the issue should be seen.

When this happens, the media stream and tracks are still active and the video element is not paused. If the video is unmuted and the stream includes audio, the audio will still play. If using an iPad, putting Safari into split screen updates the video to what looks like the most recent frame, but the video still does not update continuously. On either iPad or iPhone, backgrounding Safari then coming back will allow the video to update continuously.

Reproduces with:
Version/15.0 Safari/605.1.15 (iOS 15.0)
Version/15.0 Safari/605.1.15 (iPadOS 15.1)

Does not reproduce with:
Prior to iOS 15 beta 8
Version/14.1.2 Safari/605.1.15 (macOS Big Sur 11.6)
Comment 1 Radar WebKit Bug Importer 2021-09-29 12:53:26 PDT
<rdar://problem/83684099>
Comment 2 Will Morgan 2021-10-01 10:20:49 PDT
This regression is affecting a number of our users and is blocking them from advancing through several important user journeys - authenticating to banks and healthcare etc. Could this be looked at urgently?
Comment 3 Mark B 2021-10-04 04:20:19 PDT
This issue is affecting users of our web application in production. We would appreciate feedback on the planned resolution and timeline.
Comment 4 Nils R 2021-10-12 07:08:48 PDT
I can confirm and reproduce the bug.
It is also affecting our customers.

Feedback would be appreciate.
Comment 5 youenn fablet 2021-10-12 07:58:53 PDT
There is a known regression in canvas capture streams that is being worked on.
I am unclear though whether that bug is specific to canvas capture stream or also applies to camera and/or peer connection media streams.
Can anybody of you clarify?
Comment 6 Will Morgan 2021-10-12 08:00:09 PDT
Hi Youenn, our application makes use of getUserMedia and the video element. We don't use CanvasCaptureMediaStreamTrack et al, at all.

Hopefully that clarifies?
Comment 7 Nils R 2021-10-12 08:14:27 PDT
In our case the we use peer connection media streams and add those to a srcObject to a videoElement. It displays nothing as in the webrtc sample.
Comment 8 youenn fablet 2021-10-12 08:38:27 PDT
Thanks.
Would you have repro cases? 
https://webrtc.github.io/samples/src/content/peerconnection/pc1/ should hopefully work fine.

Or you can send privately to me sysdiagnoses (youenn@apple.com).
Comment 9 Chris Mo 2021-10-15 00:53:59 PDT
Our application have same issue start from Safari Technology Preview 126 or above.
Comment 10 youenn fablet 2021-10-22 04:55:24 PDT
https://webrtc.github.io/samples/src/content/capture/canvas-video/ has an issue with canvas capture so I do not think it is good to consider it for .
Does any of you have a different repro case that I can use, not based on canvas capture?
Comment 11 Kongpheng 2021-10-25 18:34:04 PDT
Youenn, using the sample repo you mentioned, it's possible to see video not auto-resume as expected.  Granted this isn't the same as freezing, but just wanted to share, in case it was helpful:
* Open https://webrtc.github.io/samples/src/content/peerconnection/pc1/
* Click Start button
* Grant access to microphone and camera
* Notice video is captured
* Make a call to the iPhone
* From Call UI dialog, decline it
* Notice that video doesn't get auto-resumed, instead you see a black rectangle where video used to be

This might be more similar to https://bugs.webkit.org/show_bug.cgi?id=231365 though.
Comment 12 Charlie 2021-11-04 14:58:38 PDT
I'm able to reproduce using Youenn's sample repo. I added 'muted' property on the remote video. See it here https://charliesantos.github.io/samples/src/content/peerconnection/pc1/

Steps:
1. Load https://charliesantos.github.io/samples/src/content/peerconnection/pc1/
2. Press start then call button, grant access to the mic/camera
3. Make sure the video is captured
4. Make a phone call to the iPhone. Have it ring for about 10-15 seconds
5. Swipe up the call notification to go back to the page
6. Press Hang Up button, or interact to the page. Notice that the whole page is frozen.
Comment 13 youenn fablet 2021-11-08 00:30:09 PST
@Charlie, the freeze should be fixed in latest iOS 15.2 beta.
@Kongpheng, I think this is also the case in latest iOS 15.2 beta, can you validate and if not file another bug?
@Alec, the issue with the tea pot is related to canvas capture and is being worked on. Latest Safari Tech Preview should be working. Please reopen if that is not the case.
Comment 14 Yevhen Lv 2022-02-07 04:47:56 PST
Hi. iOS 15.2 has same problem. Using Webrtc without canvas, only simple video.
Chrome, Safari, Firefox with started video call freezes after incoming call.

Black screen instead video and page doesn't react on touches. Sometimes page reload helps, but usually I should close chrome and open it again.
Comment 15 PhungDuong 2022-03-25 00:48:28 PDT
For this black camera error, we can sometimes reproduce it on iOS 15.3 and 15.4 on my WebRTC product.
Feedback would be appreciated.
Comment 16 PhungDuong 2022-06-04 02:36:22 PDT
Hi @youenn,

We can confirm and reproduce this bug on iOS/iPadOS 15.4.1
Here are steps to reproduce:
1. Open the link using webRTC (https://it.dev-akameet.link/video/room/videocall)
2. Allow micro and camera.
3. Click the button to join the meeting.
Expected: The local video is normal.
Actual: The local video is black.

Note: From step 2 to step 3 do with speed as fast as possible. When you can't repo it, close the safari, open it again, and repeat the steps above, because the frequency isn't 100%.
Comment 17 PhungDuong 2022-06-04 02:50:19 PDT
(In reply to PhungDuong from comment #16)
> Hi @youenn,
> 
> We can confirm and reproduce this bug on iOS/iPadOS 15.4.1
> Here are steps to reproduce:
> 1. Open the link using webRTC
> (https://it.dev-akameet.link/video/room/videocall)
> 2. Allow micro and camera.
> 3. Click the button to join the meeting.
> Expected: The local video is normal.
> Actual: The local video is black.
> 
> Note: From step 2 to step 3 do with speed as fast as possible. When you
> can't repo it, close the safari, open it again, and repeat the steps above,
> because the frequency isn't 100%.

Also can repo this on iOS/iPadOS 15.5