Bug 230532 - WebRTC/video rendering flaky
Summary: WebRTC/video rendering flaky
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: Other
Hardware: All Other
: P2 Blocker
Assignee: Nobody
Keywords: InRadar
Depends on:
Reported: 2021-09-21 00:45 PDT by David Zentgraf
Modified: 2024-04-25 23:59 PDT (History)
9 users (show)

See Also:

video demonstrating video not being rendered until relayout is triggered by popup (447.42 KB, video/mp4)
2021-09-21 00:45 PDT, David Zentgraf
no flags Details
flaky rendering on desktop Safari 15 (179.63 KB, image/png)
2021-09-21 07:31 PDT, David Zentgraf
no flags Details
Video demonstrating rendering flakiness (7.89 MB, video/mp4)
2021-11-09 01:46 PST, David Zentgraf
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Zentgraf 2021-09-21 00:45:07 PDT
Created attachment 438789 [details]
video demonstrating video not being rendered until relayout is triggered by popup

(WebRTC) video is not being rendered sometimes. The problem started appearing in the latest iOS 15 RC (betas were fine according to our testing). Same was found in Safari Technology Preview 15.4. In the latter on desktop, the problem can be "fixed" by slightly resizing the browser window to trigger a re-layout. On iOS that's obviously not possible. See attached video for illustration of the issue. There should be a video as soon as the yellow rectangle appears, but it doesn't render. When triggering an overlay, it suddenly appears.

I don't have an easily reproducible test code, as you can see it's a complex app. Happens in both plain Safari and in-app Safari controllers.
Comment 1 David Zentgraf 2021-09-21 07:31:31 PDT
Created attachment 438815 [details]
flaky rendering on desktop Safari 15

In fact, rendering is flaky in general in Safari 15, not just limited to iOS. This demonstrates probably the same bug in desktop Safari 15 on Stack Overflow. Clicking somewhere updates the missing parts, just like the popup in the original video shows.
Comment 2 Radar WebKit Bug Importer 2021-09-21 08:55:45 PDT
Comment 3 Davy De Durpel 2021-11-04 03:29:03 PDT
Issue is still there in iOS 15.2 Public Beta 1
Comment 4 youenn fablet 2021-11-04 08:46:39 PDT
Is there a way to reproduce the issue? Is it always happening or flakily happening?
Comment 5 David Zentgraf 2021-11-04 08:58:03 PDT
There's no simple way to provide a reproducible example. The attached Stack Overflow example happens more, and rather frequently, on external monitors connected via a dock using DisplayLink (http://displaylink.com). This never happened in Safari 14-. But it doesn't always happen and it doesn't happen consistently.

In the Stack Overflow example, the page above the fold loads fine, but when clicking any element, nothing appears to happen. Dragging the window onto another screen causes the clicked action to happen (e.g. the panel I clicked on is now expanded). Or you see the same thing when scrolling, as shown in the screenshot; it scrolls, but does not render the new content that has scrolled into view, until dragged to a different monitor. It appears as if Safari thinks it's not on screen and stops rendering new content, or something along those lines.

It doesn't happen on all websites; some sites never exhibit this behaviour, and sites that do don't *always* exhibit this behaviour. It also can't be pinned down to a particular screen; sometimes it's the one, sometimes the other, and after a while it starts working just fine on both.
Comment 6 youenn fablet 2021-11-04 09:25:55 PDT
StackOverflow issue might be a different code path, I am not sure this is related to the camera rendering code path.
It would be good to have a repro case, even if it does not repro consistently.
Comment 7 David Zentgraf 2021-11-09 01:46:16 PST
Created attachment 443660 [details]
Video demonstrating rendering flakiness

This demonstrates very flaky rendering and interaction. I'm clicking outside the window there to unfocus and re-focus it, which temporary restores correct rendering.
Comment 8 David Zentgraf 2021-11-09 01:50:16 PST
Believe me I'd love to provide a simple test case, but I don't have the time to try to produce some minimal website which triggers this behaviour. It happens on popular sites like SO which are publicly accessible, so I'll point there as a test case. I've now observed this behaviour also on the builtin screen as well (not external DisplayLink displays), so I'll discount that as a factor.

MacBook Pro 15'' 2016
2.6 GHz i7
16 GB
Radeon Pro 460 4 GB
macOS Monterey 12.0.1
Comment 9 youenn fablet 2021-11-09 03:28:06 PST
Thanks David. According the video, this rendering issue is not specifically related to WebRTC video but to general pages.
Let's see whether rendering folks have some thoughts on this.
Comment 10 rychouwei 2022-01-20 04:27:38 PST
We have same issue, the video does not render the mediaStream from getUserMedia, after page relayout, the video start rendering
Comment 11 youenn fablet 2022-01-24 08:31:49 PST
This might be fixed with https://bugs.webkit.org/show_bug.cgi?id=230623
Comment 12 komo21 2023-08-01 02:29:49 PDT
It's still the same on iOS 16, the screen rotates, triggers a reflow, and then the rendered video blacks out. It looks like even the tabbing process is hanging. Temporary processing thing of recreating the player after rotation circumvented it.
Comment 13 phoebe 2024-01-26 02:21:41 PST
It's still the same on iOS 17. The video render the mediaStream from getUserMedia, and
the video style width and height change, causing the picture to be re-rendered and flicker.
Comment 14 phoebe 2024-01-26 02:22:32 PST
It's still the same on iOS 17. The video render the mediaStream from getUserMedia, and the video style width and height change, causing the picture to be re-rendered and flicker.
Comment 15 youenn fablet 2024-01-30 01:51:08 PST
This should be fixed in iOS 17.3.
Please reopen otherwise.
Comment 16 rychouwei 2024-04-25 23:59:37 PDT
Hi youenn, this issue is able to reproduce in iOS 17.3 when playing the remote video, and works fine in iOS 17.4.1.

Can u help to confirm the fixed iOS version again?