With VA decoders ranked to the max and the DMABuf sink turned on, videos sometimes flicker with frames from +/- a couple of seconds. This can be observed wherever videos can be played, including YouTube and Apple product pages. WebKitGTK 2.39.91, GStreamer 1.20.5, Epiphany Technology Preview 44.rc-19-ga80fcd0ab+
I am unable to attach a screencast as an attachment here (413), so I hosted it here instead http://kdwk.tplinkdns.com/index.php/s/bP8gHAsLym5Nbr6
I noticed this too, thanks for the bug report!
I think this happens because of the GstMemory being released (or rather disposed) for decoder's reuse before the sampling of dmabufs from such GstMemory is complete. The GstMemory is disposed as soon as the sample object is updated in MediaPlayerPrivateGStreamer::triggerRepaint(), releasing the last reference on the memory-containing buffer object. What I think is happening is the decoder can end up taking the released memory and decode into the associated dambuf even when the dmabuf is still being used for sampling on our side. The DMABufObject we use internally has an eventfd-based mechanism for indicating release from the presentation layer, meaning we have a pretty good idea of when a given dmabuf is not sampled anymore. When this is associated somehow with the GstMemory/GstBuffer lifetime, I think the flickering should go away.
This issue doesn't seem to occur when using the vaav1dec decoder. I tested this on my laptop with AV1 hardware decoding, then set YouTube to serve videos in AV1.
Ah nevermind. It's still using CPU decode
(In reply to kdwkleung from comment #5) > Ah nevermind. It's still using CPU decode HW AV1 decoding should work though, I guess you're the first to test this. Can you file a separate bug please?
I thought org.gnome.Platform doesn’t ship vaav1dec because it’s still on GStreamer 1.20… I’m not actually sure if it’s hardware-decoded or software-decoded anymore. Clapper the video player can hardware decode (it explicitly says vaav1dec) an AV1 video with about 2-3% CPU usage. The WebKitWebProcess, for a similar video, needs around 7% for (suspected) VA+DMABuf (but I’ve seen it do it at 2-3% at the same WebKitGTK+GStreamer version before so not sure what changed). This is much higher than confirmed hardware decoding but also significantly lower than confirmed software decoding — right about in the middle, so I’m very confused what’s happening.
Specifically, this is happening on YouTube. Videos on https://wpewebkit.org/blog/05-new-svg-engine.html still work as expected with 2-3% CPU usage. Additional information: after retesting the CPU usage seems to be 5% instead of 7%. The 5% figure doesn’t seem to scale with video resolution. 1080p24 is 5%, 4K60 still 5%. This is also only happening on AV1. VP9 works as expected at 2-3%. I should probably open a new bug report for this…
Created attachment 465676 [details] WIP patch Provisional idea. I was able to reproduce the problem, but not that consistently. With the patch I wasn't able to reproduce it, but not sure cause it wasn't common before then either. This is the dumb approach of storing references and dropping them upon new samples. It still has flaws since the is-released check is not necessarily consistent and can lead to retained GstMemory references, draining the decoder of available dmabuf targets for decoding.
This works quite well here already, I can confirm it works on my laptop, 4K VP9@60FPS with vavp9dec YT renders smoothly, I can't notice flickering anymore. https://www.youtube.com/watch?v=LXb3EKWsInQ
I'm on WebKitGTK 2.41.2, and it seems the issue can no longer be observed.
(In reply to kdwkleung from comment #11) > I'm on WebKitGTK 2.41.2, and it seems the issue can no longer be observed. Without this WIP patch you mean?
Indeed.
In 2.41 (so upcoming 2.42) the dmabuf sink is on by default btw, unless this env var is set: WEBKIT_GST_DMABUF_SINK_DISABLED=1
Here with 2.41, Ephy TP emits this warning: WebKit wasn't able to find the DMABuf video sink dependencies. Hardware-accelerated zero-copy video rendering won't be achievable with this p lugin. And I suspect the va decoders are no longer in use either...
Ah, so that's why CPU usage is through the roof again. I'll open another bug report
(In reply to Philippe Normand from comment #15) > Here with 2.41, Ephy TP emits this warning: > > WebKit wasn't able to find the DMABuf video sink dependencies. > Hardware-accelerated zero-copy video rendering won't be achievable with this > plugin. > > And I suspect the va decoders are no longer in use either... [📦 org.gnome.Epiphany.Devel ~]$ GST_REGISTRY= GST_DEBUG=va*:8 gst-inspect-1.0 /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstva.so 0:00:00.047993566 12 0x55ae7026d780 INFO vadisplay gstvadisplay_drm.c:154:gst_va_display_drm_create_va_display:<vadisplaydrm0> DRM render node with kernel driver i915 0:00:00.048022485 12 0x55ae7026d780 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: VA-API version 1.15.0 0:00:00.048034108 12 0x55ae7026d780 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so 0:00:00.048051344 12 0x55ae7026d780 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/iHD_drv_video.so 0:00:00.048857978 12 0x55ae7026d780 WARN vadisplay gstvadisplay.c:287:_va_warning:<vadisplaydrm0> VA error: /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/iHD_drv_video.so has no function __vaDriverInit_1_0 0:00:00.048867446 12 0x55ae7026d780 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/GL/lib/dri/iHD_drv_video.so 0:00:00.048890511 12 0x55ae7026d780 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: va_openDriver() returns -1 0:00:00.048898563 12 0x55ae7026d780 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so 0:00:00.048911836 12 0x55ae7026d780 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/i965_drv_video.so 0:00:00.049297301 12 0x55ae7026d780 WARN vadisplay gstvadisplay.c:287:_va_warning:<vadisplaydrm0> VA error: /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/i965_drv_video.so has no function __vaDriverInit_1_0 0:00:00.049304197 12 0x55ae7026d780 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/GL/lib/dri/i965_drv_video.so 0:00:00.049321443 12 0x55ae7026d780 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: va_openDriver() returns -1 0:00:00.049325583 12 0x55ae7026d780 WARN vadisplay gstvadisplay.c:347:gst_va_display_initialize:<vadisplaydrm0> vaInitialize: unknown libva error 0:00:00.370719171 11 0x56034ada7680 INFO vadisplay gstvadisplay_drm.c:154:gst_va_display_drm_create_va_display:<vadisplaydrm0> DRM render node with kernel driver i915 0:00:00.370764331 11 0x56034ada7680 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: VA-API version 1.15.0 0:00:00.370777935 11 0x56034ada7680 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so 0:00:00.370787742 11 0x56034ada7680 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/iHD_drv_video.so 0:00:00.372541070 11 0x56034ada7680 WARN vadisplay gstvadisplay.c:287:_va_warning:<vadisplaydrm0> VA error: /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/iHD_drv_video.so has no function __vaDriverInit_1_0 0:00:00.372548829 11 0x56034ada7680 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/GL/lib/dri/iHD_drv_video.so 0:00:00.372563074 11 0x56034ada7680 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: va_openDriver() returns -1 0:00:00.372569358 11 0x56034ada7680 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so 0:00:00.372576603 11 0x56034ada7680 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/i965_drv_video.so 0:00:00.373103367 11 0x56034ada7680 WARN vadisplay gstvadisplay.c:287:_va_warning:<vadisplaydrm0> VA error: /usr/lib/x86_64-linux-gnu/dri/intel-vaapi-driver/i965_drv_video.so has no function __vaDriverInit_1_0 0:00:00.373110688 11 0x56034ada7680 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: Trying to open /usr/lib/x86_64-linux-gnu/GL/lib/dri/i965_drv_video.so 0:00:00.373125681 11 0x56034ada7680 INFO vadisplay gstvadisplay.c:299:_va_info:<vadisplaydrm0> VA info: va_openDriver() returns -1 0:00:00.373142625 11 0x56034ada7680 WARN vadisplay gstvadisplay.c:347:gst_va_display_initialize:<vadisplaydrm0> vaInitialize: unknown libva error Plugin Details: Name va Description VA-API codecs plugin Filename /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstva.so Version 1.20.5 License LGPL Source module gst-plugins-bad Source release date 2022-12-19 Binary package GStreamer Bad Plug-ins source release Origin URL freedesktop-sdk 0 features: it seems that org.gnome.Epihany.Devel doesn't "talk" with org.freedesktop.Platform.VAAPI.Intel runtime.
That's https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/issues/1579 that requires a GNOME runtime re-build, which IIUC, is somehow, in-progress.
(In reply to Zan Dobersek from comment #9) > Created attachment 465676 [details] > WIP patch > > Provisional idea. I was able to reproduce the problem, but not that > consistently. With the patch I wasn't able to reproduce it, but not sure > cause it wasn't common before then either. > > This is the dumb approach of storing references and dropping them upon new > samples. It still has flaws since the is-released check is not necessarily > consistent and can lead to retained GstMemory references, draining the > decoder of available dmabuf targets for decoding. Have you observed this draining in practice?
Before this patch bitrots too much, can we mainline it? Back then I did check it was addressing the rendering issue reported here :)
This can still be observed in the latest WebKit main. Would really appreciate if the fix can be merged :)
Pull request: https://github.com/WebKit/WebKit/pull/14548
With this PR the vast majority of flickers are gone, including YouTube. However, one video on https://www.apple.com/airpods-3rd-generation/ still flickers (the video under the paragraph that starts with "A case full of energy.")
After full reloading the page for several more times, this seems to have gone away, but it seems some videos on apple.com product pages still flicker very occassionally
Videos on https://www.apple.com/mac-studio/ still flicker severely (sometimes)
When that happens do you get warnings or errors in the gst log? Checking "videodec*:6,va*:6" ?
Created attachment 466569 [details] GStreamer log of the problem still appearing after applying the patch Here are the requested logs. The problematic event should occur towards the end.
Created attachment 466570 [details] GStreamer logs with correct env var set Oops. Env var were not correctly set last time. Here is the correct log.
I'm getting a segfault in WebCore::MediaPlayerPrivateGStreamer::triggerRepaint with webkitgtk-2.41.5 when I try to play videos on Bilibili. I guess it's related to this issue but I'm not sure.
(In reply to Xi Ruoyao from comment #29) > I'm getting a segfault in > WebCore::MediaPlayerPrivateGStreamer::triggerRepaint with webkitgtk-2.41.5 > when I try to play videos on Bilibili. I guess it's related to this issue > but I'm not sure. Nope, my crash seems related to my custom user-agent setting and not this one.
(In reply to kdwkleung from comment #25) > Videos on https://www.apple.com/mac-studio/ still flicker severely > (sometimes) I can reproduce this indeed, enabling the native-video flag in playbin3 helps a bit but there's still a queue in playsink, holding a maximum of 3 buffers. I wonder if that could prevent memory recycling? Not sure...
I wonder if the patch can be landed as a partial fix? It already solves most of the most annoying flickering, especially on YouTube which would be completely unwatchable on my machine otherwise, and it doesn't cause any regressions...
The flickering on apple.com doesn't matter for the moment because the tab freezes due to MSE anyway
Committed 270971@main (2f3793730f22): <https://commits.webkit.org/270971@main> Reviewed commits have been landed. Closing PR #14548 and removing active labels.
This still happens... a bit less that before, but still.
Pull request: https://github.com/WebKit/WebKit/pull/29980
Committed 280210@main (9d5844679af8): <https://commits.webkit.org/280210@main> Reviewed commits have been landed. Closing PR #29980 and removing active labels.