RESOLVED FIXED 251017
[GStreamer] VA-API (GstVA) hardware-accelerated video decoding playback above 1080p resolution on YouTube in VP9 is very slow and stutters
https://bugs.webkit.org/show_bug.cgi?id=251017
Summary [GStreamer] VA-API (GstVA) hardware-accelerated video decoding playback above...
Jeff Fortin
Reported 2023-01-23 09:21:08 PST
Test case: playing https://www.youtube.com/watch?v=mBw7vkcQOew in 1080p and above resolutions ("stats for nerds" reveals it is playing using the VP9 codec on my computer) Here are the specs and parameters of my main testing computer (but it was observed on various other people's computers as well, particularly on HiDPI screens): * HP Spectre X360 laptop (15" 2018 model) with 4K HiDPI display, featuring an 8th generation (Kabylake) Intel Core i7 CPU and the associated intel graphics chipset, 16 GB of RAM and NVMe SSD * Running on the Wayland GNOME session on Fedora 37, with GNOME's night light & dark mode turned off (just to make sure this can't interfere) * Testing with the "technology preview" Epiphany Devel flatpak package from the gnome-nightly repository for consistency/simplicity --- ## Setup and reproduction instructions * On Fedora, it might be that the gstva plugin is part of this package (I'm not sure), so I did: `dnf install gstreamer1-plugins-bad-*-extras`. Make sure to also have `libva libva-utils libva-intel-driver` and `igt-gpu-tools` (to have the `intel_gpu_top` commandline utility available) * Grab Epiphany.Devel from gnome-nightly as a flatpak, and then run it like this (loosely based on https://blogs.igalia.com/vjaquez/2021/12/08/gstva-in-gstreamer-1-20/ ): `flatpak run --env=GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,vavp8dec:MAX,vavp9dec:MAX org.gnome.Epiphany.Devel` ...then try to play https://www.youtube.com/watch?v=mBw7vkcQOew at a resolution higher than 1080p. ## Results Regardless of dark/light mode (better to test in light mode to avoid conflating with bug #251016), attempting to play https://www.youtube.com/watch?v=mBw7vkcQOew in VP9 with VA-API through GstVa (for hardware that supports it, such as Intel Kabylake) at any resolution higher than 1080p will lag like hell, whether in HiDPI or not (but I could expect that the problem would probably be worse on a HiDPI screen, if previous bug reports are any indication).
Attachments
tentative gstreamer debug output printed to the terminal (612.46 KB, text/x-log)
2023-01-25 12:44 PST, Jeff Fortin
no flags
tentative gstreamer debug output printed to the terminal, prise deux (596.87 KB, text/x-log)
2023-01-25 12:56 PST, Jeff Fortin
no flags
debug output from the gst-play command (5.75 KB, application/x-xz)
2023-01-25 13:13 PST, Jeff Fortin
no flags
screenshot of video graphics corruption when playing with dmabuf enforced (386.91 KB, image/jpeg)
2023-01-25 13:37 PST, Jeff Fortin
no flags
Michael Catanzaro
Comment 1 2023-01-23 09:58:38 PST
(In reply to Jeff Fortin from comment #0) > * On Fedora, it might be that the gstva plugin is part of this package (I'm > not sure), so I did: `dnf install gstreamer1-plugins-bad-*-extras`. Make > sure to also have `libva libva-utils libva-intel-driver` and `igt-gpu-tools` > (to have the `intel_gpu_top` commandline utility available) You're using flatpak, so host system packages do not matter. Host system code is not going to be used at all (except for certain host services, not relevant here). > * Grab Epiphany.Devel from gnome-nightly as a flatpak, and then run it like > this (loosely based on > https://blogs.igalia.com/vjaquez/2021/12/08/gstva-in-gstreamer-1-20/ ): > > `flatpak run > --env=GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX, > vavp8dec:MAX,vavp9dec:MAX org.gnome.Epiphany.Devel` > > ...then try to play https://www.youtube.com/watch?v=mBw7vkcQOew at a > resolution higher than 1080p. You may also need to install a VAAPI flatpak extension. I'm told this happens by default for Intel but not for AMD users? Who knows about NVIDIA.
Víctor M. Jáquez L.
Comment 2 2023-01-23 10:15:57 PST
Is it possible to run MiniBrowser in Fedora? So you can test WebKit more or less isolated. I Debian I could run $ GST_PLUGIN_FEATURE_RANK=qsvh264dec:MAX,vah265dec:MAX,vavp8dec:MAX,vavp9dec:MAX,vampeg2dec:MAX /usr/lib/x86_64-linux-gnu/webkit2gtk-4.1/MiniBrowser https://www.youtube.com/watch?v=mBw7vkcQOew And it works, though I don't have a 4K display.
Patrick Griffis
Comment 3 2023-01-23 10:33:59 PST
(In reply to Michael Catanzaro from comment #1) > You may also need to install a VAAPI flatpak extension. I'm told this > happens by default for Intel but not for AMD users? Who knows about NVIDIA. AMD is installed by default. Nvidia is not packaged (https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/issues/1385)
Philippe Normand
Comment 4 2023-01-23 11:00:20 PST
I can reproduce this issue on my Intel laptop, in Ephy TP. vainfo: VA-API version: 1.16 (libva 2.16.0) vainfo: Driver version: Intel i965 driver for Intel(R) Coffee Lake - 2.4.1 vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264Main : VAEntrypointEncSliceLP VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileH264High : VAEntrypointEncSliceLP VAProfileH264MultiviewHigh : VAEntrypointVLD VAProfileH264MultiviewHigh : VAEntrypointEncSlice VAProfileH264StereoHigh : VAEntrypointVLD VAProfileH264StereoHigh : VAEntrypointEncSlice VAProfileVC1Simple : VAEntrypointVLD VAProfileVC1Main : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD VAProfileNone : VAEntrypointVideoProc VAProfileJPEGBaseline : VAEntrypointVLD VAProfileJPEGBaseline : VAEntrypointEncPicture VAProfileVP8Version0_3 : VAEntrypointVLD VAProfileVP8Version0_3 : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointEncSlice VAProfileHEVCMain10 : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointEncSlice VAProfileVP9Profile0 : VAEntrypointVLD VAProfileVP9Profile0 : VAEntrypointEncSlice VAProfileVP9Profile2 : VAEntrypointVLD
Philippe Normand
Comment 5 2023-01-23 11:08:44 PST
Might not be a WebKit bug, you can test this too flatpak run --command=bash org.gnome.Epiphany.Devel gst-play-1.0 --videosink glimagesink https://upload.wikimedia.org/wikipedia/commons/c/c0/Big_Buck_Bunny_4K.webm and watch the video sink drop frames and emit warnings, the video is encoded in VP8. On this old laptop I don't think 4K playback will ever work TBH.
Michael Catanzaro
Comment 6 2023-01-23 11:55:45 PST
(In reply to Víctor M. Jáquez L. from comment #2) > Is it possible to run MiniBrowser in Fedora? Since this is an Ephy Tech Preview bug, the host OS is not relevant. To run MiniBrowser, you can do: $ flatpak run --command=/usr/libexec/webkitgtk-6.0/MiniBrowser org.gnome.Epiphany.Devel But in this case, it will be easier to use --command=bash so that you can set environment variables more conveniently.
Michael Catanzaro
Comment 7 2023-01-23 11:58:47 PST
(In reply to Patrick Griffis from comment #3) > AMD is installed by default. Oh you're right, it's in the Mesa (Extra) extension which I do indeed have installed.
Jeff Fortin
Comment 8 2023-01-23 12:08:34 PST
> You may also need to install a VAAPI flatpak extension. [...] I forgot to mention more clearly that I was running intel_gpu_top at the same time, and saw that the video decoder was showing activity; anything other than 0.00% means hardware decoding acceleration is happening/working, so it is an indicator for me that the flatpaked Epiphany was indeed using the hardware. > On this old laptop I don't think 4K playback will ever work TBH. I don't see why it wouldn't work with 4K playback, if your laptop's chipset is indeed Coffee Lake, and advertises VP9 support etc… it's one generation newer than mine (Kaby Lake)! On mine, Firefox has zero problems playing this video with VP9 hardware acceleration on my test machine, in 4K (both the screen and the video); while it does so, intel_gpu_top reports about 30% "video" utilization, and htop reports about 15 to 35% CPU usage for each core.
Philippe Normand
Comment 9 2023-01-23 13:49:15 PST
> I don't see why it wouldn't work with 4K playback, if your laptop's chipset is indeed Coffee Lake, and advertises VP9 support etc… It's not un-common for hardware decoders to have bounds. For instance it's not un-realistic to expect an Intel GPU generation to decode H.264 main but then to bail out on 10bit profiles or things like that.
Víctor M. Jáquez L.
Comment 10 2023-01-23 21:55:19 PST
(In reply to Michael Catanzaro from comment #6) > (In reply to Víctor M. Jáquez L. from comment #2) > > Is it possible to run MiniBrowser in Fedora? > > Since this is an Ephy Tech Preview bug, the host OS is not relevant. To run > MiniBrowser, you can do: What I want to rule out are the flatpak/bubble/portal potential issues.
Jeff Fortin
Comment 11 2023-01-25 06:46:37 PST
Using unrud's VideoDownloader (see https://flathub.org/apps/details/com.github.unrud.VideoDownloader ...) to download the 4K VP9 version of any 4K sample on youtube (https://www.youtube.com/watch?v=OfNJro9jGdE is the one I tested now, but probably any other would yield the same result), giving it a simpler filename and playing it like so: GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,vavp8dec:MAX,vavp9dec:MAX gst-play-1.0 --videosink glimagesink 4k_hdr_sample.webm or, more succintly: GST_PLUGIN_FEATURE_RANK=vavp9dec:MAX gst-play-1.0 --videosink glimagesink 4k_hdr_sample.webm ...results in 100% smooth video playback on this Intel Kabylake laptop with 4K screen, where intel_gpu_top shows the video decoder is correctly used. If I don't prefix it with the GST_PLUGIN_FEATURE_RANK env var, then it doesn't use the hardware decoder at all (as indicated by intel_gpu_top) and is unplayable, it can't keep up at all. If I don't use `--videosink glimagesink`, then it does use the hardware decoder but playback is not butter-smooth (it can keep up, though).
Philippe Normand
Comment 12 2023-01-25 07:13:34 PST
Cool cool. So next step would be to compare the pipeline graphs and see what formats is negotiated in the decoder source pad and what are the elements downstream of it. I guess there's a difference and the caps negotiated in WebKit trigger a bottleneck for rendering.
Michael Catanzaro
Comment 13 2023-01-25 09:22:52 PST
(In reply to Philippe Normand from comment #12) > Cool cool. So next step would be to compare the pipeline graphs and see what > formats is negotiated in the decoder source pad and what are the elements > downstream of it. What steps does Jeff need to do here?
Philippe Normand
Comment 15 2023-01-25 11:49:02 PST
(In reply to Jeff Fortin from comment #11) > Using unrud's VideoDownloader (see > https://flathub.org/apps/details/com.github.unrud.VideoDownloader ...) to > download the 4K VP9 version of any 4K sample on youtube > (https://www.youtube.com/watch?v=OfNJro9jGdE is the one I tested now, but > probably any other would yield the same result), giving it a simpler > filename and playing it like so: > > GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,vavp8dec: > MAX,vavp9dec:MAX gst-play-1.0 --videosink glimagesink 4k_hdr_sample.webm > > or, more succintly: > > GST_PLUGIN_FEATURE_RANK=vavp9dec:MAX gst-play-1.0 --videosink glimagesink > 4k_hdr_sample.webm > > ...results in 100% smooth video playback on this Intel Kabylake laptop with > 4K screen, where intel_gpu_top shows the video decoder is correctly used. > Here on my "recent according to you :)" laptop with no 4K monitor, the sink drops frames like there is no tomorrow.
Jeff Fortin
Comment 16 2023-01-25 12:44:18 PST
Created attachment 464651 [details] tentative gstreamer debug output printed to the terminal I've been utterly unable, no matter what commands are used, to get debug logs to be saved to files, but in case this is useful, this is the stuff that gets printed to the terminal when running with GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,vavp8dec:MAX,vavp9dec:MAX GST_DEBUG="3,webkit*:6" GST_DEBUG_FILE=/home/my_user/Downloads/foobar/gst.log GST_DEBUG_DUMP_DOT_DIR=/home/my_user/Downloads/foobar
Jeff Fortin
Comment 17 2023-01-25 12:56:51 PST
Created attachment 464653 [details] tentative gstreamer debug output printed to the terminal, prise deux
Jeff Fortin
Comment 18 2023-01-25 13:13:00 PST
Created attachment 464655 [details] debug output from the gst-play command Unlike the "GStreamer + flatpaked Epiphany/WebKitGTK" testing nightmare where it is impossible to get logs to be written to disk, here are the logs / dot file from this command: GST_PLUGIN_FEATURE_RANK=vavp9dec:MAX GST_DEBUG="3,webkit*:6" GST_DEBUG_FILE=/home/my_user/Downloads/foobar_gstplay/gst.log GST_DEBUG_DUMP_DOT_DIR=/home/my_user/Downloads/foobar_gstplay gst-play-1.0 --videosink glimagesink 4k_hdr_sample.webm
Philippe Normand
Comment 19 2023-01-25 13:21:39 PST
So glimagesink negotiates dmabuf with the decoder. Our GL sink doesn't support this but there's some experimental dmabuf sink, opt-in, through the WEBKIT_GST_DMABUF_SINK_ENABLED=1 env var. Can you try?
Jeff Fortin
Comment 20 2023-01-25 13:37:33 PST
Created attachment 464656 [details] screenshot of video graphics corruption when playing with dmabuf enforced Running with WEBKIT_GST_DMABUF_SINK_ENABLED=1 (in addition to all the other gazillion env vars mentioned previously) results in total graphics corruption of the video; that said however, looking at the motion of the corrupt graphics, I can tell it's playing smoothly, at 30 fps at least, and the CPU usage is low (maybe 10-15% per core or so, and intel_gpu_top still shows "video" having non-zero percentage), so it's "working". Unforunate that we have so far been utterly unable to get gst debug output and dotfiles from this specific combo of gstreamer + webkitgtk + epiphany + flatpak. I will await further instructions, I will need an exact foolproof command line for it. Attached here is a screenshot of what the corrupt video output looks like; note that I resized the screenshot down to 1500px, because it otherwise weighted a ton due to the 4K screen resolution.
Philippe Normand
Comment 21 2023-01-25 13:40:44 PST
+Zan Can you reproduce this issue with the dmabuf sink? Looks like a stride issue?
Philippe Normand
Comment 22 2023-02-17 06:25:07 PST
*** Bug 251679 has been marked as a duplicate of this bug. ***
Philippe Normand
Comment 23 2023-02-22 09:26:30 PST
In bug 252744 Zan has a patch fixing the dmabuf sink render issue.
Philippe Normand
Comment 24 2023-03-12 01:32:43 PST
The rendering issue is fixed now. Can you test again? With GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,vavp8dec:MAX,vavp9dec:MAX WEBKIT_GST_DMABUF_SINK_ENABLED=1
Kdwk
Comment 25 2023-03-12 17:56:36 PDT
I am no longer able to reproduce this issue
Philippe Normand
Comment 26 2023-03-13 09:41:43 PDT
And Jeff?
Philippe Normand
Comment 27 2023-03-16 05:15:13 PDT
Jeff, in case the issue still happens for you feel free to reopen.
Note You need to log in before you can comment on or make changes to this bug.