Bug 251017 – [GStreamer] VA-API (GstVA) hardware-accelerated video decoding playback above 1080p resolution on YouTube in VP9 is very slow and stutters

• Component: WebKitGTK
• Status: RESOLVED FIXED

[Jeff Fortin]

Test case: playing https://www.youtube.com/watch?v=mBw7vkcQOew in 1080p and above resolutions ("stats for nerds" reveals it is playing using the VP9 codec on my computer)

Here are the specs and parameters of my main testing computer (but it was observed on various other people's computers as well, particularly on HiDPI screens):

* HP Spectre X360 laptop (15" 2018 model) with 4K HiDPI display, featuring an 8th generation (Kabylake) Intel Core i7 CPU and the associated intel graphics chipset, 16 GB of RAM and NVMe SSD
* Running on the Wayland GNOME session on Fedora 37, with GNOME's night light & dark mode turned off (just to make sure this can't interfere)
* Testing with the "technology preview" Epiphany Devel flatpak package from the gnome-nightly repository for consistency/simplicity

---

## Setup and reproduction instructions

* On Fedora, it might be that the gstva plugin is part of this package (I'm not sure), so I did: `dnf install gstreamer1-plugins-bad-*-extras`. Make sure to also have `libva libva-utils libva-intel-driver` and `igt-gpu-tools` (to have the `intel_gpu_top` commandline utility available)
* Grab Epiphany.Devel from gnome-nightly as a flatpak, and then run it like this (loosely based on https://blogs.igalia.com/vjaquez/2021/12/08/gstva-in-gstreamer-1-20/ ):

`flatpak run --env=GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,vavp8dec:MAX,vavp9dec:MAX org.gnome.Epiphany.Devel`

...then try to play https://www.youtube.com/watch?v=mBw7vkcQOew at a resolution higher than 1080p.

## Results

Regardless of dark/light mode (better to test in light mode to avoid conflating with bug #251016), attempting to play https://www.youtube.com/watch?v=mBw7vkcQOew in VP9 with VA-API through GstVa (for hardware that supports it, such as Intel Kabylake) at any resolution higher than 1080p will lag like hell, whether in HiDPI or not (but I could expect that the problem would probably be worse on a HiDPI screen, if previous bug reports are any indication).

[Michael Catanzaro]

(In reply to Jeff Fortin from comment #0)
> * On Fedora, it might be that the gstva plugin is part of this package (I'm
> not sure), so I did: `dnf install gstreamer1-plugins-bad-*-extras`. Make
> sure to also have `libva libva-utils libva-intel-driver` and `igt-gpu-tools`
> (to have the `intel_gpu_top` commandline utility available)

You're using flatpak, so host system packages do not matter. Host system code is not going to be used at all (except for certain host services, not relevant here).

> * Grab Epiphany.Devel from gnome-nightly as a flatpak, and then run it like
> this (loosely based on
> https://blogs.igalia.com/vjaquez/2021/12/08/gstva-in-gstreamer-1-20/ ):
> 
> `flatpak run
> --env=GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,
> vavp8dec:MAX,vavp9dec:MAX org.gnome.Epiphany.Devel`
> 
> ...then try to play https://www.youtube.com/watch?v=mBw7vkcQOew at a
> resolution higher than 1080p.

You may also need to install a VAAPI flatpak extension. I'm told this happens by default for Intel but not for AMD users? Who knows about NVIDIA.

[Víctor M. Jáquez L.]

Is it possible to run MiniBrowser in Fedora?

So you can test WebKit more or less isolated. 

I Debian I could run 

$ GST_PLUGIN_FEATURE_RANK=qsvh264dec:MAX,vah265dec:MAX,vavp8dec:MAX,vavp9dec:MAX,vampeg2dec:MAX /usr/lib/x86_64-linux-gnu/webkit2gtk-4.1/MiniBrowser https://www.youtube.com/watch?v=mBw7vkcQOew

And it works, though I don't have a 4K display.

[Patrick Griffis]

(In reply to Michael Catanzaro from comment #1)
> You may also need to install a VAAPI flatpak extension. I'm told this
> happens by default for Intel but not for AMD users? Who knows about NVIDIA.

AMD is installed by default. Nvidia is not packaged (https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/issues/1385)

[Philippe Normand]

I can reproduce this issue on my Intel laptop, in Ephy TP.

vainfo: VA-API version: 1.16 (libva 2.16.0)
vainfo: Driver version: Intel i965 driver for Intel(R) Coffee Lake - 2.4.1
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Simple            :	VAEntrypointEncSlice
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSliceLP
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointEncSliceLP
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointEncSliceLP
      VAProfileH264MultiviewHigh      :	VAEntrypointVLD
      VAProfileH264MultiviewHigh      :	VAEntrypointEncSlice
      VAProfileH264StereoHigh         :	VAEntrypointVLD
      VAProfileH264StereoHigh         :	VAEntrypointEncSlice
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileJPEGBaseline           :	VAEntrypointEncPicture
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileVP8Version0_3          :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSlice
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointEncSlice
      VAProfileVP9Profile2            :	VAEntrypointVLD

[Philippe Normand]

Might not be a WebKit bug, you can test this too

flatpak run  --command=bash org.gnome.Epiphany.Devel

gst-play-1.0 --videosink glimagesink https://upload.wikimedia.org/wikipedia/commons/c/c0/Big_Buck_Bunny_4K.webm

and watch the video sink drop frames and emit warnings, the video is encoded in VP8.

On this old laptop I don't think 4K playback will ever work TBH.

[Michael Catanzaro]

(In reply to Víctor M. Jáquez L. from comment #2)
> Is it possible to run MiniBrowser in Fedora?

Since this is an Ephy Tech Preview bug, the host OS is not relevant. To run MiniBrowser, you can do:

$ flatpak run --command=/usr/libexec/webkitgtk-6.0/MiniBrowser org.gnome.Epiphany.Devel

But in this case, it will be easier to use --command=bash so that you can set environment variables more conveniently.

[Michael Catanzaro]

(In reply to Patrick Griffis from comment #3)
> AMD is installed by default.

Oh you're right, it's in the Mesa (Extra) extension which I do indeed have installed.

[Jeff Fortin]

> You may also need to install a VAAPI flatpak extension. [...]

I forgot to mention more clearly that I was running intel_gpu_top at the same time, and saw that the video decoder was showing activity; anything other than 0.00% means hardware decoding acceleration is happening/working, so it is an indicator for me that the flatpaked Epiphany was indeed using the hardware.

> On this old laptop I don't think 4K playback will ever work TBH.

I don't see why it wouldn't work with 4K playback, if your laptop's chipset is indeed Coffee Lake, and advertises VP9 support etc… it's one generation newer than mine (Kaby Lake)!

On mine, Firefox has zero problems playing this video with VP9 hardware acceleration on my test machine, in 4K (both the screen and the video); while it does so, intel_gpu_top reports about 30% "video" utilization, and htop reports about 15 to 35% CPU usage for each core.

[Philippe Normand]

> I don't see why it wouldn't work with 4K playback, if your laptop's chipset is indeed Coffee Lake, and advertises VP9 support etc…

It's not un-common for hardware decoders to have bounds. For instance it's not un-realistic to expect an Intel GPU generation to decode H.264 main but then to bail out on 10bit profiles or things like that.

[Víctor M. Jáquez L.]

(In reply to Michael Catanzaro from comment #6)
> (In reply to Víctor M. Jáquez L. from comment #2)
> > Is it possible to run MiniBrowser in Fedora?
> 
> Since this is an Ephy Tech Preview bug, the host OS is not relevant. To run
> MiniBrowser, you can do:

What I want to rule out are the flatpak/bubble/portal potential issues.

[Jeff Fortin]

Using unrud's VideoDownloader (see https://flathub.org/apps/details/com.github.unrud.VideoDownloader ...) to download the 4K VP9 version of any 4K sample on youtube (https://www.youtube.com/watch?v=OfNJro9jGdE is the one I tested now, but probably any other would yield the same result), giving it a simpler filename and playing it like so:

GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,vavp8dec:MAX,vavp9dec:MAX gst-play-1.0 --videosink glimagesink 4k_hdr_sample.webm

or, more succintly:

GST_PLUGIN_FEATURE_RANK=vavp9dec:MAX gst-play-1.0 --videosink glimagesink 4k_hdr_sample.webm

...results in 100% smooth video playback on this Intel Kabylake laptop with 4K screen, where intel_gpu_top shows the video decoder is correctly used.

If I don't prefix it with the GST_PLUGIN_FEATURE_RANK env var, then it doesn't use the hardware decoder at all (as indicated by intel_gpu_top) and is unplayable, it can't keep up at all.

If I don't use `--videosink glimagesink`, then it does use the hardware decoder but playback is not butter-smooth (it can keep up, though).

[Philippe Normand]

Cool cool. So next step would be to compare the pipeline graphs and see what formats is negotiated in the decoder source pad and what are the elements downstream of it.

I guess there's a difference and the caps negotiated in WebKit trigger a bottleneck for rendering.

[Michael Catanzaro]

(In reply to Philippe Normand from comment #12)
> Cool cool. So next step would be to compare the pipeline graphs and see what
> formats is negotiated in the decoder source pad and what are the elements
> downstream of it.

What steps does Jeff need to do here?

[Philippe Normand]

https://github.com/Igalia/meta-webkit/wiki/Providing-useful-GStreamer-Zero-copy-issue-reports#in-webkit

[Philippe Normand]

(In reply to Jeff Fortin from comment #11)
> Using unrud's VideoDownloader (see
> https://flathub.org/apps/details/com.github.unrud.VideoDownloader ...) to
> download the 4K VP9 version of any 4K sample on youtube
> (https://www.youtube.com/watch?v=OfNJro9jGdE is the one I tested now, but
> probably any other would yield the same result), giving it a simpler
> filename and playing it like so:
> 
> GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,vavp8dec:
> MAX,vavp9dec:MAX gst-play-1.0 --videosink glimagesink 4k_hdr_sample.webm
> 
> or, more succintly:
> 
> GST_PLUGIN_FEATURE_RANK=vavp9dec:MAX gst-play-1.0 --videosink glimagesink
> 4k_hdr_sample.webm
> 
> ...results in 100% smooth video playback on this Intel Kabylake laptop with
> 4K screen, where intel_gpu_top shows the video decoder is correctly used.
> 

Here on my "recent according to you :)" laptop with no 4K monitor, the sink drops frames like there is no tomorrow.

[Jeff Fortin]

Created attachment 464651 [details]
tentative gstreamer debug output printed to the terminal

I've been utterly unable, no matter what commands are used, to get debug logs to be saved to files, but in case this is useful, this is the stuff that gets printed to the terminal when running with 

GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,vavp8dec:MAX,vavp9dec:MAX GST_DEBUG="3,webkit*:6" GST_DEBUG_FILE=/home/my_user/Downloads/foobar/gst.log GST_DEBUG_DUMP_DOT_DIR=/home/my_user/Downloads/foobar

[Jeff Fortin]

Created attachment 464653 [details]
tentative gstreamer debug output printed to the terminal, prise deux

[Jeff Fortin]

Created attachment 464655 [details]
debug output from the gst-play command

Unlike the "GStreamer + flatpaked Epiphany/WebKitGTK" testing nightmare where it is impossible to get logs to be written to disk, here are the logs / dot file from this command:

GST_PLUGIN_FEATURE_RANK=vavp9dec:MAX  GST_DEBUG="3,webkit*:6" GST_DEBUG_FILE=/home/my_user/Downloads/foobar_gstplay/gst.log GST_DEBUG_DUMP_DOT_DIR=/home/my_user/Downloads/foobar_gstplay gst-play-1.0 --videosink glimagesink 4k_hdr_sample.webm

[Philippe Normand]

So glimagesink negotiates dmabuf with the decoder. Our GL sink doesn't support this but there's some experimental dmabuf sink, opt-in, through the WEBKIT_GST_DMABUF_SINK_ENABLED=1 env var. Can you try?

[Jeff Fortin]

Created attachment 464656 [details]
screenshot of video graphics corruption when playing with dmabuf enforced

Running with WEBKIT_GST_DMABUF_SINK_ENABLED=1 (in addition to all the other gazillion env vars mentioned previously) results in total graphics corruption of the video; that said however, looking at the motion of the corrupt graphics, I can tell it's playing smoothly, at 30 fps at least, and the CPU usage is low (maybe 10-15% per core or so, and intel_gpu_top still shows "video" having non-zero percentage), so it's "working".

Unforunate that we have so far been utterly unable to get gst debug output and dotfiles from this specific combo of gstreamer + webkitgtk + epiphany + flatpak.

I will await further instructions, I will need an exact foolproof command line for it.

Attached here is a screenshot of what the corrupt video output looks like; note that I resized the screenshot down to 1500px, because it otherwise weighted a ton due to the 4K screen resolution.

[Philippe Normand]

+Zan

Can you reproduce this issue with the dmabuf sink? Looks like a stride issue?

[Philippe Normand]

*** Bug 251679 has been marked as a duplicate of this bug. ***

[Philippe Normand]

In bug 252744 Zan has a patch fixing the dmabuf sink render issue.

[Philippe Normand]

The rendering issue is fixed now. Can you test again? With GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vampeg2dec:MAX,vavp8dec:MAX,vavp9dec:MAX WEBKIT_GST_DMABUF_SINK_ENABLED=1

[Kdwk]

I am no longer able to reproduce this issue

[Philippe Normand]

And Jeff?

[Philippe Normand]

Jeff, in case the issue still happens for you feel free to reopen.