Bug 260654 – [Nvidia][GStreamer][GTK] Unable to play any video whatsoever

• Status: RESOLVED
• Resolution: FIXED
• Component: WebKitGTK

[Kdwk]

Supersedes https://bugs.webkit.org/show_bug.cgi?id=260456

With Epiphany 45.beta-40-g17c238bbb+/ WebKitGTK 2.41.91, no matter what environment variable set, no video can be played (without crashing).

With no environment variable set, the web view cannot be shown.

With WEBKIT_DISABLE_DMABUF_RENDERER=1, sites with video crash.

With WEBKIT_GST_DMABUF_SINK_DISABLED=1, sites don't crash but videos are black or refuse to be played.

Disabling the GL sink (forgot the environment variable), the video plays for a few seconds then Epiphany receives SIGSEGV, with backtrace that looks like "??()".

GST_PLUGIN_FEATURE_RANK=nvav1dec:MAX,nvh264dec:MAX,nvh265dec:MAX,nvjpegdec:MAX,nvmpeg2videodec:MAX,nvmpeg4videodec:MAX,nvmpegvideodec:MAX,nvvp8dec:MAX,nvvp9dec:MAX does not appear to make any difference.

This only happens in Epiphany Technology Preview/ WebKitGTK 2.41.x, but not in Gnome Web/ WebKitGTK 2.40.x, where video playback works and hardware-accelerated decode works as well.

[Kdwk]

*** Bug 260456 has been marked as a duplicate of this bug. ***

[Erik Kurzinger]

Based on the discussion from https://bugs.webkit.org/show_bug.cgi?id=261874 at least part of the problem is that current versions of the NVIDIA driver do not support allocating GBM buffers with the R8 format. However, the next release (550) will support this. So it may be worth re-testing when that goes public later this year.

[Kdwk]

Created attachment 470157 [details]
Videos are now green

Erik, I have retested this with the 550.54.14 driver and videos became green instead

[Philippe Normand]

It would be interesting to know which format the video decoder outputs and which format the video sink negotiates... Can you share pipeline dumps?

[Kdwk]

How can I get that information?

[Philippe Normand]

https://docs.webkit.org/Ports/WebKitGTK%20and%20WPE%20WebKit/Multimedia.html

[Kdwk]

Created attachment 470179 [details]
Compressed dots folder

Thanks. Here's the compressed dot directory after running the command with youtube.com and playing a few videos there.

[Philippe Normand]

So the decoder outputs raw NV12 and for rendering we convert that to DMABufs within WebKit. That might be where the problem is.

[Philippe Normand]

> With WEBKIT_GST_DMABUF_SINK_DISABLED=1, sites don't crash but videos are black or refuse to be played.

Is this still happening?

[Kdwk]

Currently, with WEBKIT_GST_DMABUF_SINK_DISABLED=1 videos aren't black but simply don't play. Without this environment variable, videos seemingly can play, but they are green.

[Kdwk]

Erik, this issue still persists with Nvidia driver 550. Though GBM buffers with R8 formats are now supported, there remains a problem with converting raw NV12 to DMABufs. Nvidia users cannot view any video in WebKitGTK apps until this is resolved.

[Erik Kurzinger]

> there remains a problem with converting raw NV12 to DMABufs

Could you clarify what you mean by this? With the 550 driver it should be possible to allocate an NV12 GBM buffer and export it as a pair of DMA-BUFs. Note, though, that support for this format in our EGL driver is "external only". So if an NV12 buffer is imported as an EGLImage, that image can only be bound to GL_TEXTURE_EXTERNAL_OES (not, for example, GL_TEXTURE_2D). Could that maybe be the problem?

[Kdwk]

I used flatpak run --command=/bin/bash com.github.kdwk.Spidey to enter the Flatpak runtime then gst-play-1.0 to play local video files. This works fine. 

WebKitGTK 2.45.4/ GStreamer 1.24/

[Philippe Normand]

(In reply to Kdwk from comment #10)
> Currently, with WEBKIT_GST_DMABUF_SINK_DISABLED=1 videos aren't black but
> simply don't play. Without this environment variable, videos seemingly can
> play, but they are green.

If that still happens we'd need gst logs.

[Kdwk]

Created attachment 471804 [details]
Compressed dots folder (DMABuf renderer)

I have the new dots folder when playing videos with the DMABuf renderer. It now shows a black screen instead of green after the update of Nvidia drivers from 550 to 555. Sound works as usual (playback works except video).

[Philippe Normand]

That's not with nvidia decoder, but openh264... try with GST_PLUGIN_FEATURE_RANK=openh264dec:0 ?

[Philippe Normand]

Sorry i was looking at the wrong pipeline :)
What's the output of gst-inspect-1.0 nvvp9dec ?

Having pipeline dump for gst-play would be nice too.

[Kdwk]

Created attachment 471805 [details]
gst-inspect-1.0 nvvp9dec.txt

Here's the output of gst-inspect-1.0 nvvp9dec

[Philippe Normand]

GST_DEBUG="nv*:6,videodec*:6" logs too please

[Philippe Normand]

(In reply to Philippe Normand from comment #8)
> So the decoder outputs raw NV12 and for rendering we convert that to DMABufs
> within WebKit. That might be where the problem is.

Still same issue. Once we understand why the decoder negotiates raw NV12 instead of GLMemories with the sink, the issue will be closer to resolution.

[Kdwk]

(In reply to Philippe Normand from comment #17)

> Having pipeline dump for gst-play would be nice too.

I am unable to get this dump because it keeps insisting the files and directories don't exist.

[Kdwk]

However, gst-play-1.0 keep complaining about 

WARNING A lot of buffers are being dropped.
WARNING debug information: ../libs/gst/base/gstbasesink.c(3147): gst_base_sink_is_too_late (): /GstPlayBin3:playbin/GstPlaySink:playsink/GstBin:vbin/GstAutoVideoSink:videosink/GstGLImageSinkBin:videosink-actual-sink-glimage/GstGLImageSink:sink:
There may be a timestamping problem, or this computer is too slow.

[Philippe Normand]

(In reply to Kdwk from comment #21)
> (In reply to Philippe Normand from comment #17)
> 
> > Having pipeline dump for gst-play would be nice too.
> 
> I am unable to get this dump because it keeps insisting the files and
> directories don't exist.

Are you setting the dangerous env var? See https://docs.webkit.org/Ports/WebKitGTK%20and%20WPE%20WebKit/Multimedia.html#flatpak-apps

[Carlos Garcia Campos]

(In reply to Philippe Normand from comment #20)
> (In reply to Philippe Normand from comment #8)
> > So the decoder outputs raw NV12 and for rendering we convert that to DMABufs
> > within WebKit. That might be where the problem is.
> 
> Still same issue. Once we understand why the decoder negotiates raw NV12
> instead of GLMemories with the sink, the issue will be closer to resolution.

Because we always claim to support NV12?

[Philippe Normand]

Yes...

There's two different issues I think.

- with our GL sink we should ideally behave almost the same as gst-play (with glimagesink) and negotiate GLMemories all the way from decoder to sink.

- with our DMABuf sink I'm not sure how to proceed, but video/x-raw negotiation should be avoided... we'd need a dmabufupload (or maybe glupload can handle GLMemory conversion to DMABuf now) that would convert CUDA or GLMemories from the decoder to DMABufs.

[Carlos Garcia Campos]

(In reply to Philippe Normand from comment #25)
> Yes...
> 
> There's two different issues I think.
> 
> - with our GL sink we should ideally behave almost the same as gst-play
> (with glimagesink) and negotiate GLMemories all the way from decoder to sink.

If you know how to do that...

> - with our DMABuf sink I'm not sure how to proceed, but video/x-raw
> negotiation should be avoided... we'd need a dmabufupload (or maybe glupload
> can handle GLMemory conversion to DMABuf now) that would convert CUDA or
> GLMemories from the decoder to DMABufs.

This is how I think the DMABuf sink works:

We claim to support a fixed list of formats, which are the ones that can be handled by the texture mapper. Then two things can happen:

 1) gst can provide one of the formats as DMABuf: In this case we just import the DMABUf as textures and render them into the texture mapper.

 2) gst can't provide the formats as DMABuf: in this case we receive a raw video frame in one the formats we claimed in caps. We allocate a GBM buffer and upload the video frame there, then we do as if the DMABuf was provided by gst and import it as textures to render them into the texture mapper.

So, in both cases, I wonder what happens if the buffer format (and modifiers) is not supported by the driver. I guess in the first case we will fail to import the DMABuf and in the second case we will fail to allocate the buffer. I guess we should claim to support only the formats supported by the driver, the same way we do in the renderer. Do we know where exactly is failing?

And in case of not getting a DMABuf from gst, is it really more efficient to allocate, upload and then import the DMABuf? Can't we just fallback to GL or non-accelerated video?

[Philippe Normand]

Having the decoder output raw video (in main memory, not GPU) should be avoided, otherwise CPU usage might increase significantly...

Our hand-rolled GBM to DMABuf upload in the player is meant for software decoders...

@Kdwk: Can you get gst logs please? `GST_DEBUG="3,nv*:6`

[Philippe Normand]

Also for testing purpose can you disable the WebProcess sandbox? I suspect it might block access from the nv decoder to the cuda device...

[Kdwk]

Created attachment 472080 [details]
Requested gst logs

I have collected the requested logs

[Philippe Normand]

Having a pipeline graph dump for gst-play-1.0 scenario would be nice so we can compare...

Currently in WebKit the decoder negotiates sysmem on its src pad, instead of GL.

[Kdwk]

Created attachment 472121 [details]
gst-play-1.0 dots

Here is the compressed dots directory when playing an H.264 video with gst-play-1.0

[Kdwk]

Created attachment 472133 [details]
gst log when using philn's patch

gst log when using philn's patch

[Kdwk]

Created attachment 472134 [details]
Compressed dots folder when using philn's patch

Compressed dots folder when using philn's patch

[Philippe Normand]

So with that patch at least GL mem is attempted, but then... "OpenGL context is not CUDA-compatible, fallback to system memory"... :/

[Philippe Normand]

(In reply to Philippe Normand from comment #34)
> So with that patch at least GL mem is attempted, but then... "OpenGL context
> is not CUDA-compatible, fallback to system memory"... :/

Can you share your webkit://gpu infos?

[Kdwk]

Created attachment 472174 [details]
webkit-gpu.txt

Here's webkit://gpu

[Philippe Normand]

This is more graphics-related at this point, I don't think I can help further.

The GL version reported by webkit://gpu seems compatible with what nvvp9dec would expect (>= 3.0) so I don't know what else to do.

[Philippe Normand]

This seems fixed in 2.46.1. Closing.