Bug 185470 – REGRESSION(r217927): [GTK][WPE][GStreamer] Video appears as pink/green with gstreamer-imx (i.MX6)

• Status: RESOLVED
• Resolution: FIXED
• Component: WPE WebKit

[Carlos Alberto Lopez Perez]

When using WPE on i.MX6 platform (ARMv7) with GSTREAMER_GL enabled and trying to use the proprietary platform plug-ins for accelerated video decoding (provided by gstreamer element imxvpudec from gstreamer-imx) video playback is broken.
All videos appear as pink/green (even when just playing a raw .mp4 video inside WPE).

Reverting r217927 makes the issue go away (video playback works as expected).

[Philippe Normand]

I would suggest to revert this patch and instead teach the texture mapper to convert RGBA to the format Cairo expects for canvas/webgl.

[Carlos Alberto Lopez Perez]

I uploaded here a video showing how bad this looks when playing the big buck bunny video: https://people.igalia.com/clopez/wkbug/185470/

[Carlos Alberto Lopez Perez]

I reverted this downstream in Yocto layer meta-webkit <https://github.com/Igalia/meta-webkit/commit/23997e6c359178ebfdcf05a20f3c371e094564cb>

Let's wait for Miguel feedback to see how we handle this upstream.

[Miguel Gomez]

The goal of requesting BGRA (on little endian) is to avoid requiring color swap when AC is disabled and we need to pass the frames to cairo, who expects ARGB. Any other usage of the frames that goes through the TextureMapper is converted from BGRA to RGBA by the shader, with no CPU cost. So with this change we don't need to perform CPU color swapping at all.

The question here is why GStreamer is not delivering BGRA frames as it's being requested. I guess there may be some bug in the driver that is causing that, and in that case we have to fix that downstream, but not upstream IMO.

[Miguel Gomez]

One thing that I realized is that on some platforms (like the rpi2) the video decoder cannot deliver BGRA directly so it needs to perform a colorspace conversion, which causes a performance degradation. So for those platforms RGBA is requested instead of BGRA. Maybe we can generalize this to the whole WPE platform.

WPE always has AC enabled. This means that we don't have to handle the non AC case, which is the one that motivates that we request BGRA (there's an extra case, which is painting video frames to a non accelerated canvas, but it's not as relevant).

I guess we could request RGBA for the WPE platform, tell the TextureMapper not to perform any color conversion, and modify ImageGStreamerCairo so it performs the sw colorspace conversion in that case. This would fix this bug and avoid the performance penalty when we need to perform the extra color conversion.

[Philippe Normand]

glcolorconvert is already added in the pipeline (and a shader will be used for conversion in most cases) so I don't see much sense to restrict the caps, unless for very specific cases.

[Wouter Vanhauwaert]

Is there still some progress on this one? Problem is still there in 2.21.91, so I re-reverted r217927. But that's not a very nice solution. What should we do to have it fixed permanently? And where?

[Philippe Normand]

Can you check if the issue can be reproduced with gst-play-1.0 --videosink "'video/x-raw(memory:GLMemory), format=(string)RGBA ! glimagesink" ?

If so, then the bug should be forwarded to whoever maintains the imxvpu decoders, Freescale, maybe?

[Philippe Normand]

sorry I meant BGRA not RGBA, which is the format that works.

[Wouter Vanhauwaert]

These all just seem to work, but I guess there's something wrong with the command, cause I can change to BGRC, BGRD, .... whatever and all seem to work. glsink I cannot change to something else however

[Philippe Normand]

gst-play-1.0  --videosink 'glupload ! glcolorconvert ! capsfilter caps="video/x-raw(memory:GLMemory),format=BGRA" ! glimagesinkelement'

[Wouter Vanhauwaert]

That just gives me the first screen... But nothing seems to be running.
BTW, with this patch reverted, video is playing. But it plays a lot better from the console with gst-play (without specifying sink) then it does in the browser. WPEWebProcess is also hitting 100% constantly

[Philippe Normand]

(In reply to Wouter Vanhauwaert from comment #12)
> That just gives me the first screen... But nothing seems to be running.
> BTW, with this patch reverted, video is playing. But it plays a lot better
> from the console with gst-play (without specifying sink) then it does in the
> browser. WPEWebProcess is also hitting 100% constantly

Seems like the zero-copy code path isn't used then. This can be due to many different issues... Like broken GL shared context negotiation, gst-gl bugs, bad combination of imx decoders with old versions of GStreamer. And so on.

[Philippe Normand]

OK. So I think this can be closed now since bug #196346 is fixed. Right Zan?