[GTK][WPE] Add support to render video frames with non RGBA/BGRA/ARGB formats through cairo

• Component: WebKitGTK
• Status: RESOLVED INVALID

[Miguel Gomez]

There are 2 situations where we need to render video frames with cairo:
- AC disabled: in this case we use VideoSinkGStreamer, which accepts only BGRA/BGRx (or ARGB/xRGB on big endian). Inside ImageGStreamerCairo we create a cairo surface from that data, and then paint that surface into the appropriate context. No issues here.
- Render video frames into a non accelerated canvas: if we are in non AC mode, this is similar to the previous case. But if we are in AC mode, then we're using GLVideoSinkGStreamer, which accepts several formats besides RGBA and friends (I420, Y444, YV12, Y41B, Y42B, NV12, NV21 and VUYA). This means that inside ImageGStreamerCairo we should be converting those formats into RGBA so cairo can render then, and this step is missing.


At the current revision (r278153) drawing video frames into a non accelerated is not supported at all (MediaPlayerPrivateGStreamer::nativeImageForCurrentTime returns nullptr). I guess the support was removed because of these new formats. Also, this is related to need to rotate the video frame according to the orientation flags and removing the dependency of videoflip (which is being handled here https://bugs.webkit.org/show_bug.cgi?id=225456).

[Miguel Gomez]

> At the current revision (r278153) drawing video frames into a non
> accelerated is not supported at all
> (MediaPlayerPrivateGStreamer::nativeImageForCurrentTime returns nullptr). I
> guess the support was removed because of these new formats. Also, this is
> related to need to rotate the video frame according to the orientation flags
> and removing the dependency of videoflip (which is being handled here
> https://bugs.webkit.org/show_bug.cgi?id=225456).

Ah, seems that things have change and now painting into a canvas doesn't use MediaPlayerPrivateGStreamer::nativeImageForCurrentTime, but it ends calling paint in MediaPlayerPrivateGStreamer. So canvas rendering is working in the end. The need to support the video formats is still there though.

[Miguel Gomez]

After some more digging it seems that the video frame is converted to some RGBx/RBGA format before trying to create the cairo surface from it, so this is not really necessary.