Bug 225456

Summary: [GStreamer] Remove dependency on glvideoflip and videoflip
Product: WebKit Reporter: Philippe Normand <pnormand>
Component: PlatformAssignee: Miguel Gomez <magomez>
Status: RESOLVED FIXED    
Severity: Normal CC: calvaris, cgarcia, eric.carlson, ews-watchlist, glenn, gustavo, jer.noble, magomez, menard, philipj, sergio, vjaquez, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=225454
Attachments:
Description Flags
screenshot
none
Patch none

Description Philippe Normand 2021-05-06 07:10:24 PDT
Seems like it is more efficient to let the texturemapper handle video rotation. Sadly this currently breaks media/video-orientation-canvas.html as shown in the attached screenshot.
Comment 1 Philippe Normand 2021-05-06 07:10:53 PDT
Created attachment 427883 [details]
screenshot
Comment 2 Miguel Gomez 2021-06-02 04:55:08 PDT
I guess glvideoflip is performing a texture copy to perform the rotation, which is not as efficient as doing it ourselves when painting the frame (we save a copy).

I assume the same happens with the videflip element: a copy is performed in order to get the rotated frame.

IMO we should get rid of both videoflip and glvideoflip, and always perform the rotation ourselves: better performance (save a copy) and simpler code (a single case to handle).
Comment 3 Philippe Normand 2021-06-02 05:08:47 PDT
I wonder what's the performance impact of glvideoflip. Internally it uses this element:

https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/ext/gl/gstgltransformation.c

Anyways, I agree if can do all the things with the texturemapper it's nicer.
Comment 4 Miguel Gomez 2021-06-02 07:01:27 PDT
(In reply to Philippe Normand from comment #3)
> I wonder what's the performance impact of glvideoflip. Internally it uses
> this element:

As expected, it's a texture copy using a shader that performs the rotation. I can't say exactly, but I recall seeing this operations taking around 1-2ms on the rpi2 (more on bigger resolutions). This may not seem a lot, but when rendering at 60fps we only have 16ms to render each frame, those ms count. Also, the OpenGL pipeline is a shared resource, and the time spent rotating the frames is time that is not used rendering other stuff.

> https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/ext/
> gl/gstgltransformation.c
> 
> Anyways, I agree if can do all the things with the texturemapper it's nicer.

Let me cook a patch for that.
Comment 5 Miguel Gomez 2021-06-03 03:56:12 PDT
Created attachment 430458 [details]
Patch
Comment 6 EWS 2021-06-03 07:27:19 PDT
Committed r278406 (238431@main): <https://commits.webkit.org/238431@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 430458 [details].
Comment 7 Radar WebKit Bug Importer 2021-06-03 07:28:16 PDT
<rdar://problem/78814201>