Bug 225456 - [GStreamer] Remove dependency on glvideoflip and videoflip
Summary: [GStreamer] Remove dependency on glvideoflip and videoflip
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Platform (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Miguel Gomez
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-05-06 07:10 PDT by Philippe Normand
Modified: 2021-06-03 07:28 PDT (History)
13 users (show)

See Also:


Attachments
screenshot (171.85 KB, image/png)
2021-05-06 07:10 PDT, Philippe Normand
no flags Details
Patch (8.65 KB, patch)
2021-06-03 03:56 PDT, Miguel Gomez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe Normand 2021-05-06 07:10:24 PDT
Seems like it is more efficient to let the texturemapper handle video rotation. Sadly this currently breaks media/video-orientation-canvas.html as shown in the attached screenshot.
Comment 1 Philippe Normand 2021-05-06 07:10:53 PDT
Created attachment 427883 [details]
screenshot
Comment 2 Miguel Gomez 2021-06-02 04:55:08 PDT
I guess glvideoflip is performing a texture copy to perform the rotation, which is not as efficient as doing it ourselves when painting the frame (we save a copy).

I assume the same happens with the videflip element: a copy is performed in order to get the rotated frame.

IMO we should get rid of both videoflip and glvideoflip, and always perform the rotation ourselves: better performance (save a copy) and simpler code (a single case to handle).
Comment 3 Philippe Normand 2021-06-02 05:08:47 PDT
I wonder what's the performance impact of glvideoflip. Internally it uses this element:

https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/ext/gl/gstgltransformation.c

Anyways, I agree if can do all the things with the texturemapper it's nicer.
Comment 4 Miguel Gomez 2021-06-02 07:01:27 PDT
(In reply to Philippe Normand from comment #3)
> I wonder what's the performance impact of glvideoflip. Internally it uses
> this element:

As expected, it's a texture copy using a shader that performs the rotation. I can't say exactly, but I recall seeing this operations taking around 1-2ms on the rpi2 (more on bigger resolutions). This may not seem a lot, but when rendering at 60fps we only have 16ms to render each frame, those ms count. Also, the OpenGL pipeline is a shared resource, and the time spent rotating the frames is time that is not used rendering other stuff.

> https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/ext/
> gl/gstgltransformation.c
> 
> Anyways, I agree if can do all the things with the texturemapper it's nicer.

Let me cook a patch for that.
Comment 5 Miguel Gomez 2021-06-03 03:56:12 PDT
Created attachment 430458 [details]
Patch
Comment 6 EWS 2021-06-03 07:27:19 PDT
Committed r278406 (238431@main): <https://commits.webkit.org/238431@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 430458 [details].
Comment 7 Radar WebKit Bug Importer 2021-06-03 07:28:16 PDT
<rdar://problem/78814201>