Bug 246824 – [GStreamer] Video element keeps changing the aspect ratio randomly (when the orientation information is in video's metadata)

• Status: RESOLVED
• Resolution: FIXED
• Component: Media

[Miikka]

When I'm wathing videos (HTML5 elements) in WebKitGTK based browser (like Gnome Epiphany/Web or WebKit's MiniBrowser), the browser/video player keeps changing the aspect ratio randomly. Usually they are MP4 videos that have been shot by using a mobile phone in portrait mode (where the orientation information stored as a metadata inside the video). I've couple of example where to test this:

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1338 the second video here: Firefox, MPV, VLC and "gst-launch-1.0 filesrc location=scarf.mp4 ! decodebin name=dec ! videoflip method=automatic ! videoconvert ! xvimagesink" works just fine, but in Gnome Epiphany/Web or WebKit MiniBrowser the aspect ratio of that video keeps changing.

Another example, I've uploaded the video here: http://wapsi.kapsi.fi/v/ Again the video works just fine on Firefox, MPV, VLC and "gst-launch-1.0 filesrc location=example.mp4 ! decodebin name=dec ! videoflip method=automatic ! videoconvert ! xvimagesink", but in Gnome Epiphany/Web or WebKit MiniBrowser the aspect ratio of that video keeps changing.

I've verified this problem on WebKitGTK 3.38.0, which has been built/tested against Gstreamer 1.18.4 and also Gstreamer master git branch from 2022-10-19. I don't think it's an issue with the Gstreamer, because the playback works correctly when I'm using gst-launch-1.0 with videoflip method=automatic pipe (otherwise it doesn't rotate the video correctly). I'm also facing the issue with AMD Radeon and Intel graphic cards (VAAPI enabled and disabled), so I don't think it's a GPU/Mesa/Compositing issue, or something similar.

I also executed the browser with GST_DEBUG=INFO environment variable activated, and there are these events every time the aspect ratio changes:

0:00:16.192280851 143258 0x7f800c84eaa0 INFO            videodecoder gstvideodecoder.c:1448:gst_video_decoder_sink_event_default:<avdec_h264-1> upstream tags: taglist, image-orientation=(string)rotate-90, video-codec=(string)"H.264\ \(High\ Profile\)", bitrate=(uint)2084250, minimum-bitrate=(uint)433920, maximum-bitrate=(uint)5376000;
0:00:16.252621760 143258 0x7f800c84eaa0 INFO            videodecoder gstvideodecoder.c:1448:gst_video_decoder_sink_event_default:<avdec_h264-1> upstream tags: taglist, image-orientation=(string)rotate-90, video-codec=(string)"H.264\ \(High\ Profile\)", bitrate=(uint)2084250, minimum-bitrate=(uint)399360, maximum-bitrate=(uint)5376000;

What could cause this? Are you able to reproduce this by using the examples above?

[Radar WebKit Bug Importer]

<rdar://problem/101646324>

[Philippe Normand]

Thanks for the bug-report Miikka, and apologies for the delay.

The issue is that our media-player emits sizeChanged notifications everytime after receiving orientation tag events. We should do this only if the orientation has changed. I've started a patch.

[Philippe Normand]

Pull request: https://github.com/WebKit/WebKit/pull/7148

[Miikka]

I compiled the WebKitGTK with the fixes you provided (https://github.com/WebKit/WebKit/pull/7148/files) and now the issue is gone!

Thank you!

[EWS]

Committed 257912@main (07f6e1de7e30): <https://commits.webkit.org/257912@main>

Reviewed commits have been landed. Closing PR #7148 and removing active labels.