Bug 224689 – [GStreamer] Kaltura video playback is choppy (also affects cnn.com, reddit.com, various other websites)

• Component: Media
• Status: RESOLVED FIXED

[Michael Catanzaro]

Created attachment 426265 [details]
Debug log

Video playback of any Kaltura video, e.g. https://demo.kaltura.me/rapt-test.html, is slow and choppy. This happens with both Ephy Tech Preview and F34 system Epiphany, both using gstreamer-openh264.

Surprisingly, Firefox has the same issue. (Could be something up with OpenH264?)

P.S. It's open source: https://github.com/kaltura/kaltura-player-js

[Michael Catanzaro]

Created attachment 426266 [details]
Screencast of choppy playback

[Philippe Normand]

Might be an openh264dec bug indeed:

0:00:04.095260351 230244 0x7f6560003430 LOG        webkitmediaplayer MediaPlayerPrivateGStreamer.cpp:767:didLoadingProgress:<media-player-5> didLoadingProgress: false
0:00:04.104046315 230244 0x5604e2269060 ERROR            openh264dec :0::<openh264dec0> [OpenH264] this = 0x0x7f63780bf6d0, Error:DecodeCurrentAccessUnit()::::::PrefetchPic ERROR, pSps->iNumRefFrames:5.

[Michael Catanzaro]

Do you think it's a GStreamer bug, or a bug with OpenH264?

[Michael Catanzaro]

Still broken with the 21.08 runtime (including GStreamer 1.18)

[Philippe Normand]

(In reply to Michael Catanzaro from comment #4)
> Still broken with the 21.08 runtime (including GStreamer 1.18)

I'm not very surprised, given how little maintenance the gst openh264 decoder receives.

[Michael Catanzaro]

Created attachment 440927 [details]
Example on cnn.com

Here's another example: https://dynaimage.cdn.cnn.com/cnn/animations/w_447/211012075606-desktop-climate-central-london.mp4. I almost opened a new bug for this before I realized it's probably this same issue.

[Philippe Normand]

(In reply to Michael Catanzaro from comment #6)
> Created attachment 440927 [details]
> Example on cnn.com
> 
> Here's another example:
> https://dynaimage.cdn.cnn.com/cnn/animations/w_447/211012075606-desktop-
> climate-central-london.mp4. I almost opened a new bug for this before I
> realized it's probably this same issue.

This one plays just fine with gst main openh264dec.
In the future please open GStreamer bugs for openh264 issues.

[Michael Catanzaro]

(In reply to Philippe Normand from comment #7)
> This one plays just fine with gst main openh264dec.

I feel like we could avoid a lot of the "works for me" issues if we aligned the WebKit and GNOME runtimes on the same version of GStreamer.

Looks like we have gst 1.18.5 in freedesktop-sdk 21.08, so that is the version that we would need to investigate to ensure that Tech Preview works properly.

> In the future please open GStreamer bugs for openh264 issues.

Sorry, but I have no way to know what is an openh264 issue vs. what is a WebKit issue until you tell me. It seems that most of the multimedia bugs I report turn out to be WebKit issues, though, so I think this Bugzilla is at least the right place to start. ;)

[Michael Catanzaro]

(In reply to Michael Catanzaro from comment #8)
> Looks like we have gst 1.18.5 in freedesktop-sdk 21.08, so that is the
> version that we would need to investigate to ensure that Tech Preview works
> properly.

That is actually the latest stable version. Do you think this may have been fixed in openh264dec between 1.18.5 and current main?

[Michael Catanzaro]

(In reply to Philippe Normand from comment #7)
> This one plays just fine with gst main openh264dec.
> In the future please open GStreamer bugs for openh264 issues.

Are you sure it's working fine? In current Tech Preview, it seems to work for the first few seconds, but then it degrades badly. Again, this is using the very latest GStreamer 1.18.5.

[Philippe Normand]

(In reply to Michael Catanzaro from comment #10)
> (In reply to Philippe Normand from comment #7)
> > This one plays just fine with gst main openh264dec.
> > In the future please open GStreamer bugs for openh264 issues.
> 
> Are you sure it's working fine? In current Tech Preview, it seems to work
> for the first few seconds, but then it degrades badly. Again, this is using
> the very latest GStreamer 1.18.5.

when I tested this it was with the latest development version back then. Of the git main branch. Which is much ahead of 1.18.5.

[Philippe Normand]

Wasn't the openh264 flatpak extension bumped recently? Is this still happening with newer openh264? In any case I'm not sure we can do much about this in WebKit. Might be better to report the issue to the openh264 bug tracker maybe?

[Michael Catanzaro]

(In reply to Philippe Normand from comment #12)
> Wasn't the openh264 flatpak extension bumped recently? Is this still
> happening with newer openh264? 

It's still happening with the latest Ephy Tech Preview, using OpenH264 2.1.0. But the latest version of OpenH264 is 2.3.0. I have no clue why the update has not been made available. I will ask.

> In any case I'm not sure we can do much about
> this in WebKit. Might be better to report the issue to the openh264 bug
> tracker maybe?

Maybe, but (a) this should be reported by a multimedia developer, not by me, and (b) it will sadly be ignored on that issue tracker regardless.

[Philippe Normand]

Here after enabling the new gst va plugin, playback in Ephy TP works much better.

export GST_PLUGIN_FEATURE_RANK=vah264dec:MAX,vah265dec:MAX,vavp8dec:MAX,vavp9dec:MAX

If the maintenance of the openh264 plugin is not good enough I see only 2 options:

a. Hire qualified engineers to help with maintenance

or

b. Disable h.264 support in Ephy

At this point I'm even tempted to down-rank openh264 directly in WebKit. It's such a bad user experience.

[Michael Catanzaro]

I'll continue to argue in support of option a. There is a chance that funding for this might actually materialize, perhaps.

(Of course we can't disable H.264.)

> At this point I'm even tempted to down-rank openh264 directly in WebKit. It's such a bad user experience.

This would prefer the va plugin if available, but would not disable openh264 entirely, correct? If so, that sounds fine.

[Philippe Normand]

(In reply to Michael Catanzaro from comment #15)
> I'll continue to argue in support of option a. There is a chance that
> funding for this might actually materialize, perhaps.
> 
> (Of course we can't disable H.264.)
> 
> > At this point I'm even tempted to down-rank openh264 directly in WebKit. It's such a bad user experience.
> 
> This would prefer the va plugin if available, but would not disable openh264
> entirely, correct? If so, that sounds fine.

If the VA decoder can't be created because the runtime host is missing hardware capabilities then decodebin should try the next decoder in line, yes.

[Philippe Normand]

Maybe we could first do this experiment in Ephy TP? Add this env var: `GST_PLUGIN_FEATURE_RANK=openh264dec:0`

[Michael Catanzaro]

I'm not sure Tech Preview would be a good place to experiment because it's not supposed to have any encumbered codecs available. Certainly there should not be any va decoders capable of handling H.264 there; if so, that would be a bug.

[Michael Catanzaro]

BTW if you're willing to create an OpenH264 bug report for the original issue here, then I can try to get it some attention.

[Philippe Normand]

(In reply to Michael Catanzaro from comment #18)
> I'm not sure Tech Preview would be a good place to experiment because it's
> not supposed to have any encumbered codecs available. Certainly there should
> not be any va decoders capable of handling H.264 there; if so, that would be
> a bug.

flatpak run --command=bash  --user org.gnome.Epiphany.Devel
[📦 org.gnome.Epiphany.Devel ~]$ gst-inspect-1.0 va
Plugin Details:
  Name                     va
  Description              VA-API codecs plugin
  Filename                 /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstva.so
  Version                  1.20.4
  License                  LGPL
  Source module            gst-plugins-bad
  Source release date      2022-10-12
  Binary package           GStreamer Bad Plug-ins source release
  Origin URL               freedesktop-sdk

  vadeinterlace: VA-API Deinterlacer
  vah264dec: VA-API H.264 Decoder
  vah265dec: VA-API H.265 Decoder
  vampeg2dec: VA-API Mpeg2 Decoder
  vapostproc: VA-API Video Postprocessor

  5 features:
  +-- 5 elements

[Philippe Normand]

I see this more like a feature ;) Besides, enabling hw decoding is good for the environment and your laptop fans will thank you.

[Philippe Normand]

By default the va plugin is not used though, because the rank of the elements is too low...

[Michael Catanzaro]

Hm, does it actually work? It's OK for it to be installed as long as it doesn't work. I'm afraid it really needs to not work. :/

[Philippe Normand]

(In reply to Michael Catanzaro from comment #23)
> Hm, does it actually work? It's OK for it to be installed as long as it
> doesn't work. I'm afraid it really needs to not work. :/

From my personal experience on a laptop with intel GPU and a desktop with AMD GPU, the VA decoder works, and actually rendering looks correct too, which is not the case for the openh264 decoder.

It's opt-in though, using an env var...

[Michael Catanzaro]

Phil, I have no doubt that the va decoder is way better than OpenH264. But that really doesn't matter. The fact that va decoders for encumbered codecs are available in the main GNOME runtime currently is a serious oversight, which I've just reported so that GNOME can decide what to do about that. Most likely we will split those out to an extension.

Back to the original topic of this bug. Clearly you've concluded that this is an OpenH264 bug and not a WebKit bug, yes? But I don't have enough multimedia expertise to create a useful bug report to the OpenH264 developers. Is that something you can help with?

[Michael Catanzaro]

(In reply to Michael Catanzaro from comment #25)
> The fact that va decoders for encumbered codecs
> are available in the main GNOME runtime currently is a serious oversight,
> which I've just reported so that GNOME can decide what to do about that.

I'm wrong. :) The va decoders cannot be used without va drivers that are not shipped, so it's fine.

[Philippe Normand]

Is this still happening?

[Michael Catanzaro]

No, this was a longstanding bug but I think it was fixed at some point during the past two years.