[GStreamer][MSE] fix video freeze in NASA TV feed

• Component: Media
• Status: RESOLVED FIXED

[way]

Created attachment 404894 [details]
debug output from epiphany

webkit v2.28.3

https://www.nasa.gov/multimedia/nasatv/index.html

On Debian Buster/Siduction ThinkPad T430. (using epiphany 3.34 and 3.36)

This is a report, not a complaint.

The video seems to play OK, at least for a while. The audio sounds like it's not playing at the correct speed or frequency. Very difficult to listen to. I opened up cookie and content settings, but that didn't make any difference. Firefox plays it without any issues, if that helps. As does Chromium.


Michael Catanzaro recommended that I report this here (originally reported to the address below)
GNOME
EpiphanyEpiphany
Issues
#1273

https://gitlab.gnome.org/GNOME/epiphany/-/issues/1273#note_871867

[Philippe Normand]

So this is an MSE thing, h264 playback works, but aac decoding fails after a seconds here and the player restarts, and so on. Is that what you observe as well?

[Michael Catanzaro]

(In reply to Philippe Normand from comment #1)
> So this is an MSE thing, h264 playback works, but aac decoding fails after a
> seconds here and the player restarts, and so on. Is that what you observe as
> well?

That's what I observe.

[way]

To be more specific, I see the video play normally for about 10-15 seconds and then the video freezes. The audio starts out garbled and stays that way, continuing even after the video has stopped. If I pause the stream, audio will stop, but if I click on play afterward, neither the video or the audio resume playing. I can refresh the page and the above process will repeat.

Your description is a little more technical and may be what is happening under the hood. However, I don't see the video restarting at all after freezing.

[Philippe Normand]

Can you record a screencast showing the issue?

[way]

Created attachment 404933 [details]
webm screen cast

A screeencast of NASA TV streaming problem.

[Philippe Normand]

This is not exactly what I observe here, but we use 2 different webkitgtk versions so it's not so surprising. What I mentioned in comment 1 is for 2.29.3.

[way]

(In reply to Philippe Normand from comment #6)

Let me know if there is any other information I can provide.

[Philippe Normand]

Well, from what I can see, this could likely be a GStreamer bug. We'd need to collect a sample of the audio stream triggering errors in the decoder and report this in GStreamer's gitlab once we have a standalone bug reproducer.

[Philippe Normand]

Seems to work well here now, both in Ephy 3.38 TP and MiniBrowser ToT.

[Philippe Normand]

(In reply to Philippe Normand from comment #9)
> Seems to work well here now, both in Ephy 3.38 TP and MiniBrowser ToT.

Ah well, audio plays fine, but video freezes after a while -_-

[Xabier Rodríguez Calvar]

Created attachment 423051 [details]
Patch

[Xabier Rodríguez Calvar]

My patch solves the video freeze but there is still a problem with the playback restart that I should keep investigating. After that I might put it together in a single patch or split it, depending on the problem.

[Alicia Boya García]

There seem to be 3 problems with the NASA stream at the moment:

1. Video stalling sometimes.
2. The audio being appended causes problems in GStreamer *and* other players, but not other browsers.
3. The player is restarted every 20 seconds or so because of an error (could be a consequence of the previous problems)

The patch upload by calvaris could fix (1). On the other hand, I've been looking at the audio problem.... There is some problem in the audio stream being fed to WebKit. It fails in all decoders, but in a different way in each:

avdec_aac: audio plays slowly (distorted), but without returning errors.

fdkaacdec:

0:00:00.160266130 131498 0x5597ec76ee40 WARN            audiodecoder gstaudiodecoder.c:1415:gst_audio_decoder_finish_frame_or_subframe:<fdkaacdec0> warning: received more decoded frames 1 than provided 0
0:00:00.160305745 131498 0x5597ec76ee40 WARN            audiodecoder gstaudiodecoder.c:1535:gst_audio_decoder_finish_frame_or_subframe:<fdkaacdec0> Can't copy metadata because input buffers disappeared
WARNING: from element /GstPipeline:pipeline0/GstFdkAacDec:fdkaacdec0: received more decoded frames 1 than provided 0
Additional debug info:
../gst-libs/gst/audio/gstaudiodecoder.c(1415): gst_audio_decoder_finish_frame_or_subframe (): /GstPipeline:pipeline0/GstFdkAacDec:fdkaacdec0
0:00:00.160504279 131498 0x5597ec76ee40 WARN            audiodecoder gstaudiodecoder.c:3255:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 5
0:00:00.160518700 131498 0x5597ec76ee40 ERROR              fdkaacdec gstfdkaacdec.c:433:gst_fdkaacdec_flush:<fdkaacdec0> flushing error: 5
0:00:00.396790152 131498 0x5597ec76ee40 WARN            audiodecoder gstaudiodecoder.c:1535:gst_audio_decoder_finish_frame_or_subframe:<fdkaacdec0> Can't copy metadata because input buffers disappeared
0:00:00.396805458 131498 0x5597ec76ee40 WARN            audiodecoder gstaudiodecoder.c:1778:gst_audio_decoder_drain:<fdkaacdec0> audio decoder push buffers failed

faad:

0:00:00.122223811 131574 0x55ad8f7d6e40 WARN            audiodecoder gstaudiodecoder.c:1790:gst_audio_decoder_drain:<faad0> still 1 frames left after draining
0:00:00.307125461 131574 0x55ad8f7d6e40 WARN            audiodecoder gstaudiodecoder.c:1790:gst_audio_decoder_drain:<faad0> still 1 frames left after draining

The root problem could be in the demuxing however. NASA TV uses MPEG TS with some transmuxing code generating MP4, and the output of that may be triggering bugs in our demuxer.

[Xabier Rodríguez Calvar]

(In reply to Alicia Boya García from comment #13)
> 1. Video stalling sometimes.

Yes, my patch fixes that.

> 2. The audio being appended causes problems in GStreamer *and* other
> players, but not other browsers.

It does create issues in other browsers. I can hear an audio glitch around second twelve if I play in Firefox and Chromium.

> 3. The player is restarted every 20 seconds or so because of an error (could
> be a consequence of the previous problems)

Let's see if I can find the cause of this.

[Alicia Boya García]

Created attachment 423194 [details]
AAC stream causing errors in fdkaacdec.

The stream contains sample rate changes and GStreamer is not handling them correctly. Attaching test vector.

This can be reproduced without WebKit.

$ GST_DEBUG=WARN gst-launch-1.0 pushfilesrc location=SourceBuffer-1-audio-0.mp4 ! qtdemux ! fdkaacdec ! audioconvert ! autoaudiosink

[Alicia Boya García]

Comment on attachment 423051 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=423051&action=review

> Source/WebCore/ChangeLog:3
> +        [GStreamer][MSE] NASA TV feed not playing correctly.

This could use a more specific title, since this feed exposes several bugs. For instance: fix video freeze in NASA TV feed.

[Xabier Rodríguez Calvar]

Created attachment 423324 [details]
Patch for landing

[EWS]

Committed r274478: <https://commits.webkit.org/r274478>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 423324 [details].

[Radar WebKit Bug Importer]

<rdar://problem/75477282>