Bug 214625 - [GStreamer][MSE] fix video freeze in NASA TV feed
Summary: [GStreamer][MSE] fix video freeze in NASA TV feed
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: Other
Hardware: PC Linux
: P2 Normal
Assignee: Xabier Rodríguez Calvar
URL:
Keywords: Gtk, InRadar
Depends on:
Blocks:
 
Reported: 2020-07-21 19:50 PDT by way
Modified: 2021-03-16 08:05 PDT (History)
16 users (show)

See Also:


Attachments
debug output from epiphany (19.28 KB, text/plain)
2020-07-21 19:50 PDT, way
no flags Details
webm screen cast (3.02 MB, video/webm)
2020-07-22 09:54 PDT, way
no flags Details
Patch (3.11 KB, patch)
2021-03-12 08:01 PST, Xabier Rodríguez Calvar
no flags Details | Formatted Diff | Diff
AAC stream causing errors in fdkaacdec. (647.17 KB, video/mp4)
2021-03-15 10:34 PDT, Alicia Boya García
no flags Details
Patch for landing (3.10 KB, patch)
2021-03-16 07:14 PDT, Xabier Rodríguez Calvar
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description way 2020-07-21 19:50:12 PDT
Created attachment 404894 [details]
debug output from epiphany

webkit v2.28.3

https://www.nasa.gov/multimedia/nasatv/index.html

On Debian Buster/Siduction ThinkPad T430. (using epiphany 3.34 and 3.36)

This is a report, not a complaint.

The video seems to play OK, at least for a while. The audio sounds like it's not playing at the correct speed or frequency. Very difficult to listen to. I opened up cookie and content settings, but that didn't make any difference. Firefox plays it without any issues, if that helps. As does Chromium.


Michael Catanzaro recommended that I report this here (originally reported to the address below)
GNOME
EpiphanyEpiphany
Issues
#1273

https://gitlab.gnome.org/GNOME/epiphany/-/issues/1273#note_871867
Comment 1 Philippe Normand 2020-07-22 01:20:35 PDT
So this is an MSE thing, h264 playback works, but aac decoding fails after a seconds here and the player restarts, and so on. Is that what you observe as well?
Comment 2 Michael Catanzaro 2020-07-22 07:07:14 PDT
(In reply to Philippe Normand from comment #1)
> So this is an MSE thing, h264 playback works, but aac decoding fails after a
> seconds here and the player restarts, and so on. Is that what you observe as
> well?

That's what I observe.
Comment 3 way 2020-07-22 08:55:07 PDT
To be more specific, I see the video play normally for about 10-15 seconds and then the video freezes. The audio starts out garbled and stays that way, continuing even after the video has stopped. If I pause the stream, audio will stop, but if I click on play afterward, neither the video or the audio resume playing. I can refresh the page and the above process will repeat.

Your description is a little more technical and may be what is happening under the hood. However, I don't see the video restarting at all after freezing.
Comment 4 Philippe Normand 2020-07-22 09:08:16 PDT
Can you record a screencast showing the issue?
Comment 5 way 2020-07-22 09:54:57 PDT
Created attachment 404933 [details]
webm screen cast

A screeencast of NASA TV streaming problem.
Comment 6 Philippe Normand 2020-07-23 07:59:55 PDT
This is not exactly what I observe here, but we use 2 different webkitgtk versions so it's not so surprising. What I mentioned in comment 1 is for 2.29.3.
Comment 7 way 2020-07-23 08:30:37 PDT
(In reply to Philippe Normand from comment #6)

Let me know if there is any other information I can provide.
Comment 8 Philippe Normand 2020-07-23 08:35:08 PDT
Well, from what I can see, this could likely be a GStreamer bug. We'd need to collect a sample of the audio stream triggering errors in the decoder and report this in GStreamer's gitlab once we have a standalone bug reproducer.
Comment 9 Philippe Normand 2021-02-08 06:45:10 PST
Seems to work well here now, both in Ephy 3.38 TP and MiniBrowser ToT.
Comment 10 Philippe Normand 2021-02-08 06:52:19 PST
(In reply to Philippe Normand from comment #9)
> Seems to work well here now, both in Ephy 3.38 TP and MiniBrowser ToT.

Ah well, audio plays fine, but video freezes after a while -_-
Comment 11 Xabier Rodríguez Calvar 2021-03-12 08:01:52 PST
Created attachment 423051 [details]
Patch
Comment 12 Xabier Rodríguez Calvar 2021-03-12 08:38:56 PST
My patch solves the video freeze but there is still a problem with the playback restart that I should keep investigating. After that I might put it together in a single patch or split it, depending on the problem.
Comment 13 Alicia Boya García 2021-03-12 08:54:12 PST
There seem to be 3 problems with the NASA stream at the moment:

1. Video stalling sometimes.
2. The audio being appended causes problems in GStreamer *and* other players, but not other browsers.
3. The player is restarted every 20 seconds or so because of an error (could be a consequence of the previous problems)

The patch upload by calvaris could fix (1). On the other hand, I've been looking at the audio problem.... There is some problem in the audio stream being fed to WebKit. It fails in all decoders, but in a different way in each:

avdec_aac: audio plays slowly (distorted), but without returning errors.

fdkaacdec:

0:00:00.160266130 131498 0x5597ec76ee40 WARN            audiodecoder gstaudiodecoder.c:1415:gst_audio_decoder_finish_frame_or_subframe:<fdkaacdec0> warning: received more decoded frames 1 than provided 0
0:00:00.160305745 131498 0x5597ec76ee40 WARN            audiodecoder gstaudiodecoder.c:1535:gst_audio_decoder_finish_frame_or_subframe:<fdkaacdec0> Can't copy metadata because input buffers disappeared
WARNING: from element /GstPipeline:pipeline0/GstFdkAacDec:fdkaacdec0: received more decoded frames 1 than provided 0
Additional debug info:
../gst-libs/gst/audio/gstaudiodecoder.c(1415): gst_audio_decoder_finish_frame_or_subframe (): /GstPipeline:pipeline0/GstFdkAacDec:fdkaacdec0
0:00:00.160504279 131498 0x5597ec76ee40 WARN            audiodecoder gstaudiodecoder.c:3255:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 5
0:00:00.160518700 131498 0x5597ec76ee40 ERROR              fdkaacdec gstfdkaacdec.c:433:gst_fdkaacdec_flush:<fdkaacdec0> flushing error: 5
0:00:00.396790152 131498 0x5597ec76ee40 WARN            audiodecoder gstaudiodecoder.c:1535:gst_audio_decoder_finish_frame_or_subframe:<fdkaacdec0> Can't copy metadata because input buffers disappeared
0:00:00.396805458 131498 0x5597ec76ee40 WARN            audiodecoder gstaudiodecoder.c:1778:gst_audio_decoder_drain:<fdkaacdec0> audio decoder push buffers failed

faad:

0:00:00.122223811 131574 0x55ad8f7d6e40 WARN            audiodecoder gstaudiodecoder.c:1790:gst_audio_decoder_drain:<faad0> still 1 frames left after draining
0:00:00.307125461 131574 0x55ad8f7d6e40 WARN            audiodecoder gstaudiodecoder.c:1790:gst_audio_decoder_drain:<faad0> still 1 frames left after draining

The root problem could be in the demuxing however. NASA TV uses MPEG TS with some transmuxing code generating MP4, and the output of that may be triggering bugs in our demuxer.
Comment 14 Xabier Rodríguez Calvar 2021-03-15 08:44:30 PDT
(In reply to Alicia Boya García from comment #13)
> 1. Video stalling sometimes.

Yes, my patch fixes that.

> 2. The audio being appended causes problems in GStreamer *and* other
> players, but not other browsers.

It does create issues in other browsers. I can hear an audio glitch around second twelve if I play in Firefox and Chromium.

> 3. The player is restarted every 20 seconds or so because of an error (could
> be a consequence of the previous problems)

Let's see if I can find the cause of this.
Comment 15 Alicia Boya García 2021-03-15 10:34:31 PDT
Created attachment 423194 [details]
AAC stream causing errors in fdkaacdec.

The stream contains sample rate changes and GStreamer is not handling them correctly. Attaching test vector.

This can be reproduced without WebKit.

$ GST_DEBUG=WARN gst-launch-1.0 pushfilesrc location=SourceBuffer-1-audio-0.mp4 ! qtdemux ! fdkaacdec ! audioconvert ! autoaudiosink
Comment 16 Alicia Boya García 2021-03-16 06:15:37 PDT
Comment on attachment 423051 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=423051&action=review

> Source/WebCore/ChangeLog:3
> +        [GStreamer][MSE] NASA TV feed not playing correctly.

This could use a more specific title, since this feed exposes several bugs. For instance: fix video freeze in NASA TV feed.
Comment 17 Xabier Rodríguez Calvar 2021-03-16 07:14:08 PDT
Created attachment 423324 [details]
Patch for landing
Comment 18 EWS 2021-03-16 08:04:14 PDT
Committed r274478: <https://commits.webkit.org/r274478>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 423324 [details].
Comment 19 Radar WebKit Bug Importer 2021-03-16 08:05:18 PDT
<rdar://problem/75477282>