Bug 227882 - [GStreamer] Video playback on many websites is jerky (sometimes after seek)
Summary: [GStreamer] Video playback on many websites is jerky (sometimes after seek)
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: WebKit Nightly Build
Hardware: PC Linux
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-07-12 12:53 PDT by Michael Catanzaro
Modified: 2021-08-13 16:35 PDT (History)
4 users (show)

See Also:


Attachments
Screencast (4.70 MB, video/webm)
2021-07-12 12:53 PDT, Michael Catanzaro
no flags Details
GStreamer debug log (536.15 KB, text/x-log)
2021-07-12 12:53 PDT, Michael Catanzaro
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Catanzaro 2021-07-12 12:53:22 PDT
Created attachment 433344 [details]
Screencast

* Choose some video from https://vimeo.com/watch
 * Start playing the video in Ephy Tech Preview and note it is working fine
 * Seek the video. The first seek does not work and causes video playback to restart from the beginning. Loads of GStreamer warnings print on the terminal.
 * Seek the video again. The video playback gets all messed up.

Screencast and GStreamer log attached. The warnings from the initial seek look like this:

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.088: ../gst/gstpad.c:4549:gst_pad_push_data:<fdkaacdec0:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<audio_0:proxypad18> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<decodebin3-0:audio_0> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<audio_0:proxypad19> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<uridecodebin3-0:audio_0> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<audio-concat:sink_0> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<audio-concat:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<playsink:audio_sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<audio_sink:proxypad7> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<audiotee:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<audiotee:src_0> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<streamsynchronizer0:sink_1> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<streamsynchronizer0:src_1> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<abin:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<sink:proxypad27> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<filter-convert:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<filter-convert:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<scaletempo0:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<scaletempo0:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<aqueue:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<aqueue:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<aconv:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<sink:proxypad25> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<conv:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<conv:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<resample:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<resample:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<src:proxypad26> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<aconv:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<audio-sink:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<sink:proxypad1> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<audioTee:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<audioTee:src_0> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<queue0:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<queue0:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<audioconvert0:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<audioconvert0:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<audioresample0:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<audioresample0:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<volume:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-CRITICAL **: 14:51:26.089: gst_segment_to_stream_time: assertion 'segment->format == format' failed

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<volume:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<audioconvert1:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<audioconvert1:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<audioresample1:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<audioresample1:src> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<autoaudiosink0:sink> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4549:gst_pad_push_data:<sink:proxypad0> Got data flow before segment event

(WebKitWebProcess:170): GStreamer-WARNING **: 14:51:26.089: ../gst/gstpad.c:4296:gst_pad_chain_data_unchecked:<autoaudiosink0-actual-sink-pulse:sink> Got data flow before segment event
Comment 1 Michael Catanzaro 2021-07-12 12:53:55 PDT
Created attachment 433345 [details]
GStreamer debug log
Comment 2 Philippe Normand 2021-07-13 06:10:26 PDT
Works fine in ToT.
Comment 3 Philippe Normand 2021-07-13 06:11:39 PDT
Maybe another symptom of a broken runtime ;)

0:00:12.769677211   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 4098
0:00:12.769714743   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 4098
0:00:12.769738808   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 4098
0:00:12.769779385   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 4098
0:00:12.769804864   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 4098
0:00:12.769840241   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 4098
0:00:12.769868665   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 4098
0:00:12.769892410   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 4098
0:00:12.769968976   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 5
0:00:12.770011286   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 4098
0:00:12.770050791   170 0x7fe79000fea0 WARN            audiodecoder gstaudiodecoder.c:3181:_gst_audio_decoder_error:<fdkaacdec0> error: decoding error: 4098
0:00:12.770133909   170 0x7fe77800e4c0 WARN                basesink gstbasesink.c:3495:gst_base_sink_chain_unlocked:<autoaudiosink0-actual-sink-pulse> warning: Internal data flow problem.
0:00:12.770191508   170 0x7fe77800e4c0 WARN                basesink gstbasesink.c:3495:gst_base_sink_chain_unlocked:<autoaudiosink0-actual-sink-pulse> warning: Received buffer without a new-segment. Assuming timestamps start from 0.
0:00:12.770149148   170 0x5600ec3bf750 LOG        webkitmediaplayer MediaPlayerPrivateGStreamer.cpp:1775:handleMessage:<MSE-media-player-0> Message error received from element fdkaacdec0
0:00:12.770245521   170 0x5600ec823aa0 ERROR         webkitmediasrc WebKitMediaSourceGStreamer.cpp:518:webKitMediaSrcLoop:<source:src_A1> Pushing buffer returned error
0:00:12.770247825   170 0x5600ec3bf750 ERROR      webkitmediaplayer MediaPlayerPrivateGStreamer.cpp:1781:handleMessage: Error 7: Could not decode stream. (url=mediasourceblob:https://vimeo.com/7614ff3b-8476-48d2-982c-a682b4d0b6b5)
Comment 4 Sun 2021-07-15 00:29:37 PDT
I have a similar issue : vimeo video jerky but with a getUserMedia.

- On an iPhone with OS 14.5, go to https://codepen.io/sunlimet/pen/yLbgOEx
- Button share, then "Add to Home Screen"
- Open PWA, press play = video is lagging
- As you can see, there's only the Vimeo iframe and a navigator.mediaDevices.getUserMedia

Android do not have this issue, Chrome either.
Can reproduce on old IPhones (SE first and second generation, iPhone 6), but it's ok on iPhone 11

In Safari standalone (not PWA), it is better but it's also lagging
Comment 5 Philippe Normand 2021-07-15 00:45:22 PDT
(In reply to Sun from comment #4)
> I have a similar issue : vimeo video jerky but with a getUserMedia.
> 
> - On an iPhone with OS 14.5, go to https://codepen.io/sunlimet/pen/yLbgOEx
> - Button share, then "Add to Home Screen"
> - Open PWA, press play = video is lagging
> - As you can see, there's only the Vimeo iframe and a
> navigator.mediaDevices.getUserMedia
> 
> Android do not have this issue, Chrome either.
> Can reproduce on old IPhones (SE first and second generation, iPhone 6), but
> it's ok on iPhone 11
> 
> In Safari standalone (not PWA), it is better but it's also lagging

This specific bug is about GStreamer ports. Apple ports don't use GStreamer, please file a new bug.
Comment 6 Michael Catanzaro 2021-07-15 07:27:21 PDT
(In reply to Philippe Normand from comment #3)
> Maybe another symptom of a broken runtime ;)

FWIW I've been prodding about GStreamer updates, and we are now *tentatively* planning to start updating GStreamer in the GNOME runtime, but only once we switch to buildstream 2, which apparently makes it easier to override elements built by freedesktop-sdk. That way, freedesktop-sdk can stick with one branch of GStreamer for an entire year, but GNOME can update to a newer branch twice per year.

However, buildstream seems to have run out of development funding, so the timeline for this is uncertain. In the meantime, GStreamer devs are well aware of our release cycle....
Comment 7 Michael Catanzaro 2021-07-23 05:58:07 PDT
Still broken with the 21.08 runtime (including GStreamer 1.18)
Comment 8 Michael Catanzaro 2021-08-11 13:15:15 PDT
Also happens on cnn.com:

https://www.cnn.com/videos/politics/2021/08/11/cory-booker-tommy-tuberville-defund-the-police-newday-vpx.cnn/video/playlists/this-week-in-politics/

Sometimes it even starts flickering without requiring a seek, but if not, then a seek should be sufficient to reproduce.

I also recently noticed this happening inconsistently on youtube.com as well, although it's a lot harder to reproduce on youtube.com for some reason.
Comment 9 Michael Catanzaro 2021-08-13 16:35:55 PDT
Much lighter example: https://dynaimage.cdn.cnn.com/cnn/animations/w_307/210813092733-desktop-race-dotmap.mp4