Bug 198053 - [GStreamer][MSE] Desync between audio/video
Summary: [GStreamer][MSE] Desync between audio/video
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: WebKit Nightly Build
Hardware: PC Linux
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-05-20 15:11 PDT by Michael Catanzaro
Modified: 2019-07-29 05:27 PDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Catanzaro 2019-05-20 15:11:56 PDT
Visit any YouTube video featuring a talking head (to make the problem more obvious), e.g. https://www.youtube.com/watch?v=jTBNHkCjgnc

Click to either fullscreen the video, or enlarge into theater mode. Video playback will hang for a second or two while audio continues (unlike in other browsers, where playback is smooth).

Then when playback resumes, notice that the audio and video are slightly desynced. The audio is playing properly (it was never stalled). The video is noticeably a quarter of a second or so behind the audio. This is our fault; they should be perfectly synced.
Comment 1 Michael Catanzaro 2019-05-24 10:27:30 PDT
Actually I've seen this several times even when not changing video size.

Another example where it can be really obvious, at the 3m20s mark:

https://www.youtube.com/watch?v=le1goFKE2B8

Note that seeking directly there will cause the bug to not occur.
Comment 2 Michael Catanzaro 2019-05-24 10:49:34 PDT
We get up to something like 5s of delay between the audio and the video... I'm not sure if the delay increases as the video progresses, or if it just starts out really bad.
Comment 3 Xabier Rodríguez Calvar 2019-05-27 01:36:25 PDT
I can't reproduce the desync in trunk, neither in release or debug mode.
Comment 4 Philippe Normand 2019-05-27 10:14:49 PDT
That's probably because AC is deactivated and you use software decoders. How's your CPU usage?
Comment 5 Michael Catanzaro 2019-05-27 16:46:51 PDT
(In reply to Philippe Normand from comment #4)
> That's probably because AC is deactivated and you use software decoders.
> How's your CPU usage?

<1% CPU usage... even if there was a performance issue, there should not be any desync.
Comment 6 Philippe Normand 2019-06-11 03:02:18 PDT
In Ephy TP it starts to be noticeable around 2:45. I can't reproduce the issue in MiniBrowser from ToT...
Comment 7 Michael Catanzaro 2019-06-17 15:46:55 PDT
(In reply to Michael Catanzaro from comment #2)
> I'm not sure if the delay increases as the video progresses, or if it just
> starts out really bad.

The former. I'm pretty sure the video is correct. Perhaps the audio is playing just slightly too fast and so the difference between the video and audio increases incrementally as playback continues. I agree that around 2-3m it becomes noticeable. Later it becomes more and more obvious. Just reproduced on https://www.youtube.com/watch?v=zxT8CM8XntA.

(In reply to Philippe Normand from comment #6)
> In Ephy TP it starts to be noticeable around 2:45. I can't reproduce the
> issue in MiniBrowser from ToT...

Using our flatpak environment that's based on the GNOME runtime? We seem to have a frequently-recurring problem with multimedia bugs that occur for our users but aren't reproducible in our development environment. I think we're shooting ourselves in the foot by building our own GStreamer in org.webkit.CommonModules.yaml. I suspect removing the custom multimedia stuff and getting it from the upstream runtime would go a long way towards helping to reproduce such issues.
Comment 8 Charlie Turner 2019-07-11 07:27:36 PDT
I reproduced this today testing my deadlock work on Twitter. This tweet seems to reproduce it everytime when I switch back and forth to fullscreen: https://twitter.com/espn/status/1149133320107646977

When you fullscreen, the audio gets way ahead of the video. When you minimize, the video starts playing much faster and then catches up and syncs. Very strange!