Bug 198053 – [GStreamer][MSE] Desync between audio/video

• Component: Media
• Status: RESOLVED FIXED

[Michael Catanzaro]

Visit any YouTube video featuring a talking head (to make the problem more obvious), e.g. https://www.youtube.com/watch?v=jTBNHkCjgnc

Click to either fullscreen the video, or enlarge into theater mode. Video playback will hang for a second or two while audio continues (unlike in other browsers, where playback is smooth).

Then when playback resumes, notice that the audio and video are slightly desynced. The audio is playing properly (it was never stalled). The video is noticeably a quarter of a second or so behind the audio. This is our fault; they should be perfectly synced.

[Michael Catanzaro]

Actually I've seen this several times even when not changing video size.

Another example where it can be really obvious, at the 3m20s mark:

https://www.youtube.com/watch?v=le1goFKE2B8

Note that seeking directly there will cause the bug to not occur.

[Michael Catanzaro]

We get up to something like 5s of delay between the audio and the video... I'm not sure if the delay increases as the video progresses, or if it just starts out really bad.

[Xabier Rodríguez Calvar]

I can't reproduce the desync in trunk, neither in release or debug mode.

[Philippe Normand]

That's probably because AC is deactivated and you use software decoders. How's your CPU usage?

[Michael Catanzaro]

(In reply to Philippe Normand from comment #4)
> That's probably because AC is deactivated and you use software decoders.
> How's your CPU usage?

<1% CPU usage... even if there was a performance issue, there should not be any desync.

[Philippe Normand]

In Ephy TP it starts to be noticeable around 2:45. I can't reproduce the issue in MiniBrowser from ToT...

[Michael Catanzaro]

(In reply to Michael Catanzaro from comment #2)
> I'm not sure if the delay increases as the video progresses, or if it just
> starts out really bad.

The former. I'm pretty sure the video is correct. Perhaps the audio is playing just slightly too fast and so the difference between the video and audio increases incrementally as playback continues. I agree that around 2-3m it becomes noticeable. Later it becomes more and more obvious. Just reproduced on https://www.youtube.com/watch?v=zxT8CM8XntA.

(In reply to Philippe Normand from comment #6)
> In Ephy TP it starts to be noticeable around 2:45. I can't reproduce the
> issue in MiniBrowser from ToT...

Using our flatpak environment that's based on the GNOME runtime? We seem to have a frequently-recurring problem with multimedia bugs that occur for our users but aren't reproducible in our development environment. I think we're shooting ourselves in the foot by building our own GStreamer in org.webkit.CommonModules.yaml. I suspect removing the custom multimedia stuff and getting it from the upstream runtime would go a long way towards helping to reproduce such issues.

[Charlie Turner]

I reproduced this today testing my deadlock work on Twitter. This tweet seems to reproduce it everytime when I switch back and forth to fullscreen: https://twitter.com/espn/status/1149133320107646977

When you fullscreen, the audio gets way ahead of the video. When you minimize, the video starts playing much faster and then catches up and syncs. Very strange!

[Philippe Normand]

(In reply to Michael Catanzaro from comment #0)
> Visit any YouTube video featuring a talking head (to make the problem more
> obvious), e.g. https://www.youtube.com/watch?v=jTBNHkCjgnc
> 
> Click to either fullscreen the video, or enlarge into theater mode. Video
> playback will hang for a second or two while audio continues (unlike in
> other browsers, where playback is smooth).
> 
> Then when playback resumes, notice that the audio and video are slightly
> desynced. The audio is playing properly (it was never stalled). The video is
> noticeably a quarter of a second or so behind the audio. This is our fault;
> they should be perfectly synced.

I can't reproduce this with Ephy TP. The openh264 decoder was choking a bit at the beginning and introducing stuttering but then it settled... Attaching a screencast (without audio though).

[Philippe Normand]

Created attachment 464352 [details]
screencast

[Michael Catanzaro]

This bug was fixed a long time ago. Let's close it.