Bug 163433 – playbackRate update while playing creates a small pause in the playback

• Component: Media
• Status: NEW

[Njål Borch]

Created attachment 291592 [details]
Test page to demonstrate bug with a few measurements

Scenario: play() audio or video, adjust playbackRate property, observe a small hickup (approx. 90ms) both in audio and video playback.

We have a use case where we need to do minor adjustments to the playback rate of audio and video elements. Other browsers can update the playbackRate property at any time without any audio/visual artifacts. Safari produces a significant pause in both audio and video playback.

I've created a small test page to demonstrate, even thought it can easily be reproduced and noticed by hand by something like element.play();setTimeout(function() {element.playbackRate = 1.2;}, 1000);

The attached test will first play 5s at speed 1.0, then 5s at speed 1.1 and finally 5s at speed 1.2.  The expected playback time is 16.5s of content. The printout shows measured speed (should be close to 1.0, 1.1 and 1.2), and the next line shows the difference in actual content played back compared to what was expected. The first is expected to be a bit off as it typically includes the media element to actually start playing, the next two should be close to 0 and fairly consistent.

Quick test results, Safari 10 on iOS: the last two diffs are about 80ms off. Safari 9.1 on osX: about 300ms off, same laptop with Chrome: about 10ms (no audio/visual effects).

[Jer Noble]

I'd be interested to see a native implementation of this testcase using AVPlayer to see if it's an underlying platform issue (in which case this discussion needs to move to a Radar), or if it's WebKit.

[Jer Noble]

Njål, could you also attach your audio and video resources to this bug?

[Njål Borch]

Created attachment 291648 [details]
Audio test file (blank sound)

[Njål Borch]

Created attachment 291649 [details]
30 seconds of Big Buck Bunny

[Njål Borch]

Added the files I used in this test, but it is very consistent on any mediafile, even with a console update of playbackRate while playing.

[Jer Noble]

Created attachment 291663 [details]
AVPlayerSeekTest

Attaching the equivalent test using AVPlayer directly. Here's the log for running the video and audio tests:

Testing Audio.
Running (will take about 17 seconds)...
Speed 1.0: 0.949601, 1.1: 1.08663, 1.2: 1.18765
Diff 1.0: -0.252159, 1.1: -0.252159, 1.2: -0.252159
Wall clock: 1.0: 5.00329, 1.1: 5.17626, 1.2: 5.03575
Testing Video.
Running (will take about 17 seconds)...
Speed 1.0: 0.973931, 1.1: 1.09151, 1.2: 1.19286
Diff 1.0: -0.143297, 1.1: -0.143297, 1.2: -0.143297
Wall clock: 1.0: 5.49682, 1.1: 5.03157, 1.2: 5.53344

So the audio-only test case shows about a 250ms gap due to rate changes, and the audio+video test case shows a 143ms gap.

This should be filed as a Radar on http://bugreport.apple.com/ against AVFoundation.

[Jer Noble]

Created attachment 291664 [details]
AVPlayerSeekTest

Whoops, ignore those last results. Here's a corrected test case and corrected results:

Testing Audio.
Running (will take about 17 seconds)...
Speed 1.0: 0.953977, 1.1: 1.0916, 1.2: 1.18753
Diff 1.0: -0.253184, 1.1: -0.0462079, 1.2: -0.0628837
Wall clock: 1.0: 5.50124, 1.1: 5.49772, 1.2: 5.0417
Testing Video.
Running (will take about 17 seconds)...
Speed 1.0: 0.970443, 1.1: 1.09085, 1.2: 1.19289
Diff 1.0: -0.148004, 1.1: -0.0460803, 1.2: -0.03929
Wall clock: 1.0: 5.00743, 1.1: 5.03773, 1.2: 5.52882

Still a gap due to seeking, but a much smaller one. Still quite audible however.

[Njål Borch]

Has this bug been reported against AVFoundation? I tried to look at the Apple bug report site, but I really can't find my way around any of it.

[Radar WebKit Bug Importer]

<rdar://problem/29068765>

[Eric Carlson]

*** Bug 208142 has been marked as a duplicate of this bug. ***

We have to follow up on the issue of audio/video freezes after playbackRate changes. We currently experience freezes of ~200-300ms on iOS.
To our knowledge the issue appeared or became clearly noticable in iOS v15 and still persists in iOS v17.2. We didn't notice it in iOS versions prior to v15.

Regarding the impact on user experience it might be considered a small quirk if the playback rate changes are initiated by user interaction, but it's far more disturbing with use cases including playback rate changes issued from the application level.
Please be aware that playback rate adjustments are a commonly used tool in low-latency video players to control/fine-tune the amount of buffer and latency without(!) causing noticable playback interruptions.

Is there any progress with the OS/framework development side?

[Mathieu POUX]

Hello,

We are facing exactly to the same issue on iOS only (>=17.1) => if you change dynamically the playbackRate during playback on HTMLVideoElement, it creates audio flickering.