Created attachment 291592 [details] Test page to demonstrate bug with a few measurements Scenario: play() audio or video, adjust playbackRate property, observe a small hickup (approx. 90ms) both in audio and video playback. We have a use case where we need to do minor adjustments to the playback rate of audio and video elements. Other browsers can update the playbackRate property at any time without any audio/visual artifacts. Safari produces a significant pause in both audio and video playback. I've created a small test page to demonstrate, even thought it can easily be reproduced and noticed by hand by something like element.play();setTimeout(function() {element.playbackRate = 1.2;}, 1000); The attached test will first play 5s at speed 1.0, then 5s at speed 1.1 and finally 5s at speed 1.2. The expected playback time is 16.5s of content. The printout shows measured speed (should be close to 1.0, 1.1 and 1.2), and the next line shows the difference in actual content played back compared to what was expected. The first is expected to be a bit off as it typically includes the media element to actually start playing, the next two should be close to 0 and fairly consistent. Quick test results, Safari 10 on iOS: the last two diffs are about 80ms off. Safari 9.1 on osX: about 300ms off, same laptop with Chrome: about 10ms (no audio/visual effects).
I'd be interested to see a native implementation of this testcase using AVPlayer to see if it's an underlying platform issue (in which case this discussion needs to move to a Radar), or if it's WebKit.
Njål, could you also attach your audio and video resources to this bug?
Created attachment 291648 [details] Audio test file (blank sound)
Created attachment 291649 [details] 30 seconds of Big Buck Bunny
Added the files I used in this test, but it is very consistent on any mediafile, even with a console update of playbackRate while playing.
Created attachment 291663 [details] AVPlayerSeekTest Attaching the equivalent test using AVPlayer directly. Here's the log for running the video and audio tests: Testing Audio. Running (will take about 17 seconds)... Speed 1.0: 0.949601, 1.1: 1.08663, 1.2: 1.18765 Diff 1.0: -0.252159, 1.1: -0.252159, 1.2: -0.252159 Wall clock: 1.0: 5.00329, 1.1: 5.17626, 1.2: 5.03575 Testing Video. Running (will take about 17 seconds)... Speed 1.0: 0.973931, 1.1: 1.09151, 1.2: 1.19286 Diff 1.0: -0.143297, 1.1: -0.143297, 1.2: -0.143297 Wall clock: 1.0: 5.49682, 1.1: 5.03157, 1.2: 5.53344 So the audio-only test case shows about a 250ms gap due to rate changes, and the audio+video test case shows a 143ms gap. This should be filed as a Radar on http://bugreport.apple.com/ against AVFoundation.
Created attachment 291664 [details] AVPlayerSeekTest Whoops, ignore those last results. Here's a corrected test case and corrected results: Testing Audio. Running (will take about 17 seconds)... Speed 1.0: 0.953977, 1.1: 1.0916, 1.2: 1.18753 Diff 1.0: -0.253184, 1.1: -0.0462079, 1.2: -0.0628837 Wall clock: 1.0: 5.50124, 1.1: 5.49772, 1.2: 5.0417 Testing Video. Running (will take about 17 seconds)... Speed 1.0: 0.970443, 1.1: 1.09085, 1.2: 1.19289 Diff 1.0: -0.148004, 1.1: -0.0460803, 1.2: -0.03929 Wall clock: 1.0: 5.00743, 1.1: 5.03773, 1.2: 5.52882 Still a gap due to seeking, but a much smaller one. Still quite audible however.
Has this bug been reported against AVFoundation? I tried to look at the Apple bug report site, but I really can't find my way around any of it.
<rdar://problem/29068765>
*** Bug 208142 has been marked as a duplicate of this bug. ***
We have to follow up on the issue of audio/video freezes after playbackRate changes. We currently experience freezes of ~200-300ms on iOS. To our knowledge the issue appeared or became clearly noticable in iOS v15 and still persists in iOS v17.2. We didn't notice it in iOS versions prior to v15. Regarding the impact on user experience it might be considered a small quirk if the playback rate changes are initiated by user interaction, but it's far more disturbing with use cases including playback rate changes issued from the application level. Please be aware that playback rate adjustments are a commonly used tool in low-latency video players to control/fine-tune the amount of buffer and latency without(!) causing noticable playback interruptions. Is there any progress with the OS/framework development side?