Bug 232728 – REGRESSION (Safari 15): AudioContext.currentTime speeds up (and audio won't play) when Bluetooth speaker connected

• Component: Web Audio
• Status: RESOLVED FIXED

[Luke Abbott]

When certain Bluetooth speakers are connected to certain Apple devices, AudioContext stops behaving properly. The most obvious thing is that my app's audio cuts out after a couple seconds, and will not return until the page is reloaded. Upon further inspection, I discovered that there is actually some *time dilation* going on inside the AudioContext – serious Einstein business!

Minimal reproduction steps, which can be executed on any webpage in Safari after connecting an affected speaker:

  ctx = new AudioContext();
  window.addEventListener('click', () => ctx.resume());
  setInterval(() => console.log(ctx.currentTime), 1000);

After tapping to start the AudioContext, the console should show the current time as increasing by one second every second. When it's acting up, I saw it increasing by 1.84 every second.

Many users have reported this issue to me, who are using various speakers, including expensive Sony and Bose speakers. At least one user has this issue on a Mac, although I have so far only reproduced it on my iPhone with my Tribit XSound Go. I ordered another affected speaker, the OontZ Angle 3 Ultra, and will be testing on that in a few days. The issue is intermittent; it was happening for a solid 30 reloads, and then it fixed itself just now, but I'm sure it'll stop working again. Users have reported that it persists even after multiple device reboots.

This issue first appeared in Safari 15. It also affects Cordova apps in iOS 15.

[Luke Abbott]

Update: I just caught the issue happening on my iPhone (SE 2020) while it was NOT connected to a Bluetooth speaker. At first, I noticed around a second of latency between when the sound was supposed to be played and when I heard it (through the internal speaker). Later, it was a few seconds, at which point the sound cut out and would not come back until the page was reloaded, as before.

I quickly connected the Safari DevTools and ran the same console test, and determined that the AudioContext clock was advancing 1.09 seconds for every real second.

FYI, I also checked AudioContext.getOutputTimestamp().performanceTime, and that was advancing correctly (one second per second).

[Gavin Lefebvre]

We just started seeing this behavior after releasing our Web Audio-based workaround for quiet WebRTC audio last week and not seeing it AT ALL through the beta process (and this was with intensive testing and focusing on Bluetooth behavior when backgrounding the app, etc to hand off best-effort audio playback by unmuting HTMLAudioElements whenever the AudioContext is not running).

So, while we generally fixed audio for folks who weren't using external devices previously, it seems like we broke it for most others. This issue extends back to earlier 15.0 releases suddenly, as if it were one of those "silent" updates all devices get to address fingerprinting or similar.

We have had some reports of the performance stabilizing if you pair the Bluetooth device after instantiating the AudioContext.

How does Apple & the Safari/WebKit teams intend this to be used? It would be great to see a performant reference application that runs on actual hardware.

[Radar WebKit Bug Importer]

<rdar://problem/85075538>

[TomG]

We've also discovered this behaviour on iOS15 devices.

Using this WebAudio example: https://mdn.github.io/webaudio-examples/audiocontext-states/ 

After creating the context, we've had some of our devices have sound cut off at 6 seconds, some at 30 seconds. But the time display below is always faster than what it should be. Once it gets a few seconds ahead of what it should be (about 2), it cuts off. Suspending and renewing does nothing for the audio. The context needs to be killed and then recreated. But then it happens all over again.

One interesting thing to note. We only experience that behaviour remotely. If we run the same code for the test above in a local server, we have no problem. The context time ticks along at the correct speed.

[Chris Dumez]

(In reply to TomG from comment #4)
> We've also discovered this behaviour on iOS15 devices.
> 
> Using this WebAudio example:
> https://mdn.github.io/webaudio-examples/audiocontext-states/ 
> 
> After creating the context, we've had some of our devices have sound cut off
> at 6 seconds, some at 30 seconds. But the time display below is always
> faster than what it should be. Once it gets a few seconds ahead of what it
> should be (about 2), it cuts off. Suspending and renewing does nothing for
> the audio. The context needs to be killed and then recreated. But then it
> happens all over again.
> 
> One interesting thing to note. We only experience that behaviour remotely.
> If we run the same code for the test above in a local server, we have no
> problem. The context time ticks along at the correct speed.

Ok, it looks like we are able to reproduce with https://mdn.github.io/webaudio-examples/audiocontext-states/ on macOS (without bluetooth), simply by changing the hardware sample rate (using Audio MIDI Setup app) while the demo is rendering.
After a little bit, the audio will stop rendering.

Seems related to our WebAudio stack not dealing properly with the hardware sample rate changing. We are investigating, thanks for the report.

[Chris Dumez]

Created attachment 443725 [details]
WIP Patch

Still under discussion whether this is the best approach but this does fix the issue for me locally.

[Chris Dumez]

Created attachment 443733 [details]
Patch

[youenn fablet]

Comment on attachment 443733 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=443733&action=review

> Source/WebKit/GPUProcess/media/RemoteAudioDestinationManager.cpp:112
> +        if (m_ringBuffer->fetchIfHasEnoughData(ioData, numberOfFrames, m_startFrame, WebCore::CARingBuffer::Copy, &remainingFramesInRingBuffer)) {

RemoteAudioMediaStreamTrackRendererInternalUnitManager::Unit::render is probably dealing with the same issue without having to tweak fetchIfHasEnoughData.

[Chris Dumez]

(In reply to youenn fablet from comment #8)
> Comment on attachment 443733 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=443733&action=review
> 
> > Source/WebKit/GPUProcess/media/RemoteAudioDestinationManager.cpp:112
> > +        if (m_ringBuffer->fetchIfHasEnoughData(ioData, numberOfFrames, m_startFrame, WebCore::CARingBuffer::Copy, &remainingFramesInRingBuffer)) {
> 
> RemoteAudioMediaStreamTrackRendererInternalUnitManager::Unit::render is
> probably dealing with the same issue without having to tweak
> fetchIfHasEnoughData.

Could you clarify how? As far as I can tell RemoteAudioMediaStreamTrackRendererInternalUnitManager::Unit::render() may have the same bug?

[Chris Dumez]

(In reply to Chris Dumez from comment #9)
> (In reply to youenn fablet from comment #8)
> > Comment on attachment 443733 [details]
> > Patch
> > 
> > View in context:
> > https://bugs.webkit.org/attachment.cgi?id=443733&action=review
> > 
> > > Source/WebKit/GPUProcess/media/RemoteAudioDestinationManager.cpp:112
> > > +        if (m_ringBuffer->fetchIfHasEnoughData(ioData, numberOfFrames, m_startFrame, WebCore::CARingBuffer::Copy, &remainingFramesInRingBuffer)) {
> > 
> > RemoteAudioMediaStreamTrackRendererInternalUnitManager::Unit::render is
> > probably dealing with the same issue without having to tweak
> > fetchIfHasEnoughData.
> 
> Could you clarify how? As far as I can tell
> RemoteAudioMediaStreamTrackRendererInternalUnitManager::Unit::render() may
> have the same bug?

Oh I see, it seems to rely on m_generateOffset. I can probably do the same if we prefer.

[Chris Dumez]

Comment on attachment 443733 [details]
Patch

Clearing r? flag while I investigate Youenn's proposal.

[Chris Dumez]

Created attachment 443737 [details]
Patch

[EWS]

Committed r285550 (244064@main): <https://commits.webkit.org/244064@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 443737 [details].

[Luke Abbott]

Wonderful news! My users and I thank you very much. 🙏