Bug 221334 – Audio passed through WebAudio is delayed and glitchy on Safari

• Component: Web Audio
• Status: NEW

[Josh]

Hi, currently in all latest versions of Safari (Mac and iOS) there are audio issues when passing audio through WebAudio. Here's a codepen showing the issue: https://codepen.io/jjv360/full/dyOYXvE

On Mac the problem is only to do with delayed audio (the iOS issue below does happen, but much more rarely). You can run the audio sync test in the codepen and see that it almost always has around a 1 second delay. This does not happen in Chrome or Firefox.

On iOS the problem is far worse (this does happen on Mac too, but not nearly as often) and has caused us to warn our users to not use Safari for now. On iOS, the audio sometimes stutters and has tiny gaps of silence (about 1 in 4 times of refreshing the page), which widen the longer the video is played. Eventually, there is no audio at all and it's completely silent. Oddly enough, killing the browser results in about a 1 second of that silent audio playing, almost as if a buffer is full and it just got flushed. This is most noticeable on the iPhone 7, though it happens on all iOS devices. The best way to reproduce it, is if you connect bluetooth headphones, play the video in the codepen above, and then press Access Microphone, this seems to almost always trigger the bug.

This has been tested and reproduced on:
- Safari on iOS 14.4 on an iPhone 7
- Safari on iOS 14.4 on an iPad Pro 4th gen
- Safari on macOS 11.2 on a MacBook Pro 15" 2017

[Chris Dumez]

Have you tried the latest Safari Technology Preview? We've recently completely reworked (and unprefixed) our WebAudio implementation but it has not shipped yet.

I just tried the demo on Safari Technology Preview and it did not seem obviously wrong although I could have missed something.

[Josh]

Can confirm there is no difference in Safari Technical Preview... Here's a video showing the issue: https://www.loom.com/share/62c314b5697246698408a7389ffaac7e
- Using Bluetooth earphones
- Started the GTI video and then pressed Access Microphone
- At 0:22 the first small audio glitch occurred when switching windows
- At 2:13 the second small audio glitch
- At 2:42 things start going really wrong with the audio
- At 2:55 the audio is just entirely gone, it's silent

FYI this was happening before I installed Loom, so I doubt the fact that loom was recording the audio makes any difference...

Here's a video of the exact same bug on an iPad Pro: https://www.loom.com/share/1cbf9afc9cbc4b8ebb4b9134461c1120

[Adrien iWebDJ]

@Chris Dumez

What WebAudio improvements did you do in the latest Safari Technology Preview exactly? 

Please tell me that you have implemented AudioWorklet ;)

[Chris Dumez]

(In reply to Adrien iWebDJ from comment #3)
> @Chris Dumez
> 
> What WebAudio improvements did you do in the latest Safari Technology
> Preview exactly? 
> 
> Please tell me that you have implemented AudioWorklet ;)

We sure did :) Our implementation is now up to W3C standard and unprefixed in STP (including AudioWorklets).

[Adrien iWebDJ]

@Chris Dumez

Omg are you serious?? you are our hero!!! I can't believe it...

This is such a good news for my WebAudio(WKWebView) app for iOS that I am developping for years : https://apps.apple.com/us/app/you-dj-music-mixer-no-ad/id1459666155

The desktop and Android version (https://you.dj) are using AudioWorklet for a long time but safari kept making my life difficult :D

Thanks a lot for your work!

[Chris Dumez]

(In reply to Adrien iWebDJ from comment #5)
> @Chris Dumez
> 
> Omg are you serious?? you are our hero!!! I can't believe it...
> 
> This is such a good news for my WebAudio(WKWebView) app for iOS that I am
> developping for years :
> https://apps.apple.com/us/app/you-dj-music-mixer-no-ad/id1459666155
> 
> The desktop and Android version (https://you.dj) are using AudioWorklet for
> a long time but safari kept making my life difficult :D
> 
> Thanks a lot for your work!

We don't have Safari Technology Preview on iOS but I *think* the new WebAudio API (including AudioWorklet) is present in iOS 14.5 beta. I encourage you to test try it out :)

[Radar WebKit Bug Importer]

<rdar://problem/74190913>

[Smoley]

I tried out the test case at https://codepen.io/jjv360/full/dyOYXvE on Safari 14.0.3 and on TOT and I hear a lot of clipping and stuttering on both builds but only over Bluetooth audio. The issue did not reproduce on Safari 13.1.3.

[Chris Dumez]

I just tried the car review demo on iOS 14 with AirPods, the audio is initially out-of-sync by <1sec. However, if I request microphone permission, then audio gets louder and starts getting very glitchy. I have no idea why the microphone request permission is having such an impact. Some of the media experts in CC may have an idea.

I have also tried on macOS (with AirPods) and I can indeed reproduce the audio being out of sync by <1sec. Audio is sometimes a little bit glitchy. Requesting microphone permission makes audio louder.

I know this is tagged as WebAudio but this is likely at MediaElementAudioSourceNode (or lower level like AudioSourceProvider) level so assistance from media people would be appreciated.

It does not appear those are recent regression as I can reproduce on older builds, before we refactored WebAudio.

[Chris Dumez]

I just tried WebKit ToT on macOS and it looks like there may be some improvement. The audio is initially in sync (even over AirPods). I also don't hear any glitches.

However, if I request microphone permission, there is a pause in audio for >1sec after giving permission and the audio is then out-of-sync with the video when it resumes.

[Chris Dumez]

AudioSourceProviderAVFObjC::process() uses m_writeAheadCount to keep track of how much ahead the writer is compared to media player so that we don't render too early and get out of sync. However, m_writeAheadCount seems to get updated only once, when switching from paused to playing. Maybe when requesting microphone permission (and the audio glitches for 1 second), we'd need to update m_writeAheadCount again?

[Michael]

Hi there, I'm having huge issues with the glitchy audio on iOS 14.4. When I go to Safari Settings > Advanced > Experimental and enable "Modern WebAudio API" it doesn't make things any better. Is the "Modern WebAudio" experimental feature different to the reworked WebAudio in Safari Technlogy Preview?

The website I'm working on is basically completely broken in Safari. I can't use normal HTML5 audio because I need to be able to change audio volume. I can't use WebAudio because I need to load a few tracks on the page, and decoding more than 5 crashes the tab due to memory usage, and I can't even pass audio through MediaElementAudioSourceNode because of the glitchy audio.

I have tried a lot of things including AudioContext hints, using different audio file formats (mp3 and aac, both @ 44100hz). Things go really badly when I change the playbackRate property of the HTMLMediaElement. I also tried different buffer sizes for my ScriptProcessorNode, but the only one that seems to have made a difference is using 16384 bytes (the largest possible?). That makes audio work for a few seconds, still slightly glitchy, but then eventually it goes silent. Then when I stop/start the audio playback it works for about 5 seconds before going silent again.

Tested on iOS 14.4.1 on iPhone 8

This is without any microphones or bluetooth speakers being involved.

[Josh]

Has there been any updates on this? It's still preventing Safari users from using our app... (which locks out every iPad / iPhone user)

[Josh]

Have tried the same tests on iOS 15 beta, there is no change... It is still broken... On an iPhone 8 it even happens without Bluetooth now.

[youenn fablet]

(In reply to Josh from comment #14)
> Have tried the same tests on iOS 15 beta, there is no change... It is still
> broken... On an iPhone 8 it even happens without Bluetooth now.

I tried it on Big Sur:
- Stock Safari has delay
- STP129 seems in sync

[Ian Lovejoy]

Tested AirPods on Big Sur 11.6.7 (Safari 15.5), Monterey 12.4 (Safari 15.5), and iOS 15.5, confirmed that the following sequence still causes severe WebAudio glitching:

1. Connect AirPods
2. Request microphone access via navigator.mediaDevices.getUserMedia()
3. Attempt to play audio via WebAudio API

The symptoms are as described by others: stuttering, long pauses, and (in our case at least) a slowed down playback speed.

Interestingly, wired earbuds and the built in microphone both seem to work fine.

If anyone has found a workaround I'd love to hear it!

[Ian Lovejoy]

Created attachment 460472 [details]
Self-contained, minimal test case that reproduces the issue

In case it is helpful, I am attaching a self-contained, minimal HTML page (36 lines of JS, 47 lines total) that reproduces this issue 100% of the time for me.

The identical code is in CodePen:

https://codepen.io/LegDip/pen/KKQOMRr

Procedure to reproduce:

1. Pair device or computer with AirPods (in my testing it also occurs with other Bluetooth headsets)
2. Launch Safari & load page
3. Press "Request Mic"
4. Allow microphone permissions
5. Press "Play Tone"

Expected: Continuous 440hz tone
Actual: Strongly stuttering 440hz tone

Step #3 creates an AudioContext, calls getUserMedia() then loads an empty AudioWorklet module (from a Blob for this self-contained test case, but the same issue occurs when loading from a non-empty external file).

Step #5 simply plays an OscillatorNode and a GainNode (just for volume control)

If any other background would be helpful, please let me know.