Bug 241223 – Web audio rendering becomes garbled with switching from speakers to headphones (and vice-versa)

• Status: RESOLVED
• Resolution: FIXED
• Component: Web Audio

[Ludo]

When plugin headphones while an audioworklet is outputing sound, the sound in headphones has a big drop in quality and it becomes very choppy and full of glitches.
It seems it can be reproduced with any webpage using audioworklet for playback.
Open a page with an audioworklet which output a signal to destination node for instance: 
https://googlechromelabs.github.io/web-audio-samples/audio-worklet/basic/hello-audio-worklet/

Start playback, sound is fine through macBook speacker, plug headphones, then speaker stop rendering sound and sound playback is done through headphones but quality is very low, rendered sound is choppy and full of artefacts.

[Radar WebKit Bug Importer]

<rdar://problem/94721398>

[Chris Dumez]

I have just tried the demo site but it didn't even output anything on the speakers for me. It did work on Chrome though.

[Chris Dumez]

OK, I am able to reproduce now. A few findings:
1. This has nothing to do with AudioWorklet and it is reproducible with just an OscillatorNode in the audio graph

2. It is likely due to some state that remains in the GPUProcess.
The reason I say this is that if you start with speakers and then plug in your headphones, the sound will become bad. If you reload the tab, the sound will remain bad. If you open the demo in a new tab, the sound is still bad as well. However, if you restart Safari, the sound becomes OK (with headphones). Then if you unplug the headphones, the sound becomes bad with the speakers.

[Chris Dumez]

One thing I noticed is that RemoteAudioDestination::render() normally gets called with numberOfFrames=128. However, when I plug in the headphones, RemoteAudioDestination::render() then gets called consistently with numberOfFrames=480.

[Chris Dumez]

(In reply to Chris Dumez from comment #4)
> One thing I noticed is that RemoteAudioDestination::render() normally gets
> called with numberOfFrames=128. However, when I plug in the headphones,
> RemoteAudioDestination::render() then gets called consistently with
> numberOfFrames=480.

And when I restart the browser with the headphone plugged in, numberOfFrames becomes 128 again. As far as I know, our code assumes a rendering quantum of 128 for WebAudio so it is probably not too surprising that rendering goes bad when the rendering quantum becomes 480.

As far as I know MediaSessionManagerCocoa::updateSessionState() is supposed to set the preferred buffer size of 128 when Web Audio is in use. However, clearly this gets ignored when switching audio output.

[Chris Dumez]

(In reply to Chris Dumez from comment #5)
> (In reply to Chris Dumez from comment #4)
> > One thing I noticed is that RemoteAudioDestination::render() normally gets
> > called with numberOfFrames=128. However, when I plug in the headphones,
> > RemoteAudioDestination::render() then gets called consistently with
> > numberOfFrames=480.
> 
> And when I restart the browser with the headphone plugged in, numberOfFrames
> becomes 128 again. As far as I know, our code assumes a rendering quantum of
> 128 for WebAudio so it is probably not too surprising that rendering goes
> bad when the rendering quantum becomes 480.
> 
> As far as I know MediaSessionManagerCocoa::updateSessionState() is supposed
> to set the preferred buffer size of 128 when Web Audio is in use. However,
> clearly this gets ignored when switching audio output.

Ok, so interestingly,  MediaSessionManagerCocoa::updateSessionState() does get called when I plug in my headphones and does call `AudioSession::sharedSession().setPreferredBufferSize(128)`. However, `AudioSessionMac::setPreferredBufferSize()` early returns because `m_bufferSize == bufferSize`.

If I comment out the early return when `m_bufferSize == bufferSize`, then the audio stays good when plugging in the headphones.

[Chris Dumez]

(In reply to Chris Dumez from comment #6)
> (In reply to Chris Dumez from comment #5)
> > (In reply to Chris Dumez from comment #4)
> > > One thing I noticed is that RemoteAudioDestination::render() normally gets
> > > called with numberOfFrames=128. However, when I plug in the headphones,
> > > RemoteAudioDestination::render() then gets called consistently with
> > > numberOfFrames=480.
> > 
> > And when I restart the browser with the headphone plugged in, numberOfFrames
> > becomes 128 again. As far as I know, our code assumes a rendering quantum of
> > 128 for WebAudio so it is probably not too surprising that rendering goes
> > bad when the rendering quantum becomes 480.
> > 
> > As far as I know MediaSessionManagerCocoa::updateSessionState() is supposed
> > to set the preferred buffer size of 128 when Web Audio is in use. However,
> > clearly this gets ignored when switching audio output.
> 
> Ok, so interestingly,  MediaSessionManagerCocoa::updateSessionState() does
> get called when I plug in my headphones and does call
> `AudioSession::sharedSession().setPreferredBufferSize(128)`. However,
> `AudioSessionMac::setPreferredBufferSize()` early returns because
> `m_bufferSize == bufferSize`.
> 
> If I comment out the early return when `m_bufferSize == bufferSize`, then
> the audio stays good when plugging in the headphones.

Normally, you'd expect AudioSessionMac::handleBufferSizeChange() to get called here and update m_bufferSize since the buffer size did change. However, we set the listener in AudioSessionMac::addBufferSizeObserverIfNeeded(), we pass in the defaultDevice(). However, the default device is changing when we connect the headphones. As a result, our observer becomes useless.

[Chris Dumez]

Pull request: https://github.com/WebKit/WebKit/pull/6501

[EWS]

Committed 256712@main (91ac45e95dc3): <https://commits.webkit.org/256712@main>

Reviewed commits have been landed. Closing PR #6501 and removing active labels.

[Chris Dumez]

*** Bug 248592 has been marked as a duplicate of this bug. ***