251091 2023-01-24 09:08:51 -0800 Distorted audio after getUserMedia when playing with AudioWorkletNode 2023-02-07 09:49:40 -0800 1 1 1 Unclassified WebKit Web Audio Safari 16 Unspecified iOS 16 RESOLVED FIXED InRadar P2 Major --- 235317 1 webkit cdumez cdumez eric.carlson jer.noble tobias.hegemann webkit-bug-importer youennf oldest_to_newest 1927834 0 webkit 2023-01-24 09:08:51 -0800 If audio is played with a connected AudioWorkletNode after calling mediaDevices.getUserMedia, audio is badly distorted. A demonstration of the problem is given here: https://goldwave.com/audio.html Choose the Get Media button, then choose the Play button. Audio is badly distorted. Reload the page, choose the Play button, audio is clear. This issue occurs on iOS devices and MacOS all fully updated (Safari 16.3) 1929775 1 webkit-bug-importer 2023-01-31 09:09:18 -0800 <rdar://problem/104870451> 1930866 2 464823 tobias.hegemann 2023-02-03 07:43:38 -0800 Created attachment 464823 Simple test to recreate the audio worklet bug on iOS 1930868 3 tobias.hegemann 2023-02-03 07:50:27 -0800 We also encountered this issue on iOS 16.2 It affects the whole web audio context or its nodes, e.g. OscillatorNode, AudioBufferSourceNode etc. It can be reproduced when using the first AudioWorklet inside an audio context. I've attached a simple test to reproduce and deployed it also on gh: https://delude88.github.io/ios-audioworklet-sample-rate/ Be aware: It is playing sinus waves at full volume. Besides: When closing and recreating another audio context, the issue is gone (until again the first AudioWorklet is used) 1931388 4 cdumez 2023-02-06 08:47:13 -0800 Looking at https://goldwave.com/audio.html, the issue we experience is that the rendering quantum becomes 960 frames instead of 128 frames, which our code doesn't deal with well. Normally, as soon as WebAudio is in use, we request that CoreAudio uses a rendering quantum of 128. However, it is not working here and that's causing the issue. 1931395 5 cdumez 2023-02-06 09:14:54 -0800 It is because of this logic: ``` void RemoteAudioSessionProxyManager::updatePreferredBufferSizeForProcess() { #if ENABLE(MEDIA_STREAM) if (CoreAudioCaptureSourceFactory::singleton().isAudioCaptureUnitRunning()) { CoreAudioCaptureSourceFactory::singleton().whenAudioCaptureUnitIsNotRunning([weakThis = WeakPtr { *this }] { if (weakThis) weakThis->updatePreferredBufferSizeForProcess(); }); return; } #endif // ... ``` If we're capturing (which we are here since we called getUserMedia), then we defer the setting of the preferred buffer size (128) until we're done capturing. Since capturing is ongoing, we just keep using 960, which breaks Web Audio. 1931397 6 cdumez 2023-02-06 09:15:34 -0800 (In reply to Chris Dumez from comment #5) > It is because of this logic: > ``` > void RemoteAudioSessionProxyManager::updatePreferredBufferSizeForProcess() > { > #if ENABLE(MEDIA_STREAM) > if > (CoreAudioCaptureSourceFactory::singleton().isAudioCaptureUnitRunning()) { > > CoreAudioCaptureSourceFactory::singleton(). > whenAudioCaptureUnitIsNotRunning([weakThis = WeakPtr { *this }] { > if (weakThis) > weakThis->updatePreferredBufferSizeForProcess(); > }); > return; > } > #endif > // ... > ``` > > If we're capturing (which we are here since we called getUserMedia), then we > defer the setting of the preferred buffer size (128) until we're done > capturing. Since capturing is ongoing, we just keep using 960, which breaks > Web Audio. This is a regression from Youenn's Bug 235317. 1931484 7 cdumez 2023-02-06 13:58:26 -0800 Pull request: https://github.com/WebKit/WebKit/pull/9711 1931756 8 ews-feeder 2023-02-07 09:49:36 -0800 Committed 259964@main (87395a602807): <https://commits.webkit.org/259964@main> Reviewed commits have been landed. Closing PR #9711 and removing active labels. 464823 2023-02-03 07:43:38 -0800 2023-02-03 07:43:38 -0800 Simple test to recreate the audio worklet bug on iOS sample-rate.html text/html 2829 tobias.hegemann PCFET0NUWVBFIGh0bWw+CjxodG1sPgo8aGVhZD4KICA8bWV0YSBuYW1lPSJ2aWV3cG9ydCIgY29u dGVudD0id2lkdGg9ZGV2aWNlLXdpZHRoLCBpbml0aWFsLXNjYWxlPTEsIG1heGltdW0tc2NhbGU9 MSI+CiAgPHNjcmlwdCB0eXBlPSd3b3JrbGV0Jz4KICByZWdpc3RlclByb2Nlc3NvcignYW55LXdv cmtsZXQnLCBjbGFzcyBBbnlXb3JrbGV0IGV4dGVuZHMgQXVkaW9Xb3JrbGV0UHJvY2Vzc29yIHsK ICAgIGNvbnN0cnVjdG9yKCkgewogICAgICBzdXBlcigpOwogICAgICB0aGlzLnBoYXNlID0gMDsK ICAgIH0KICAgIHByb2Nlc3MoaW5wdXQsIG91dHB1dCwgcGFyYW1ldGVycykgewogICAgICBjb25z dCBjaGFubmVsID0gb3V0cHV0WzBdWzBdOwogICAgICBmb3IgKGxldCBpID0gMDsgaSA8IGNoYW5u ZWwubGVuZ3RoOyArK2kpIHsKICAgICAgICAgIGNoYW5uZWxbaV0gPSBNYXRoLnNpbih0aGlzLnBo YXNlKTsKICAgICAgICAgIHRoaXMucGhhc2UgKz0gMC4xOwogICAgICB9CiAgICAgIHJldHVybiB0 cnVlOwogICAgfQogIH0pOwogIDwvc2NyaXB0PgogIDxzY3JpcHQ+CiAgICBmdW5jdGlvbiBwbGF5 T3NjaWxsYXRvcihhdWRpb0NvbnRleHQpIHsKICAgICAgY29uc3Qgb3NjaWxsYXRvciA9IGF1ZGlv Q29udGV4dC5jcmVhdGVPc2NpbGxhdG9yKCk7CiAgICAgIG9zY2lsbGF0b3IuY29ubmVjdChhdWRp b0NvbnRleHQuZGVzdGluYXRpb24pOwogICAgICBvc2NpbGxhdG9yLmZyZXF1ZW5jeS52YWx1ZSA9 IDQ0MDsKICAgICAgb3NjaWxsYXRvci5zdGFydCgpOwogICAgICBvc2NpbGxhdG9yLnN0b3AoYXVk aW9Db250ZXh0LmN1cnJlbnRUaW1lICsgMik7CiAgICAgIHJldHVybiBuZXcgUHJvbWlzZSgocmVz b2x2ZSkgPT4gewogICAgICAgIG9zY2lsbGF0b3Iub25lbmRlZCA9ICgpID0+IHJlc29sdmUoKTsK ICAgICAgfSk7CiAgICB9CiAgICBmdW5jdGlvbiBwbGF5V29ya2xldChhdWRpb0NvbnRleHQpIHsK ICAgICAgY29uc3Qgd29ya2xldFNjcmlwdCA9IGRvY3VtZW50LnF1ZXJ5U2VsZWN0b3IoJ3Njcmlw dFt0eXBlPXdvcmtsZXRdJyk7CiAgICAgIGNvbnN0IGJsb2IgPSBuZXcgQmxvYihbd29ya2xldFNj cmlwdC5pbm5lclRleHRdLCB7IHR5cGU6ICdhcHBsaWNhdGlvbi9qYXZhc2NyaXB0JyB9KTsKICAg ICAgY29uc3QgdXJsID0gVVJMLmNyZWF0ZU9iamVjdFVSTChibG9iKTsKICAgICAgYXVkaW9Db250 ZXh0LmF1ZGlvV29ya2xldC5hZGRNb2R1bGUodXJsKS50aGVuKCgpID0+IHsKICAgICAgICBjb25z dCB3b3JrbGV0ID0gbmV3IEF1ZGlvV29ya2xldE5vZGUoYXVkaW9Db250ZXh0LCAnYW55LXdvcmts ZXQnKTsKICAgICAgICB3b3JrbGV0LmNvbm5lY3QoYXVkaW9Db250ZXh0LmRlc3RpbmF0aW9uKTsK ICAgICAgICByZXR1cm4gbmV3IFByb21pc2UoKHJlc29sdmUpID0+IHsKICAgICAgICAgIHNldFRp bWVvdXQoKCkgPT4gewogICAgICAgICAgICB3b3JrbGV0LmRpc2Nvbm5lY3QoKTsKICAgICAgICAg ICAgcmVzb2x2ZSgpOwogICAgICAgICAgfSwgMjAwMCk7CiAgICAgICAgfSkKICAgICAgfSk7CiAg ICB9CiAgICBmdW5jdGlvbiBydW4oKSB7CiAgICAgIGNvbnN0IGF1ZGlvQ29udGV4dCA9IG5ldyBB dWRpb0NvbnRleHQoKTsKICAgICAgYXVkaW9Db250ZXh0LnJlc3VtZSgpCiAgICAgICAgLnRoZW4o KCkgPT4gewogICAgICAgICAgY29uc29sZS5sb2coIlBsYXlpbmcgb3NjaWxsYXRvciBmb3IgMiBz ZWNvbmRzIik7CiAgICAgICAgICByZXR1cm4gcGxheU9zY2lsbGF0b3IoYXVkaW9Db250ZXh0KTsK ICAgICAgICB9KQogICAgICAgIC50aGVuKCgpID0+IHsKICAgICAgICAgIGNvbnNvbGUubG9nKCJM b2FkIGFuZCBwbGF5IHdvcmtsZXQgZm9yIDIgc2Vjb25kcyIpOwogICAgICAgICAgcmV0dXJuIHBs YXlXb3JrbGV0KGF1ZGlvQ29udGV4dCk7CiAgICAgICAgfSkKICAgICAgICAudGhlbigoKSA9PiB7 CiAgICAgICAgICBjb25zb2xlLmxvZygiUGxheWluZyBvc2NpbGxhdG9yIGFnYWluIGZvciAyIHNl Y29uZHMiKTsKICAgICAgICAgIHJldHVybiBwbGF5T3NjaWxsYXRvcihhdWRpb0NvbnRleHQpOwog ICAgICAgIH0pCiAgICAgICAgLnRoZW4oKCkgPT4gewogICAgICAgICAgY29uc29sZS5sb2coIkZp bmlzaGVkLCBjbGVhbmluZyB1cCIpOwogICAgICAgICAgYXVkaW9Db250ZXh0LmNsb3NlKCk7CiAg ICAgICAgfSkKICAgICAgICAuY2F0Y2goZXJyID0+IGNvbnNvbGUuZXJyb3IoZXJyKSk7CiAgICB9 CiAgPC9zY3JpcHQ+CjwvaGVhZD4KPGJvZHk+CjxoMT5GaXhtZTogaU9TIEF1ZGlvV29ya2xldCBi dWc8L2gxPgo8cD4KICBUaGUgZm9sbG93aW5nIHJ1biB3aWxsIHBlcmZvcm0gdGhlIGZvbGxvd2lu ZyBzdGVwczoKPC9wPgo8b2w+CiAgPGxpPlBsYXlpbmcgYW4gb3NjaWxsYXRvciBub2RlIGZvciAy IHNlY29uZHM8L2xpPgogIDxsaT5Mb2FkaW5nIGFuZCBwbGF5aW5nIGFuIEF1ZGlvV29ya2xldCBm b3IgMiBzZWNvbmRzIChnZW5lcmF0aW5nIGEgc2ludXMgc291bmQpPC9saT4KICA8bGk+UGxheWlu ZyBhbiBvc2NpbGxhdG9yIG5vZGUgZm9yIDIgc2Vjb25kcyAoYWdhaW4pPC9saT4KPC9vbD4KPHA+ CiAgTm90aWNlIGhvdyB0aGUgc291bmQgZ2V0cyBhZmZlY3RlZCBvbiB0aGUgc2Vjb25kIGFuZCB0 aGlyZCBzdGVwIG9uIGlPUyBkZXZpY2VzIChXZWJLaXQpLgogIEl0IHNvdW5kcyBsaWtlIGFuIGlu dGVybmFsIHNhbXBsZSByYXRlIG1pc21hdGNoID8hPwogIEhvcGUgdGhpcyB0ZXN0IGhlbHBzIQog IDotKQo8L3A+CjxpbnB1dCB0eXBlPSdzdWJtaXQnIHZhbHVlPSdSdW4nIG9uY2xpY2s9J3J1bigp OyByZXR1cm4gZmFsc2U7JyAvPgo8L2JvZHk+CjwvaHRtbD4K