Safari 15 breaks all Web Audio content using WebM Opus

• Component: Web Audio
• Status: RESOLVED FIXED

[Ashley Gullen]

Thanks for adding WebM Opus support in Safari 15! We've been waiting for this for a long time. However currently it is supported only for the HTML <audio> element. Web Audio's decodeAudioData() still fails to decode WebM Opus. This breaks all existing web content using Web Audio with support for WebM Opus, which includes all content published with Construct 3 (www.construct.net) since 2017.

The reason for this is there is no API to feature detect which audio formats are supported by Web Audio specifically. So in practice all web content uses HTMLMediaElement.canPlayType() to detect which formats are supported for Web Audio. This has worked for years since in all other browsers decodeAudioData() has always supported the same formats as <audio>. However in Safari 15 canPlayType() indicates WebM Opus is playable, but that misleads existing web content in to trying to decode WebM Opus with decodeAudioData(), which doesn't work.

In our case if WebM Opus is detected as not playable, we fall back to a WebAssembly decoder. But since Safari 15 indicates support for WebM Opus, our code skips the WebAssembly decoder and tries to use decodeAudioData() directly, which fails. Since no feature detection API exists for Web Audio, it's also difficult to detect this case.

Minimal repro: https://downloads.scirra.com/labs/audioCodecTests/
Look under the "WebM Opus" section. <audio> support is indicated via "HTML5 audio support: probably". However decodeAudioData support is missing via "Web Audio API decodeAudioData() support: No (error: null)".

Real-world example: https://downloads.scirra.com/labs/kiwistory/
In Safari 14.x and older, both music and sound effects play fine. In Safari 15, the music plays (as it uses <audio>), but no sound effects play (as those use Web Audio). The console logs decoding errors as it has attempted to use decodeAudioData() with WebM Opus, which failed.

Therefore users upgrading to Safari 15 will find Web Audio content regresses and stops playing sound.

Even if we deploy a workaround in our engine, as noted there is years of content already published, and much of it is unlikely to be updated, so many Safari 15 users will still find audio regresses and stops working. So it's important this is fixed before Safari 15 is fully released.

In general, Safari must support all the same formats with <audio> and decodeAudioData() in order to be compatible with existing Web Audio content, since canPlayType() is also the de-facto feature detection API for decodeAudioData().

[Radar WebKit Bug Importer]

<rdar://problem/79218243>

[Jean-Yves Avenard [:jya]]

It’s actually in the spec, that decodeAudioData :  "Audio file data can be in any of the formats supported by the audio element."
ref: https://webaudio.github.io/web-audio-api/#dom-baseaudiocontext-decodeaudiodata

[Jean-Yves Avenard [:jya]]

Created attachment 431831 [details]
Patch

[Jean-Yves Avenard [:jya]]

Created attachment 431833 [details]
Patch

[Eric Carlson]

Comment on attachment 431833 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=431833&action=review

> Source/WebCore/html/HTMLMediaElement.cpp:1049
> +    // Temporary work around bug 226922. For now claim that the opus and vorbis codecs aren't supported

s/Temporary/Temporarily/

[Jean-Yves Avenard [:jya]]

Created attachment 431945 [details]
Patch

apply comment

[EWS]

Committed r279103 (239020@main): <https://commits.webkit.org/239020@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 431945 [details].

[Jean-Yves Avenard [:jya]]

*** Bug 227198 has been marked as a duplicate of this bug. ***

When will this fix be published? I noticed it was closed way back in June. Seems like it would've been published sooner since it's marked as critical.

[Jean-Yves Avenard [:jya]]

(In reply to sam from comment #9)
> When will this fix be published? I noticed it was closed way back in June.
> Seems like it would've been published sooner since it's marked as critical.

It is now fixed.

The fix is now available in Safari Technology Preview 132 and later.