Bug 226922 - Safari 15 breaks all Web Audio content using WebM Opus
Summary: Safari 15 breaks all Web Audio content using WebM Opus
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Audio (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Critical
Assignee: Jean-Yves Avenard [:jya]
Keywords: InRadar
: 227198 (view as bug list)
Depends on:
Reported: 2021-06-11 08:32 PDT by Ashley Gullen
Modified: 2021-06-29 06:05 PDT (History)
17 users (show)

See Also:

Patch (1.91 KB, patch)
2021-06-20 23:28 PDT, Jean-Yves Avenard [:jya]
no flags Details | Formatted Diff | Diff
Patch (1.92 KB, patch)
2021-06-21 00:41 PDT, Jean-Yves Avenard [:jya]
no flags Details | Formatted Diff | Diff
Patch (1.92 KB, patch)
2021-06-21 21:19 PDT, Jean-Yves Avenard [:jya]
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ashley Gullen 2021-06-11 08:32:47 PDT
Thanks for adding WebM Opus support in Safari 15! We've been waiting for this for a long time. However currently it is supported only for the HTML <audio> element. Web Audio's decodeAudioData() still fails to decode WebM Opus. This breaks all existing web content using Web Audio with support for WebM Opus, which includes all content published with Construct 3 (www.construct.net) since 2017.

The reason for this is there is no API to feature detect which audio formats are supported by Web Audio specifically. So in practice all web content uses HTMLMediaElement.canPlayType() to detect which formats are supported for Web Audio. This has worked for years since in all other browsers decodeAudioData() has always supported the same formats as <audio>. However in Safari 15 canPlayType() indicates WebM Opus is playable, but that misleads existing web content in to trying to decode WebM Opus with decodeAudioData(), which doesn't work.

In our case if WebM Opus is detected as not playable, we fall back to a WebAssembly decoder. But since Safari 15 indicates support for WebM Opus, our code skips the WebAssembly decoder and tries to use decodeAudioData() directly, which fails. Since no feature detection API exists for Web Audio, it's also difficult to detect this case.

Minimal repro: https://downloads.scirra.com/labs/audioCodecTests/
Look under the "WebM Opus" section. <audio> support is indicated via "HTML5 audio support: probably". However decodeAudioData support is missing via "Web Audio API decodeAudioData() support: No (error: null)".

Real-world example: https://downloads.scirra.com/labs/kiwistory/
In Safari 14.x and older, both music and sound effects play fine. In Safari 15, the music plays (as it uses <audio>), but no sound effects play (as those use Web Audio). The console logs decoding errors as it has attempted to use decodeAudioData() with WebM Opus, which failed.

Therefore users upgrading to Safari 15 will find Web Audio content regresses and stops playing sound.

Even if we deploy a workaround in our engine, as noted there is years of content already published, and much of it is unlikely to be updated, so many Safari 15 users will still find audio regresses and stops working. So it's important this is fixed before Safari 15 is fully released.

In general, Safari must support all the same formats with <audio> and decodeAudioData() in order to be compatible with existing Web Audio content, since canPlayType() is also the de-facto feature detection API for decodeAudioData().
Comment 1 Radar WebKit Bug Importer 2021-06-11 15:10:11 PDT
Comment 2 Jean-Yves Avenard [:jya] 2021-06-16 16:50:56 PDT
It’s actually in the spec, that decodeAudioData :  "Audio file data can be in any of the formats supported by the audio element."
ref: https://webaudio.github.io/web-audio-api/#dom-baseaudiocontext-decodeaudiodata
Comment 3 Jean-Yves Avenard [:jya] 2021-06-20 23:28:41 PDT
Created attachment 431831 [details]
Comment 4 Jean-Yves Avenard [:jya] 2021-06-21 00:41:06 PDT
Created attachment 431833 [details]
Comment 5 Eric Carlson 2021-06-21 10:09:10 PDT
Comment on attachment 431833 [details]

View in context: https://bugs.webkit.org/attachment.cgi?id=431833&action=review

> Source/WebCore/html/HTMLMediaElement.cpp:1049
> +    // Temporary work around bug 226922. For now claim that the opus and vorbis codecs aren't supported

Comment 6 Jean-Yves Avenard [:jya] 2021-06-21 21:19:15 PDT
Created attachment 431945 [details]

apply comment
Comment 7 EWS 2021-06-21 21:57:31 PDT
Committed r279103 (239020@main): <https://commits.webkit.org/239020@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 431945 [details].
Comment 8 Jean-Yves Avenard [:jya] 2021-06-29 06:05:48 PDT
*** Bug 227198 has been marked as a duplicate of this bug. ***