Bug 262781 – Silent mode plays back compressed audio, but not uncompressed audio

• Status: NEW
• Component: Web Audio

STR:

1. Visit iPhone to http://clb.confined.space/audio_test_suite/
2. Open the first entry 8bit_detective

TEST 1, UNCOMPRESSED PLAYBACK:

3. Flip the switch on the side of the iPhone to make sure the phone is *not* in silent mode. (I am testing on iPhone Xs that still has physical silent/ringer switch, I recall newer iPhones moved to an Action button or something else?)
4. Click on the first entry under "Play uncompressed:" heading.
4b. Observe that audio plays back properly.

5. Switch iPhone to silent mode (the physical switch on the side/via Action button)
5b. Observe that audio will now go silent.

TEST 2, COMPRESSED PLAYBACK:

6. Flip the switch on the side of the iPhone to make sure the phone is *not* in silent mode.
7. Click on the first entry under "Play compressed:" heading.
7b. Observe that audio plays back properly.

8. Switch iPhone to silent mode
8b. Observe that audio will still play back.

EXPECTED:

In 5b and 8b, audio should either go silent in both cases, or in neither.

Silencing only uncompressed audio in silent mode makes little sense.

Not sure if iOS 17 has the same issue, I don't have a device at hand to test.

[Alexey Proskuryakov]

I think that this is about Web Audio vs <audio>, not about compressed vs. uncompressed, and is actually by design.

Folks CC'ed would know for certain.

Thanks for the reply.

In the test case, <audio> nodes are played in the Web Audio graph by using https://developer.mozilla.org/en-US/docs/Web/API/MediaElementAudioSourceNode , and not direct <audio> playback.

Unity uses this technique to play back audio in web games. In a typical game, some audio clips will use compressed playback path, and others will use an uncompressed audio playback path.

Different use cases require users to choose between the two. To give you examples, in a native (non-web) game engine, the typical distinction is:

1) all audio clips should be primarily considered to be played back as compressed so they will save the most RAM, except for
 - 2) very short audio clips (footstep sounds, gunshots, similar) would want to use uncompressed audio (or maybe ADPCM, which Web Audio unfortunately does not support), because setting up the decompressor for many very small clips has memory and performance overhead of its own that outweighs the benefit.
 - 3) playing back multiple compressed audio files simultaneously can have a noticeable CPU overhead, so one might use uncompressed playback for some clips in such case, to balance CPU usage vs RAM usage
 - 4) compression quality might be lossy, so to ensure highest quality audio one might go for uncompressed files.

The above is a typical recommendation for memory and performance management e.g. for Unity games.

On the web, there unfortunately additionally exists a random assortment of Web Audio specification issues that were deemed unfixable by the spec working group, which give the following additional requirements to the user for choosing compressed vs uncompressed:

A) if playback pitch needs animation (e.g. a car engine revving up effect, or a 3D doppler shift effect), audio needs to be played back uncompressed, because compressed <audio> does not support this kind of pitch animation
B) if audio playback requires custom loop points (e.g. maybe X seconds of an intro/prelude part, followed by a middle part that loops N times, then maybe Y seconds of an outro part), then uncompressed audio playback must be used, because <audio> does not enable custom loop points, only full start-to-finish looping.
C) if looping should be seamless, then compressed <audio> cannot be used, because <audio> does not guarantee seamless looping -> only uncompressed playback does.

Hopefully the above examples illustrate how there already exist several constraints to when users should use compressed vs uncompressed audio.

So if it is true that it is by design that uncompressed audio playback is subject to iPhone's silent mode, but compressed audio playback is not, I hope you can now see how impossible navigating all these constraints simultaneously for game audio development on the web for Safari would become? Game developers would never want a seemingly arbitrary (to the end user) mix of their game audio to be silenced when the user places their device in silent mode.

I hope there would be a better design to fix this issue, maybe by Apple working with W3C to come up with boolean AudioNode.shouldPlaybackEvenIfInSilentMode and HTMLAudioElement.shouldPlaybackEvenIfInSilentMode properties on audio nodes, or maybe by Apple adding a Safari-specific boolean webkitShouldPlaybackEvenIfInSilentMode property on AudioNode and HTMLAudioElement?

That would allow Unity developers to choose to either mute all their audio when in silent mode (e.g. Unity developer developing a web game), or to have all their audio unmuted even when in silent mode (e.g. Unity developer writing a video or an audio player or some other business application, e.g. 3D virtual chatrooms and such).

[Radar WebKit Bug Importer]

<rdar://problem/116916376>