REGRESSION (iOS 14.2): Can't play html audio with muted attribute

• Component: Media
• Status: RESOLVED FIXED

[Kyungmin]

Created attachment 414883 [details]
distorted playtime ( original time is 408 )

Audio plays well with the muted attribute in versions below ios 14.2. (In muted state, but time of music continues.)

However, since iOS 14.2, audio has been acting strangely.

If audio is played with the muted attribute applied, the audio stops immediately. 

If the audio is played again, it stops immediately and the audio duration is distorted. ( like 0.0006 )

I attach jsfiddle and screenshot. ( Testing in ios 14.2, you can find issue )

https://jsfiddle.net/BangKyungmin/42mwdgqv/42/

[Kyungmin]

I tested it at ios 14.3 beta2 safari, but still can not play

[Jer Noble]

This could be a regression from http://trac.webkit.org/changeset/266844/webkit, "[Cocoa] PERF: Don't instantiate AVPlayer-based audio decoders or renderers if an element is initially muted."

Trying to determine the severity here; Kyungmin, what is your use case for playing a muted mp3 file?

(MP3 files themselves are strange, in that the duration of the file is not encoded into the metadata of the file; to discover the duration you have to run the decoder.)

[Kyungmin]

(In reply to Jer Noble from comment #2)
> This could be a regression from
> http://trac.webkit.org/changeset/266844/webkit, "[Cocoa] PERF: Don't
> instantiate AVPlayer-based audio decoders or renderers if an element is
> initially muted."
> 
> Trying to determine the severity here; Kyungmin, what is your use case for
> playing a muted mp3 file?
> 
> (MP3 files themselves are strange, in that the duration of the file is not
> encoded into the metadata of the file; to discover the duration you have to
> run the decoder.)

We're making a chat-type novel, and we're letting background music come out at a certain point in time. However, all the media is in a muted state to prevent users from being surprised. The background music is based on the flow of the novel (playtime, atmosphere, etc.). At this time, users can infer the flow of this novel from the time of background music, and expect to hear the background music that fits the part they are currently reading by turning off the muted at any time.
However, the background music is not playing at the moment and the total time of the background music is not known, so it is not possible to display background music information to the user normally. The total duration is 1 second (we use the Math.ceil function), making the user confused as if it was a 1 second BGM. Because of this, we are using hacks that force us to release the mud and then play, which is a really bad development experience.

[Kyungmin]

(In reply to Kyungmin from comment #3)
> (In reply to Jer Noble from comment #2)
> > This could be a regression from
> > http://trac.webkit.org/changeset/266844/webkit, "[Cocoa] PERF: Don't
> > instantiate AVPlayer-based audio decoders or renderers if an element is
> > initially muted."
> > 
> > Trying to determine the severity here; Kyungmin, what is your use case for
> > playing a muted mp3 file?
> > 
> > (MP3 files themselves are strange, in that the duration of the file is not
> > encoded into the metadata of the file; to discover the duration you have to
> > run the decoder.)
> 
> We're making a chat-type novel, and we're letting background music come out
> at a certain point in time. However, all the media is in a muted state to
> prevent users from being surprised. The background music is based on the
> flow of the novel (playtime, atmosphere, etc.). At this time, users can
> infer the flow of this novel from the time of background music, and expect
> to hear the background music that fits the part they are currently reading
> by turning off the muted at any time.
> However, the background music is not playing at the moment and the total
> time of the background music is not known, so it is not possible to display
> background music information to the user normally. The total duration is 1
> second (we use the Math.ceil function), making the user confused as if it
> was a 1 second BGM. Because of this, we are using hacks that force us to
> release the mud and then play, which is a really bad development experience.

(In reply to Jer Noble from comment #2)
> This could be a regression from
> http://trac.webkit.org/changeset/266844/webkit, "[Cocoa] PERF: Don't
> instantiate AVPlayer-based audio decoders or renderers if an element is
> initially muted."
> 
> Trying to determine the severity here; Kyungmin, what is your use case for
> playing a muted mp3 file?
> 
> (MP3 files themselves are strange, in that the duration of the file is not
> encoded into the metadata of the file; to discover the duration you have to
> run the decoder.)


I think there's a problem with this work. 
It's not that I don't understand the intent of the work, but I think it's problematic to make the user aware like a bug.
In fact, I posted this question on Stackoverflow, and the answer was, "That's like a bug, so report a bug to the webkit."

I think it's strange that the myted state affects playback. Even if you intended it, wouldn't it feel like a bug if the user took a play action and it didn't play?

[Eric Carlson]

(In reply to Kyungmin from comment #4)
> 
> I think there's a problem with this work. 
> It's not that I don't understand the intent of the work, but I think it's
> problematic to make the user aware like a bug.
> In fact, I posted this question on Stackoverflow, and the answer was,
> "That's like a bug, so report a bug to the webkit."
> 
> I think it's strange that the myted state affects playback. Even if you
> intended it, wouldn't it feel like a bug if the user took a play action and
> it didn't play?

It is a bug, the muted state shouldn't affect duration.

Setting the volume to an inaudible value, say 0.01, and setting muted to false might work around the bug while we work on a proper fix.

[Kyungmin]

(In reply to Eric Carlson from comment #5)
> (In reply to Kyungmin from comment #4)
> > 
> > I think there's a problem with this work. 
> > It's not that I don't understand the intent of the work, but I think it's
> > problematic to make the user aware like a bug.
> > In fact, I posted this question on Stackoverflow, and the answer was,
> > "That's like a bug, so report a bug to the webkit."
> > 
> > I think it's strange that the myted state affects playback. Even if you
> > intended it, wouldn't it feel like a bug if the user took a play action and
> > it didn't play?
> 
> It is a bug, the muted state shouldn't affect duration.
> 
> Setting the volume to an inaudible value, say 0.01, and setting muted to
> false might work around the bug while we work on a proper fix.

Thank you for the good news. I'm looking forward to a quick fix :)

[Radar WebKit Bug Importer]

<rdar://problem/71863148>

[Jer Noble]

Created attachment 418899 [details]
Patch

[Jer Noble]

The patch should address the issue you're seeing with muted .mp3 files. But I would really encourage you to try to refactor your apps logic to not play muted audio tracks; it is a constant drain on users' batteries, even when the Safari tab is in the background, which is something we may need to address in a future update.

[EWS]

Committed r272185: <https://trac.webkit.org/changeset/272185>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 418899 [details].