Bug 222225 – 'Play' buttons for 'music playing' web app not working at all on iPhone 12 Pro running iOS 14.4

• Status: NEW
• Component: UI Events

[Richard Nash]

Created attachment 421086 [details]
basic infographic with code pics explaining the issue with images.

# Overview

I am working on a Music Player web app using the web framework called 'Svelte' and have made an online Svelte REPL to create the app, which can be found here:

https://svelte.dev/repl/56928dbff1b141a39f635522e7113f3c?version=3.32.3

The REPL runs perfectly on almost every browser I have tested it on, including all the latest versions of desktop Safari, Chrome, Firefox, Edge and Opera while running macOS 10.15.7. The web app also works on several mobile devices I have tested it on, or have asked to have it tested on, including Android Chrome and Android Samsung Internet Browser, running Android 8.2, and 11. I have even had it tested on iOS devices and it runs fine in iOS Safari and Chrome and Firefox on an iPhone 6, and 11.

The one environment that it does not seem to work in is on an iPhone 12 Pro running iOS 14.4. On three separate iPhone 12 Pro devices running iOS 14.4 the same thing happens...

# Steps to reproduce

1. Open the REPL using the URL provided above on an iPhone 12 Pro running iOS Safari 14.4 (Same issue in iOS Chrome, iOS Firefox, iOS Brave, iOS Opera Mini, which are all the iOS browsers that I have tested on my device.)
2. Scroll the window of the REPL down to switch from the code (input) to the app (output).
3. Press any of the 'Play" buttons for any of the tracks listed, other than than main play button in the controls, and you will get two errors in the console and the music never plays.

# Exptected Result

Music Plays.

# Actual Result

Music dos not play.

# Additional Information

I am attaching a basic infographic with pictures of code snippets and a very brief explainers to help clarify the issue. But it is easy to reproduce, just follow the steps listed above.

It would be amazing if this was figured out and fixed :-)

Thank you very much for your time and effort in the matter,


Richard Nash

[Richard Nash]

Tested on the following family and friends devices: iOS 14.4 AND iPhone 12 Pro consistently break...everywhere else is fine:

# Android using Chrome

- Unknown Android Phone - Works (3rd Party Source, not fully tested?) 😁
- Umidigi One Pro - Android 8.1.0 - Works (3rd Party Source, not fully tested?) 😁
- Samsung Note 20 - Android 11 / Android Chrome - Works 😁
- Samsung S21 - Android 11/ Android Chrome- Works 😁
- Pixel 3a - Android 11/ Android Chrome - Works (Ailene’s Phone) 😁

# Other iPhones

- iPhone SE - iOS 13.6.1 - Works (3rd Party Source, not fully tested?) 😁
- iPhone 11 - iOS 14.4 - Works (Thomas’ Phone) 😁
- iPhone 6s Plus - iOS 14.4 - Works (Grandma’s phone) 😁
- iPhone 7 Plus - iOS 14.4 - Works (Grandpa’s Phone) 😁

# iPhone 12 Pro's

- iPhone 12 Pro - iOS 14.4 - No Work  (Richard’s Phone) 🤬
- iPhone 12 Pro - iOS 14.4 - No Work (Gia’s Phone) 🤬
- iPhone 12 Pro - iOS 14.4 - No Work (Corona’s’s Phone) 🤬

[Richard Nash]

Here is a version of the little web app outside of the svelte REPL:

http://svelte-music-player.surge.sh

[Alexey Proskuryakov]

This doesn't feel like an issue that would be hardware model specific. My guess would be that iPhone 12 Pro is the fastest device tested, so some race in the code plays out differently.

[Richard Nash]

is there any difference in the version of safari and webkit that each phone receives? And how would I be able to test for a race like that? Any thoughts?

[Alexey Proskuryakov]

(In reply to Richard Nash from comment #4)
> is there any difference in the version of safari and webkit that each phone
> receives? 

No.

> And how would I be able to test for a race like that? Any thoughts?

Reducing to a small reproducible case can be one approach. Less mechanically, adding artificial slowdowns in affected code paths sometimes sheds light on what's going on, too.

[Richard Nash]

working on really small reproductions now...another person in the svelte discord tested the REPL on their iPhone 12 mini running iOS 14.3 and they said everything worked fine for them...does that change your perception of the issue at all, considering that the mini uses the same processor as the pro? Or are there other difference in speed that could lead to a race?

[Alexey Proskuryakov]

This seems like a somewhat odd data point because we don't have any others from latest iPhones on pre-14.4 OS? In any case, reducing the test continues to appear as the best path forward.

[Radar WebKit Bug Importer]

<rdar://problem/74822299>

[Jer Noble]

It looks like that, when playback stalls, the audio element's readyState never progresses beyond HAVE_CURRENT_DATA, even though the buffered range indicates a full 7 minutes of data (the entire duration) available.

> $0.duration
< 436.19265306122446
> $0.buffered.end(0)
< 436.19265306122446
> $0.readyState
< 2

[Jer Noble]

And it looks like the underlying player _does_ progress to HAVE_ENOUGH_DATA, but something is keeping that change from being reflected in readyState:

[Log] HTMLMediaElement::setReadyState(2277DA2F2B6F5A43) new state = HaveEnoughData, current state = HAVE_METADATA
> $0.readyState
< 2

[Jer Noble]

The only situation where HTMLMediaElement::setReadyState() would enter with HaveEnoughData and leave with HAVE_CURRENT_DATA is if the media element thought there were non-loaded text tracks:

```
    if (tracksAreReady)
        m_readyState = newState;
    else {
        // If a media file has text tracks the readyState may not progress beyond HAVE_FUTURE_DATA until
        // the text tracks are ready, regardless of the state of the media file.
        if (newState <= HAVE_METADATA)
            m_readyState = newState;
        else
            m_readyState = HAVE_CURRENT_DATA;
    }
```

And indeed:

```
> $0.textTracks.length
< 1
> $0.textTracks[0]
< TextTrack {id: "", kind: "captions", label: "", language: "", inBandMetadataTrackDispatchType: "", …}
```

Somehow, this particular file is advertising that it has a captionsTrack!

```
> $0.currentSrc
< "https://sveltejs.github.io/assets/music/beethoven.mp3"
```

However, as an in-band text track, it should be fully loaded when the file itself is fully loaded, so that's not a complete explanation for this behavior.

[Jer Noble]

Aha! It's not an in-band track; there's literally a `<track kind="captions">` with no src= attribute in the DOM. And a track with no src can never progress to a loaded state, and blocks it's parent <audio> element from getting to a .readyState > HAVE_CURRENT_DATA.

There's probably still a bug here; we should probably not include src-less <track> elements when considering whether to block updates to the readyState, but at the  very least there's a client-side workaround available: don't add an empty caption track to the DOM.

[Richard Nash]

holy doodle spanks! You solved it!

Svelte yells at us in the REPL to add:

<track kind="captions">

...to all media elements for a11y reasons, it says, so I did what it told me to do.

Thinking about it, though, this is an instrumental audio track, not sure how captions would be of a benefit here...I get it for audio with vocals of some kind, but I also don't know too much about a11y, obviously...

Anyway, thank you Jer Noble! It works now...

But, like you said, maybe don't block <audio> elements with empty captions tracks from playing, or offer an error message explaining the issue? IDK what would be the best solution going forward, but...well...

Thanks again!

[Eric Carlson]

(In reply to Richard Nash from comment #13)
> holy doodle spanks! You solved it!
> 
> Svelte yells at us in the REPL to add:
> 
> <track kind="captions">
> 
> ...to all media elements for a11y reasons, it says, so I did what it told me
> to do.
> 
Captions *are* generally a good idea, but a track without cues won't help much ;-)

[Richard Nash]

that's kind of what I was thinking...is there an a11y recommendation for instrumental audio tracks? Should there be a <track kind"____"> in any way? What would be useful for people who are hearing impaired when listening to an instrumental audio track? I dunnoooo...

[Eric Carlson]

(In reply to Richard Nash from comment #15)
> that's kind of what I was thinking...is there an a11y recommendation for
> instrumental audio tracks? Should there be a <track kind"____"> in any way?
> What would be useful for people who are hearing impaired when listening to
> an instrumental audio track? I dunnoooo...

The problem is that a <audio> element has no intrinsic visual output, so there is no place in the page to display caption/subtitles. We discussed this when defined the <track> element, and decided that it was OK because a author that wants to provide captions for audio-only content can put use a <video> element with an audio-only url.