Bug 221754 – iOS 14: HLS live stream playback is getting extremely jerky permanently while microphone capture is enabled via getUserMedia

• Status: NEW
• Component: Media

HLS live stream playback is getting extremely jerky permanently while microphone capture is enabled via getUserMedia. 
It is known that calling getUserMedia can cause short interruptions of the media playback. 
This is not the subject of this report. 

Severity 
The issue is business critical for us as it's disabling related use cases. 

Use case 
Audio-only or audio plus video webrtc upstream while watching a one-to-many HLS live stream 

Scope of the issue 

iOS versions 
The issue is related to iOS 14/Safari 14 and has not been recreated or reported for iOS 13 or earlier. 
We have recreated the problem on iOS 14.3, 14.4 and 14.5 public beta (18E5140k). 
We have received a report for iOS 14.2. We have no reliable knowledge about earlier iOS 14 versions atm. 

iOS devices 
The problem is reliably recreatable on: 
* iPhone 12 
* iPhone 11 
* iPhone XR 
The problem is not recreatable on: 
* iPhone X
* iPhone SE 2020

Audio devices 
The problem is recreatable with: 
* The internal microphone and speaker 
* Airpods pro headset with iPhone 12 in one setup 
* A JBL bluetooth headset with iPhone 12 in one setup 

The problem has not been recreatable with: 
* Wired earpods including microphone in all setups 
* External bluetooth speakers in all tests 
* Airpods pro headset with iPhone 11 in one setup 
* Airpods (external test, less reliable) 

Web page workflow 

One VideoElement is playing the HLS live stream unmuted. 
Tested codecs: H.264/AAC
Tested multiplex: fMP4 and ts 
Tested stream sources: Own service and youtube live 
Apple LL-HLS: no 

Audio capture is being enabled via getUserMedia. 
Including/excluding camera capture had no effect. 
Problem does not occur with camera-only capture. 
Tested several audio constraints without effect including: 
* Echo cancellation
* Auto gain
* Noise suppression
Adding/removing a MediaElement to preview the user media stream has no effect. 

The order of playback and capture start had no effect. 
Our use case is to start the playback first. 
No actual webrtc streams required to reproduce. 
Apart from the HLS playback there is no additional MediaElements or WebAudio involved. 

Description of the appearance 

As soon as the audio capture is getting active, video and audio of the 
media playback is getting very jerky, sometimes stalled, sometimes with 
very low update rates of 1-2 fps. Audio is often completely inaudible, 
sometimes very short bursts of audio occur. In two tests the playback recovered 
after around 5 minutes, in other the issue has been persistent. 

Disabling and enabling the capture session in the browser address bar is 
reliably recovering and reintroducing the issue. 

Sometimes the media playback can be recovered by applying pause/play, 
but this seems to be very unreliable. Furthermore the issue can reoccur 
either without interaction or forced by e.g. 
* Changing the mute state of the playback, even from unmuted to muted 
* Entering or exiting fullscreen mode
* Sometimes by tapping, scrolling or zooming the page 

How to recreate 
* Test page requires a getUserMedia call with audio:true and a VideoElement with live HLS source 
* Start video playback unmuted 
* Activate audio capture

The inconsistent results with Airpods Pro seemed to be related to different Airpod settings. 

Turning off "spatial audio" seems to prevent the issue. 
Assigning the microphone to either left or right Airpod too.

[Radar WebKit Bug Importer]

<rdar://problem/74477329>

The problem persists in iOS 14.5 beta 2 (18E5154f).

[youenn fablet]

https://jsfiddle.net/y8qfbcs1/ reproduces the issue:
- start2 is showing the issue as described
- start1 is pausing video and restarting video two seconds after audio capture started. This seems to work on my setup.

[youenn fablet]

Muting/unmuting microphone clearly affects the video rendering.

The problem persists in iOS 14.5 beta 3 (18E5164h).

The problem persists in iOS 14.5 (18E199).

iOS 14.5: We have been able to recreate the same issue with live http/mp3 feeds, e.g. web radio, without HLS being involved.

(In reply to youenn fablet from comment #5)
Thanks for confirming, we experience this issue as well with HLS. Can you confirm that the issue is in WebKit? Is there an internal ticket to resolve this bug?