Bug 224874 – Stuttering audio from WebRTC in iOS 14.4.2

• Component: WebRTC
• Status: NEW

[Jeremy Noring]

Created attachment 426698 [details]
Here's an extreme example of how bad it sounds.  I'd love for people to confirm this is what they're hearing.

A (likely very related) defect:

https://bugs.webkit.org/show_bug.cgi?id=218762

We have an application that can leverage both opentok and twilio backends to create video conferences via WebRTC.  In safari, on iOS, and mostly on 14.4 and up, our users are suffering from severe audio issues (I'd describe the output as "choppy", "robotic" or "stutter-y"; I'm happy to post an example of what it sounds like).  We've had a very hard time pinning down a reliable set of reproduction steps; the running theories include A) it may be more likely to reproduce on low-powered devices, devices in low-power mode, or devices where the page javascript is CPU intensive, B) it may have something to do with adverse network conditions (jitter, most likely), or C) ???.

I've reproduced this on opentok's demo page, in our own app using opentok (multiple versions, ranging from 2.19.0 up to 2.19.3), on mediasoup's test page, and even on appr.tc (reproduces less commonly). Some coworkers of mine have more difficulty reproducing it.

Here's the best set of reproduction steps I've seen thus far:

1. Visit opentok's demo page: https://opentokdemo.tokbox.com/ on an iOS device running 14.4.2 (I use an iPhone X)
2. Have another user join the same page via Chrome Desktop (our users are typically on Windows, but I've reproduced on macOS)
3. Both should be able to see and hear one another.  Duration of the discussion may matter.
4. The simplest way I've seen to reproduce the issue at this point is to pull down the control center menu on the iPhone (swipe down from the upper right corner), and/or background the safari tab.  Duration of the conference may matter, whether or not the phone is in low-power mode may matter, and network may matter, but... for some percentage of users, the audio ranges from static-y, to completely unusable.

Referring back to https://bugs.webkit.org/show_bug.cgi?id=218762 for a moment... I believe that jsfiddle posted by juberti is fixed, but there's something else at play here.

[youenn fablet]

Hi Jeremy, since you can reproduce, can you send me privately a sysdiagnose ([e-mail]), I would like to look at some logs.

[Radar WebKit Bug Importer]

<rdar://problem/76962873>

[youenn fablet]

Would be interesting to see if you can repro in the latest iOS 14.5 beta as well.

[youenn fablet]

Thanks for the example of how bad it looks. This seems like a different issue from https://bugs.webkit.org/show_bug.cgi?id=218762

[Jeremy Noring]

Will do, Youenn(In reply to youenn fablet from comment #1)
> Hi Jeremy, since you can reproduce, can you send me privately a sysdiagnose
> ([e-mail]), I would like to look at some logs.

Will do.  Standby.

[Jeremy Noring]

I have a bit more info--I think device may matter?  Seems like iPhone X reproduces fairly reliably, and iPhone 12 doesn't.  Strangely enough I have a 6s that doesn't reproduce reliably.

[youenn fablet]

Thanks Jeremy.
Tried on an iPhone Xs and it does not reproduce either apparently.

[youenn fablet]

Provided repro case from https://bugs.webkit.org/show_bug.cgi?id=225880.

> Note: if you just comment out line 41 within
> https://jsfiddle.net/ericamram/afhzbcu2/48/ 
> 
>   // createConnection(dest.stream, audioElem);
> 
> your voice (from getUserMedia) will sound perfect, which shows that it's the
> combo WebAudio + WebRTC that generates stuttering.

[youenn fablet]

*** Bug 225880 has been marked as a duplicate of this bug. ***

[Marc]

Just to elaborate on my own setup ( continuing from bug #225880 ), I am using an iPhoneX ( MQAN2LL/A ) with iOS 14.5.1. No headphones. I have encountered this issue for at least 5 months, so not a newly introduced issue. Audio begins to stutter, and in many cases very badly. It always feels tied to CPU or heat generated by the phone (after a WebRTC session is going for a short bit). Killing Safari and restarting it results in the same issue, but will reset to normal if I do not use the phone for a few minutes. It will start again after testing. Video playback is fine during all of it. I am using Opus Audio. It effects all the incoming RTC Audio streams when it is happening.  It does not effect the outgoing microphone Opus stream.

[Teodor]

Allow me to interject.

We were experiencing bug 218762 in our application back in November 2020. It seemed to have been resolved, and we didn't hear of it from users for a few months.

Now it is back. The attachment 426698 [details] from Jeremy sounds exactly like bug 218762 sounded back in 2020. So I would say that whatever the fix for bug 218762 was, it just alleviated the symptoms, but the underlying issue remained, and now it just resurfaced after some code was change somewhere else.

While this does not affect the captured audio (aka audio sent from an iPhone to all other participant) directly, it does sometimes get picked up by the microphone (when headphones are not being used), and echo/noise cancellation algorithms have a hard time isolating and removing the static from the stream, so the stuttering makes its way to all participants of the call.

[Eric]

@Teodor:
Does it behave the same on a fresh Safari (just killed and relaunched)
vs when you reload the page?

  

(In reply to Teodor from comment #11)
> Allow me to interject.
> 
> We were experiencing bug 218762 in our application back in November 2020. It
> seemed to have been resolved, and we didn't hear of it from users for a few
> months.
> 
> Now it is back. The attachment 426698 [details] from Jeremy sounds exactly
> like bug 218762 sounded back in 2020. So I would say that whatever the fix
> for bug 218762 was, it just alleviated the symptoms, but the underlying
> issue remained, and now it just resurfaced after some code was change
> somewhere else.
> 
> While this does not affect the captured audio (aka audio sent from an iPhone
> to all other participant) directly, it does sometimes get picked up by the
> microphone (when headphones are not being used), and echo/noise cancellation
> algorithms have a hard time isolating and removing the static from the
> stream, so the stuttering makes its way to all participants of the call.

[Teodor]

@Eric, just like the first time we've encountered this bug last year, it is difficult to reproduce reliably. There is little pattern, if any, to how and when it happens for us.

This is from last week. It occurred to one of the participants using iPhone 11 in a group call (aka, there are multiple <video> elements being mixed together). The person connected their AirPods and the problem stopped – the sound they were hearing was clear (and we were not hearing the static picked up by the microphone any more).

When trying to reproduce this yesterday on iPhone 8 (14.7 Beta 1) or iPhone XR (14.6), I was not able to get it to break.

Historically, it either suddenly stopped after about one minute of stuttering, or it went on "forever". Then if you restart Safari, it may or may not happen again. If it didn't happen, then there is a high chance that it won't happen again for a while.

[Eric]

Hello,

What makes Webkit/Safari select AudioContext sampleRate as 44100 or 48000 Hz?

Could it be a reason of audio stuttering?

[Benard Mesander]

Created attachment 456716 [details]
Issue reproduced on iPhone 7, iOS 15.3.1

[Benard Mesander]

Issue does not reliably reproduce on every WebRTC call. It may require 10 or so tries to get it to happen. Backgrounding/foregrounding the browser may be a trigger, albeit delayed a few seconds.

[Benard Mesander]

I have only repro'd this on iPhone 7 and iPhone X - could the problem possibly be specific to certain hardware?

[youenn fablet]

Was anybody able to reproduce in iOS 15.4?

[Benard Mesander]

Yes, I have now reproduced it on iOS 15.4.1

[youenn fablet]

(In reply to Benard Mesander from comment #19)
> Yes, I have now reproduced it on iOS 15.4.1

Can you send a sysdiagnose ([e-mail]) with the exact time when the stuttering happens?

[Suhail]

Does anyone know how to solve it? I reproduced it on iphone14Plus(ios 17). I use Webrtc to receive the audio stream and use <Audio> to play it. I use mediaRecord to record the stream and find the stream is OK, but the audio from the iPhone speaker is very bad. At that time, getUserMedia got a bad stream.
I don't find a way to reproduce it stably. But it often happens after I switch to another APP that will play audio and back to Safari.

[Suhail]

Does anyone know how to solve it? I reproduced it on iphone14Plus(ios 17). I use Webrtc to receive the audio stream and use <Audio> to play it. I use mediaRecord to record the stream and find the stream is OK, but the audio from the iPhone speaker is very bad. At that time, getUserMedia got a bad stream.
I don't find a way to reproduce it stably. But it often happens after I switch to another APP that will play audio and back to Safari.