Bug 218762 - Stuttering audio from WebRTC in iOS 14.2
Summary: Stuttering audio from WebRTC in iOS 14.2
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: Safari 14
Hardware: iPhone / iPad Other
: P1 Normal
Assignee: Nobody
Keywords: InRadar
Depends on:
Reported: 2020-11-10 09:52 PST by Justin Uberti
Modified: 2021-01-07 01:25 PST (History)
20 users (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description Justin Uberti 2020-11-10 09:52:39 PST
Run https://jsfiddle.net/juberti/bnmfvjo0/ (which creates an oscillator, connects it to WebRTC, and plays out in an <audio> tag) and hit the play button.

On iOS 14.1 audio plays out normally. On iOS 14.2 the audio stutters immediately. 
Playout is normal on Safari Mac 14.0, and repros in STP 115 (14.1).

Does not repro if the oscillator is connected directly to <audio>.
Comment 1 Smoley 2020-11-10 15:23:41 PST
Thanks for filing Justin.

Apple Internal See:
Comment 2 Kevbo 2020-11-11 15:13:37 PST
We are also seeing this as well with our customers who have upgraded to iOS 14.2. This is a significant issue for us and impacts the usability of our product and needs a fix as soon as feasible.
Comment 3 Justin Uberti 2020-11-11 17:18:15 PST
Can anyone comment on where the root issue is so that a workaround can be attempted?
Comment 4 Kevbo 2020-11-11 17:40:41 PST
We have tried changing a few options such as audio bit rate and audio processing options with no luck. We are preparing an email to our customers right now, sadly, if we have any left after this.
Comment 5 Justin Uberti 2020-11-11 17:42:44 PST
The issue is in the playout path so those things won't help. Are you using Web Audio in your app?
Comment 6 Kevbo 2020-11-11 17:45:28 PST
We are using WebRTC for audio and video communications between two or more parties. We just had a report of this today and confirmed it was 14.2. None of our < 14.2 devices have the issue. All the 14.2 devices do. An device we upgrade to 14.2 then had the issue (didn't before the upgrade). It occurs 9 out of 10 times and the level of stutter often varies.
Comment 7 Teodor 2020-11-11 17:59:08 PST
We are also experiencing this. Both Safari and SFWebView.

We use WebAudio – new AudioContext() that is then attached to existing <audio> element. Removing context creation seemed to help initially (two consecutive tries, in both the crackling happened for less than a second). Third try it persisted for about 7 seconds and then cleared, so I am guessing this is not the right workaround.
Comment 8 Kevbo 2020-11-11 18:03:35 PST
Sometimes get the stutter for about 5 seconds then it goes away. But most of the time the stutter stays the entire time. Frustrating.
Comment 9 Don Borski 2020-11-11 19:58:46 PST
Comment 10 youenn fablet 2020-11-12 02:21:40 PST
The issue is indeed in the playback being too aggressive in its attempt to read.
We changed the behavior to be less aggressive.
This should be fixed in WebKit ToT.
Will post it when it will become testable in MacOS and iOS through STP or betas.
Comment 11 Teodor 2020-11-12 05:54:03 PST
@youenn Thank you for figuring out the problem.

Are there any workarounds that you can think of at this point? Would playing with playbackRate on <audio> element that has the AudioTrack attached be a good starting point? Or maybe implementing own buffer through Web Audio?
Comment 12 Kevbo 2020-11-12 07:56:52 PST
Yes, we are desperate for any possible workaround. We can easily change playback rate if that would help, oddly sometimes when it starts if we lower the volume (physically) and raise it a few times it goes away, but we tried doing that in software and it didn't do anything. We also tried unsubscribing to audio then subscribing many times to see if that would help (because 1 out of 10 times it doesn't stutter). We were up much of last night trying different things and we'll spend the rest of the day seeing if anything can mitigate the problem. We'll report back here if we find anything.
Comment 13 Justin Uberti 2020-11-12 11:56:42 PST
Youenn, can you link to the patch that addresses the issue so we can better understand the root cause? Some apps and devices seem to be unaffected, so would like to understand why.
Comment 14 youenn fablet 2020-11-12 12:05:42 PST
Fix is in https://github.com/WebKit/webkit/commit/d2beea459699aef7b58ed59cd6b435ee9d6a1b72.

The reduced fix is as follows:

diff --git a/Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm b/Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm
index ef7fd04391c5..9e6d5452571d 100644
--- a/Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm
+++ b/Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm
@@ -260,9 +260,8 @@ bool AudioSampleDataSource::pullSamplesInternal(AudioBufferList& buffer, size_t&
             if (timeStamp >= endFrame)
                 m_endFrameWhenNotEnoughData = endFrame;
         } else {
-            // We are too close from endFrame, let's back up a little bit.
-            uint64_t framesAvailable = endFrame - timeStamp;
-            m_outputSampleOffset -= sampleCount - framesAvailable;
+            // We are too close from endFrame, let's wait for more data to be pushed.
+            m_outputSampleOffset -= sampleCount;
             dispatch_async(dispatch_get_main_queue(), [logIdentifier = LOGIDENTIFIER, outputSampleOffset = m_outputSampleOffset, this, protectedThis = makeRefPtr(*this)] {
                 ALWAYS_LOG(logIdentifier, "updating offset to ", outputSampleOffset);
Comment 15 yuta moriyama 2020-11-12 20:34:13 PST
Before the fix has applied, is there another way to play normally?
like using audio api etc?
Comment 16 Alexey Proskuryakov 2020-11-12 20:50:35 PST
Could you please check 14.3 beta that was made available today?
Comment 17 Teodor 2020-11-12 21:52:11 PST
Still no workaround, but here is a short progress report on what we've tried:

- Trying to mix in white noise as dither through WebAudio API. However, they did not mix as expected in MediaStreamAudioDestinationNode. Furthermore, several times the white noise itself would stutter before any media arrived, but I assume this is purely due to CPU starvation when setting up the PeerConnection, and not related to this issue.

- Trying to delay the playback via DelayNode through WebAudio API. Regardless of the delay value, and whenever the value is increased, reduced or kept constant, the stuttering occurs whenever the stream is played back.

- Trying to forcefully restart offset computation by sending and/or receiving large DataChannel payloads at the beginning of the session. It didn't have an effect, and since audio is higher priority I doubt this can have an effect without SDP munging. Maybe something worth exploring.

- Trying to pause/mute/reattach the stream and trying to switch between devices at sender. Stuttering still occurs for "new" streams.

- Anything obvious like bitrate, codec and codec parameter manipulations. I guess dropping bitrate to zero would work, but picking it back up would likely trigger the same issue. Maybe worth experimenting.

What was not tried by us yet:

- Implementing "userland" buffer for audio frames and trying to counteract what the WebKit code is doing (skipping chunks of buffer) by repeating the frames at strategic offsets to force the continuous playback. Big problem with this approach, assuming it is even doable, is knowing when to stop - at some point the stuttering may end by itself (after reaching a stable buffer state) and we no longer want to be outputting duplicate frames any more.

- Attempting to capture video and audio separately, which may (or may not) allow us to forcefully desync audio and video tracks, allowing audio to catch up on its own. I am not sure this would work without also doing some SDP munging to deassociate video media lines from audio media lines. Another possibility is trying separate PeerConnections for video and audio streams.

Overall, based on the stats that we've captured, the issue only occurs if jitter goes above ~16ms (on our test bench), then once it goes down below ~12ms the stuttering disappears. This, of course, assumes that jitter at rest is at most around 10ms, otherwise it is theoretically possible to reach the edge of the buffer window again and get the stuttering back.

Skipping relay candidates, if you can, may help. At least for "nearby" connections, and ignoring other variable, as this largely depends on relay performance and whenever it is itself responsible for any jitter (for example if the relay does protocol switching between UDP and TCP). Stuttering is still there, but it stabilizes faster for us, which is better than never.
Comment 18 Kevbo 2020-11-13 09:41:22 PST

We also tried:

delaying playback
pause/mute/reattach the stream
setting bitrate to zero and increasing it (with several permutation of values in a loop)
disabling subscribing to audio/video then re-subscribing 

with no deterministic benefit.
Comment 19 Kevbo 2020-11-13 09:55:40 PST
Heads up, we tested:

14.3 beta 1 (last night)

and are still getting the stuttering.

@teodor: did you try the above and did it fix it for you?
Comment 20 Kevbo 2020-11-13 10:25:43 PST

Is there any way to change the "Status" to reflect, at least for us, the 
stuttering is still occurring even with the 14.3 beta 1 version?
Comment 21 youenn fablet 2020-11-13 10:59:56 PST
Reopening until we get validation we have a fix.
@Kevbo, you can privately send repro steps at youenn@apple.com.
Comment 22 Justin Uberti 2020-11-13 12:35:48 PST
Confirming this still repros on iOS 14.3 b1.
Comment 23 Kevbo 2020-11-13 12:39:48 PST
Thanks @justin - Glad to know it is not just us :)

@youenn - I think Justin already posted a way to repro in the original ticket:

Run https://jsfiddle.net/juberti/bnmfvjo0/ (which creates an oscillator, connects it to WebRTC, and plays out in an <audio> tag) and hit the play button.
Comment 24 Teodor 2020-11-13 14:15:29 PST

We also tried 14.3 beta 1. Same behavior across devices for remote WebRTC and local WebRTC from Justin's fiddle – for both it typically stabilized after between 10 and 60 seconds.

We haven't made any new workaround attempts.
Comment 25 Kevbo 2020-11-13 14:33:31 PST
@Teodor - thanks for the tip! we never tried waiting 60 seconds. I just tried that and it did stop after 60 seconds. We'll add this to our temporary remediation steps for our customers. Very appreciative you mentioned that. Hopefully it doesn't come back in the next 60 second interval ;)
Comment 26 yuta moriyama 2020-11-15 04:02:17 PST
this noise has happen only by iphone and not by ipad.
Is it true?

from my ipad, this noise does not exist with iOS 14.2
Comment 27 mawojtcz 2020-11-17 02:01:13 PST
We're seeing this issue also with our Webex webrtc app - it happens both on iOS and macOS. Usually the crackling is heard at the beginning of the call and then goes away, but it comes back later at random times and persists for another 5-60s and goes away again.
Comment 28 mawojtcz 2020-11-18 03:35:44 PST
The new 14.3 Beta 2 seems much better - I can't hear any crackling in webex webrtc app when testing on an Ipad. We'll try to test more devices later today.
Comment 29 German Goldenstein 2020-11-18 04:24:33 PST
Is there any workaround?
Comment 30 Teodor 2020-11-18 09:19:58 PST
I can confirm iOS 14.2 Beta 2 no longer has stuttering after the call starts. Haven't done any long running tests yet.

Now we just sit tight and wait for 14.3 to be released a couple of weeks from now.
Comment 31 Kevbo 2020-11-18 09:23:39 PST
Same for us.
Comment 32 Balazs Zubak 2020-11-19 08:49:58 PST

We experience the same issue. Fond no workaround yet, but seems on Janus test site it works well with the same iOS 14.2:

Also seems working on Apprtc test site:

We are looking for the reason, but meanwhile if anyone find something please share it here.
Comment 33 Balazs Zubak 2020-11-19 10:50:37 PST
Looks like there is some connection between send-receive and receive-only modes, but still need to verify it.
Comment 34 Alexey Proskuryakov 2020-11-21 16:46:15 PST
Marking resolved/fixed again, because 14.3 Beta 2 definitely contains the code change in question, and there were confirmations of it having helped. 

Please file new bugs if any similar issues remain, and of course please feel free to reference them from here.
Comment 35 poplark 2020-11-24 18:11:49 PST
(In reply to Teodor from comment #7)
> We are also experiencing this. Both Safari and SFWebView.
> We use WebAudio – new AudioContext() that is then attached to existing
> <audio> element. Removing context creation seemed to help initially (two
> consecutive tries, in both the crackling happened for less than a second).
> Third try it persisted for about 7 seconds and then cleared, so I am
> guessing this is not the right workaround.

When I removed WebAudio - new AudioContenxt(), the crackling disappeared, so I am guessing the bug with iOS 14.2 affects the WebAudio API - AudioContext, you can have a try.
Comment 36 Mikael Nousiainen 2020-11-29 05:27:12 PST
I've experienced the same issue -- very choppy/intermittent playback -- in Safari (on both iPhone and iPad) and more recently also on desktop Safari on macOS.

I have not been able to get smooth WebRTC live Opus audio playback on Apple Safari browsers since June (maybe somewhere around iOS 13.5 release?) and it seems the affects the macOS desktop Safari as well. I've had no issues running the same code in Firefox or Chrome at any point, so the audio problem seems to be very specific to Safari.

While the problem appears in a more pronounced way in an application I'm developing and the JS fiddle presented in this bug, I'm able to reproduce similar short pauses in audio when using the official Janus WebRTC gateway Streaming test page using the "War of the Worlds" live Opus audio stream: https://janus.conf.meetecho.com/streamingtest.html -- this could be yet another test case.

I've tested this using the following devices, which both produce exactly the same issue on audio being very choppy:
* Safari on iOS 14.2 (iPhone 7)
* Safari 14.0.1 (16610. on macOS Big Sur 11.0.1
Comment 37 Mikael Nousiainen 2020-11-29 05:33:37 PST
Is anyone else able to confirm whether the 14.3 beta release actually fixes the issue? This is a real blocker for me, preventing use of WebRTC audio playback completely on Apple mobile devices :(
Comment 38 Mikael Nousiainen 2020-11-29 11:59:43 PST
I was able to install iOS 14.3 beta 2 on an iPhone 7 and I can confirm that it fixes the issue demonstrated by the JS fiddle (oscillator -> WebRTC -> audio tag).

However, a real use case with a live Opus audio WebRTC stream coming in from Janus WebRTC Gateway server, fed through AudioContext processing (mainly audio gain) and then played by an audio tag still suffers from exactly the same kind of stuttering/choppiness as the example of the oscillator audio.

This may be a similar bug, but not exactly the same case. I'll try to write some example code to see whether I can reproduce this other issue.
Comment 39 Mike Puglisi 2020-12-02 16:33:59 PST
This is a big deal for us. I've reproduced the issue both with Wowza and Twilio WebRTC services. 

Twilio's Quickstart exhibits the problem


Here's the actual github issue

Comment 40 Anna Vasilko 2020-12-03 15:37:43 PST
We have confirmed the issue is fixed with iOS 14.3 Beta 3 (tested with Twilio Video JavaScript SDK).

When is this beta expected to be released to the public?
Comment 41 rychou 2021-01-04 02:40:58 PST
Is there any way to circumvent the bug of iOS 14.2? At present, we have a lot of users who use iOS 14.2, and it is very expensive to guide them to upgrade to iOS 14.3.
Comment 42 Tim Panton 2021-01-07 01:25:42 PST
We haven't had this problem in rendezvous.family or rendezvous.zone - But we have ptime set to 60ms - perhaps that is a work around for others too.
Here is a munge :

    function tweakSDP(sdp){
        var lines = sdp.split("\r\n");
        for (var i=0;i<lines.length;i++){
            var line = lines[i];
            if (line.startsWith("a=fmtp:111")){
                lines[i]= "a=fmtp:111 minptime=20; stereo=1; ptime=60; useinbandfec=1;";
        return (lines.join("\r\n"));