Run https://jsfiddle.net/juberti/bnmfvjo0/ (which creates an oscillator, connects it to WebRTC, and plays out in an <audio> tag) and hit the play button. On iOS 14.1 audio plays out normally. On iOS 14.2 the audio stutters immediately. Playout is normal on Safari Mac 14.0, and repros in STP 115 (14.1). Does not repro if the oscillator is connected directly to <audio>.
Thanks for filing Justin. Apple Internal See: rdar://70952670
We are also seeing this as well with our customers who have upgraded to iOS 14.2. This is a significant issue for us and impacts the usability of our product and needs a fix as soon as feasible.
Can anyone comment on where the root issue is so that a workaround can be attempted?
We have tried changing a few options such as audio bit rate and audio processing options with no luck. We are preparing an email to our customers right now, sadly, if we have any left after this.
The issue is in the playout path so those things won't help. Are you using Web Audio in your app?
We are using WebRTC for audio and video communications between two or more parties. We just had a report of this today and confirmed it was 14.2. None of our < 14.2 devices have the issue. All the 14.2 devices do. An device we upgrade to 14.2 then had the issue (didn't before the upgrade). It occurs 9 out of 10 times and the level of stutter often varies.
We are also experiencing this. Both Safari and SFWebView. We use WebAudio – new AudioContext() that is then attached to existing <audio> element. Removing context creation seemed to help initially (two consecutive tries, in both the crackling happened for less than a second). Third try it persisted for about 7 seconds and then cleared, so I am guessing this is not the right workaround.
Sometimes get the stutter for about 5 seconds then it goes away. But most of the time the stutter stays the entire time. Frustrating.
SERIOUS PROBLEM HERE TOO. THIS IS NOT A P2. PLEASE EXPEDITE FIX.
The issue is indeed in the playback being too aggressive in its attempt to read. We changed the behavior to be less aggressive. This should be fixed in WebKit ToT. Will post it when it will become testable in MacOS and iOS through STP or betas.
@youenn Thank you for figuring out the problem. Are there any workarounds that you can think of at this point? Would playing with playbackRate on <audio> element that has the AudioTrack attached be a good starting point? Or maybe implementing own buffer through Web Audio?
Yes, we are desperate for any possible workaround. We can easily change playback rate if that would help, oddly sometimes when it starts if we lower the volume (physically) and raise it a few times it goes away, but we tried doing that in software and it didn't do anything. We also tried unsubscribing to audio then subscribing many times to see if that would help (because 1 out of 10 times it doesn't stutter). We were up much of last night trying different things and we'll spend the rest of the day seeing if anything can mitigate the problem. We'll report back here if we find anything.
Youenn, can you link to the patch that addresses the issue so we can better understand the root cause? Some apps and devices seem to be unaffected, so would like to understand why.
Fix is in https://github.com/WebKit/webkit/commit/d2beea459699aef7b58ed59cd6b435ee9d6a1b72. The reduced fix is as follows: diff --git a/Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm b/Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm index ef7fd04391c5..9e6d5452571d 100644 --- a/Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm +++ b/Source/WebCore/platform/audio/cocoa/AudioSampleDataSource.mm @@ -260,9 +260,8 @@ bool AudioSampleDataSource::pullSamplesInternal(AudioBufferList& buffer, size_t& if (timeStamp >= endFrame) m_endFrameWhenNotEnoughData = endFrame; } else { - // We are too close from endFrame, let's back up a little bit. - uint64_t framesAvailable = endFrame - timeStamp; - m_outputSampleOffset -= sampleCount - framesAvailable; + // We are too close from endFrame, let's wait for more data to be pushed. + m_outputSampleOffset -= sampleCount; dispatch_async(dispatch_get_main_queue(), [logIdentifier = LOGIDENTIFIER, outputSampleOffset = m_outputSampleOffset, this, protectedThis = makeRefPtr(*this)] { ALWAYS_LOG(logIdentifier, "updating offset to ", outputSampleOffset); });
Before the fix has applied, is there another way to play normally? like using audio api etc?
Could you please check 14.3 beta that was made available today?
Still no workaround, but here is a short progress report on what we've tried: - Trying to mix in white noise as dither through WebAudio API. However, they did not mix as expected in MediaStreamAudioDestinationNode. Furthermore, several times the white noise itself would stutter before any media arrived, but I assume this is purely due to CPU starvation when setting up the PeerConnection, and not related to this issue. - Trying to delay the playback via DelayNode through WebAudio API. Regardless of the delay value, and whenever the value is increased, reduced or kept constant, the stuttering occurs whenever the stream is played back. - Trying to forcefully restart offset computation by sending and/or receiving large DataChannel payloads at the beginning of the session. It didn't have an effect, and since audio is higher priority I doubt this can have an effect without SDP munging. Maybe something worth exploring. - Trying to pause/mute/reattach the stream and trying to switch between devices at sender. Stuttering still occurs for "new" streams. - Anything obvious like bitrate, codec and codec parameter manipulations. I guess dropping bitrate to zero would work, but picking it back up would likely trigger the same issue. Maybe worth experimenting. What was not tried by us yet: - Implementing "userland" buffer for audio frames and trying to counteract what the WebKit code is doing (skipping chunks of buffer) by repeating the frames at strategic offsets to force the continuous playback. Big problem with this approach, assuming it is even doable, is knowing when to stop - at some point the stuttering may end by itself (after reaching a stable buffer state) and we no longer want to be outputting duplicate frames any more. - Attempting to capture video and audio separately, which may (or may not) allow us to forcefully desync audio and video tracks, allowing audio to catch up on its own. I am not sure this would work without also doing some SDP munging to deassociate video media lines from audio media lines. Another possibility is trying separate PeerConnections for video and audio streams. Overall, based on the stats that we've captured, the issue only occurs if jitter goes above ~16ms (on our test bench), then once it goes down below ~12ms the stuttering disappears. This, of course, assumes that jitter at rest is at most around 10ms, otherwise it is theoretically possible to reach the edge of the buffer window again and get the stuttering back. Skipping relay candidates, if you can, may help. At least for "nearby" connections, and ignoring other variable, as this largely depends on relay performance and whenever it is itself responsible for any jitter (for example if the relay does protocol switching between UDP and TCP). Stuttering is still there, but it stabilizes faster for us, which is better than never.
@teodor We also tried: delaying playback pause/mute/reattach the stream setting bitrate to zero and increasing it (with several permutation of values in a loop) disabling subscribing to audio/video then re-subscribing with no deterministic benefit.
Heads up, we tested: 14.3 beta 1 (last night) and are still getting the stuttering. @teodor: did you try the above and did it fix it for you?
@alexey Is there any way to change the "Status" to reflect, at least for us, the stuttering is still occurring even with the 14.3 beta 1 version?
Reopening until we get validation we have a fix. @Kevbo, you can privately send repro steps at youenn@apple.com.
Confirming this still repros on iOS 14.3 b1.
Thanks @justin - Glad to know it is not just us :) @youenn - I think Justin already posted a way to repro in the original ticket: Run https://jsfiddle.net/juberti/bnmfvjo0/ (which creates an oscillator, connects it to WebRTC, and plays out in an <audio> tag) and hit the play button.
@Kevbo We also tried 14.3 beta 1. Same behavior across devices for remote WebRTC and local WebRTC from Justin's fiddle – for both it typically stabilized after between 10 and 60 seconds. We haven't made any new workaround attempts.
@Teodor - thanks for the tip! we never tried waiting 60 seconds. I just tried that and it did stop after 60 seconds. We'll add this to our temporary remediation steps for our customers. Very appreciative you mentioned that. Hopefully it doesn't come back in the next 60 second interval ;)
this noise has happen only by iphone and not by ipad. Is it true? from my ipad, this noise does not exist with iOS 14.2
We're seeing this issue also with our Webex webrtc app - it happens both on iOS and macOS. Usually the crackling is heard at the beginning of the call and then goes away, but it comes back later at random times and persists for another 5-60s and goes away again.
The new 14.3 Beta 2 seems much better - I can't hear any crackling in webex webrtc app when testing on an Ipad. We'll try to test more devices later today.
Is there any workaround?
I can confirm iOS 14.2 Beta 2 no longer has stuttering after the call starts. Haven't done any long running tests yet. Now we just sit tight and wait for 14.3 to be released a couple of weeks from now.
Same for us.
Hi, We experience the same issue. Fond no workaround yet, but seems on Janus test site it works well with the same iOS 14.2: janus.conf.meetecho.com/videocalltest.html Also seems working on Apprtc test site: https://appr.tc/ We are looking for the reason, but meanwhile if anyone find something please share it here.
Looks like there is some connection between send-receive and receive-only modes, but still need to verify it.
Marking resolved/fixed again, because 14.3 Beta 2 definitely contains the code change in question, and there were confirmations of it having helped. Please file new bugs if any similar issues remain, and of course please feel free to reference them from here.
(In reply to Teodor from comment #7) > We are also experiencing this. Both Safari and SFWebView. > > We use WebAudio – new AudioContext() that is then attached to existing > <audio> element. Removing context creation seemed to help initially (two > consecutive tries, in both the crackling happened for less than a second). > Third try it persisted for about 7 seconds and then cleared, so I am > guessing this is not the right workaround. When I removed WebAudio - new AudioContenxt(), the crackling disappeared, so I am guessing the bug with iOS 14.2 affects the WebAudio API - AudioContext, you can have a try.
I've experienced the same issue -- very choppy/intermittent playback -- in Safari (on both iPhone and iPad) and more recently also on desktop Safari on macOS. I have not been able to get smooth WebRTC live Opus audio playback on Apple Safari browsers since June (maybe somewhere around iOS 13.5 release?) and it seems the affects the macOS desktop Safari as well. I've had no issues running the same code in Firefox or Chrome at any point, so the audio problem seems to be very specific to Safari. While the problem appears in a more pronounced way in an application I'm developing and the JS fiddle presented in this bug, I'm able to reproduce similar short pauses in audio when using the official Janus WebRTC gateway Streaming test page using the "War of the Worlds" live Opus audio stream: https://janus.conf.meetecho.com/streamingtest.html -- this could be yet another test case. I've tested this using the following devices, which both produce exactly the same issue on audio being very choppy: * Safari on iOS 14.2 (iPhone 7) * Safari 14.0.1 (16610.2.11.51.8) on macOS Big Sur 11.0.1
Is anyone else able to confirm whether the 14.3 beta release actually fixes the issue? This is a real blocker for me, preventing use of WebRTC audio playback completely on Apple mobile devices :(
I was able to install iOS 14.3 beta 2 on an iPhone 7 and I can confirm that it fixes the issue demonstrated by the JS fiddle (oscillator -> WebRTC -> audio tag). However, a real use case with a live Opus audio WebRTC stream coming in from Janus WebRTC Gateway server, fed through AudioContext processing (mainly audio gain) and then played by an audio tag still suffers from exactly the same kind of stuttering/choppiness as the example of the oscillator audio. This may be a similar bug, but not exactly the same case. I'll try to write some example code to see whether I can reproduce this other issue.
This is a big deal for us. I've reproduced the issue both with Wowza and Twilio WebRTC services. Twilio's Quickstart exhibits the problem https://github.com/twilio/video-quickstart-js Here's the actual github issue https://github.com/twilio/twilio-video.js/issues/1296
We have confirmed the issue is fixed with iOS 14.3 Beta 3 (tested with Twilio Video JavaScript SDK). When is this beta expected to be released to the public?
Is there any way to circumvent the bug of iOS 14.2? At present, we have a lot of users who use iOS 14.2, and it is very expensive to guide them to upgrade to iOS 14.3.
We haven't had this problem in rendezvous.family or rendezvous.zone - But we have ptime set to 60ms - perhaps that is a work around for others too. Here is a munge : function tweakSDP(sdp){ var lines = sdp.split("\r\n"); for (var i=0;i<lines.length;i++){ var line = lines[i]; if (line.startsWith("a=fmtp:111")){ lines[i]= "a=fmtp:111 minptime=20; stereo=1; ptime=60; useinbandfec=1;"; } } return (lines.join("\r\n")); }