REGRESSION (iOS 15): Safari loses Network access when establishing WebRTC session

• Component: WebRTC
• Status: RESOLVED FIXED

[David Gölzhäuser]

Some users of our Intercom noticed a strange behaviour regarding WebRTC calls on iPhones running iOS 15.
When trying to establish a connection with the intercom via WebRTC no network communication is possible anymore, Websockets and HTTPs requests will just run in a timeout.

After some investigation with Wireshark we noticed that it looked like the iPhone is not answering the DTLS Handshake correctly, as Wireshark shows "DTLS" instead of "DTLSv1.2" or "DTLSv1.0".
The iOS OS Log shows that boringssl successfully finishes, but shortly after I see the following messages:

08:48:15.515922+0100	com.apple.WebKit.Networking	nw_read_request_report [C35] Receive failed with error "Connection refused"
08:48:15.516001+0100	com.apple.WebKit.Networking	NetworkRTCUDPSocketCocoaConnections failed processing UDP data with error 61

There is something going on with UDP which would match the not finished DTLS handshake.

This issue described only affects srflx candidates, host or relay candidates would work, but we can't relay every iPhone user through our Relay server yet. 
As the feature is implemented for an Intercom (Door Camera) remote access via srflx is essential

I also found this thread which I think describes a similar issue: https://developer.apple.com/forums/thread/689293

Further Logs or dumps can be supplied if nessesary.

[David Gölzhäuser]

Created attachment 445689 [details]
Wireshark log

This is a Wireshark of the issue, here you can see the DTLS without any version and that the iPhone is cycling through Binding requests and and then a DTLS from the Intercom. The iPhone also gets a bit warm.

I redacted the external IP addresses:
Red = Our STUN/TURN server
Yellow = The Public IP of my network
Blue = The Public IP of the network with the Intercom in it

10.7.10.145 is the iPhone
192.168.62.20 is the Intercom

[David Gölzhäuser]

As our app is Cordova based we tried to implement cordova-plugin-iosrtc (https://github.com/cordova-rtc/cordova-plugin-iosrtc), it implements many aspects of WebRTC on its own and also used libwebrtc from Chrome M87.

This strongly indicates that there is a bug in iOS Safaris WebRTC handling.

[David Gölzhäuser]

When disabling the Experimental Feature "WebRTC Platform UDP Sockets" in the iOS Safari Settings and restarting Safari the issue is resolved.

I assume this is due to this commit https://git.webkit.org/?p=WebKit.git;a=commit;h=94f31e8bfa57b5f69a8be638ba7560008a9b7757

This enables the Socket abstraction to enable iCloud Private Relay for WebRTC, although WebRTC still leaks public IP addresses, I assume the proxying is still work in progress.

[David Gölzhäuser]

However this issue persists in any Cordova based applications as the WKWebView ignores the setting from Safari. I couldn't find a way how to disable this flag programmatically.

[David Gölzhäuser]

This issue is fixed with iOS 15.2 Developer Beta 4.

Can someone explain what the issue was? I am unable to find a matching commit in WebKit, so I assume it was a general issue in the Socket handling in iOS?

[youenn fablet]

(In reply to David Gölzhäuser from comment #5)
> This issue is fixed with iOS 15.2 Developer Beta 4.
> 
> Can someone explain what the issue was? I am unable to find a matching
> commit in WebKit, so I assume it was a general issue in the Socket handling
> in iOS?

Difficult to say, I am glad this is fixed.
@David, if you can still reproduce, you can privately send me a sysdiagnose ([e-mail]). Or if you have a way for me to reproduce this issue.

> Websockets and HTTPs requests will just run in a timeout.

So all networking to the intercom was broken? Was private relay on?

[David Gölzhäuser]

I am unable to reproduce this in our development environment, only three users (two of are employees of our company) report these issues. I have access to two systems which I used to analyze this issue.

I may be able to ask for access.

> So all networking to the intercom was broken? Was private relay on?

This issue happens in the Safari App and in our Cordova based app.

When using the Safari App all network communication is timed out, even if I switch to a different tab and try to load https://www.apple.com.

In our Cordova based app all network traffic is timing out, so I guess its for the whole application process?

iCloud Private Relay was not turned on. There is no difference when its on or off.


Further:
I checked the WebKit code and it looks like the libWebRTC sockets abstraction to Cocoa Network API is always active as long as the Experimental feature "WebRTC Platform UDP Sockets" is turned on. However proxying WebRTC through iCloud Private Relay is not enabled (yet), as the public IP address leaks when using a STUN server.

[youenn fablet]

This might be

[Radar WebKit Bug Importer]

<rdar://problem/86196093>

[David Gölzhäuser]

The issue is back in iOS 15.2 RC in WKWebView, Safari has no issue tho.

[youenn fablet]

Created attachment 446557 [details]
Patch

[Geoffrey Garen]

Comment on attachment 446557 [details]
Patch

r=me

[EWS]

Committed r286841 (245074@main): <https://commits.webkit.org/245074@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 446557 [details].

[youenn fablet]

@David, I hope you will be able to test in MacOS Safari Tech Preview when the fix gets there. Since I do not have your exact setup, it would be good to validate that this now works.
If not, another sysdiagnose might help since the behavior will have changed.

[youenn fablet]

Comment on attachment 446557 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=446557&action=review

> Source/WebKit/NetworkProcess/webrtc/NetworkRTCUDPSocketCocoa.mm:292
> +        RELEASE_LOG_ERROR_IF(!!error, WebRTC, "NetworkRTCUDPSocketCocoaConnections failed processing UDP data with error %d", nw_error_get_error_code(error));

But now we have excessive logging.
I guess we need to log only if there is an error and the error code is different from the previous one.

[David Gölzhäuser]

I am currently updating my Mac to the latest Technology Preview and macOS Beta. I can confirm that the issue is still available in iOS 15.2 RC 2 Released today, but I assume the fix has not landed in that build yet. I will keep you updated

[David Gölzhäuser]

@Youenn, I sent you sysdiagnistic from macOS using the latest Safari Tech Preview via email.

[Truitt Savell]

It looks like the changes here caused many webrtc tests to flakily timeout or fail on iOS. tracking in https://bugs.webkit.org/show_bug.cgi?id=234181

[David Gölzhäuser]

I'll change the status to "Reopened" again

[David Gölzhäuser]

The issue appears to be fixed again in iOS 15.3 Beta 1.

[David Gölzhäuser]

This issue appears to be back in iOS 15.3 Beta 2.

[David Gölzhäuser]

This issue appears to be back in iOS 15.3 Beta 2.

[David Gölzhäuser]

This issue is still present in iOS 15.3 RC.

[youenn fablet]

Can you try iOS 15.4 beta1?

[David Gölzhäuser]

This issue is fixed again in iOS 15.4 Beta 1

[David Gölzhäuser]

This issue is still fixed in iOS 15.4 Beta 2

Did you do anything to fix this?

[youenn fablet]

(In reply to David Gölzhäuser from comment #26)
> This issue is still fixed in iOS 15.4 Beta 2
> 
> Did you do anything to fix this?

The above fix made it to iOS 15.4 betas (but not iOS 15.3).
Marking at resolved, thanks for your testing!