Bug 247943 – REGRESSION (Safari 15.6.1-16.6): WebSocket in latest Safari does not emit onclose event when internet is turned off (wifi or cellular)

• Component: WebCore Misc.
• Status: RESOLVED CONFIGURATION CHANGED

[Mike]

Created attachment 463530 [details]
Screens from Safari

WebSocket in latest Safari v16.1 (on macOS Big Sur, Monterey, Ventura) and in latest Safari in iOS 15/16 does not emit onclose event when internet is turned off (wifi or cellular). Problem is on any device (MacBook, iPhone, iPad) with latest Safari. Bug details and demo available here: https://dev.bikeapp.sk/apple-bugs/2/ In e.g. Firefox on macOS or older Safari versions you will see "WebSocket fired event onopen: yes" and when you turn internet off you will see EXPECTED "WebSocket fired event onclose: yes". But in latest Safari v16.1 you still see UNEXPECTED "WebSocket fired event onclose: no" after turning internet off. This URL also contains proofs - screens from older Safari versions that is working there. Bug is on any device (MacBooks, iPhones, iPads) with latest Safari. Advanced debug tool with timestamps available here: https://dev.bikeapp.sk/apple-bugs/debug/ This tool was used for creating screens.

[Alexey Proskuryakov]

Steps to reproduce:
1. Open https://dev.bikeapp.sk/apple-bugs/debug/
2. Disconnect from the network

Expected results: should see "Internet Status: OFFLINE" after disconnecting.

[Radar WebKit Bug Importer]

<rdar://problem/102398431>

[Mike]

Im just improving comment https://bugs.webkit.org/show_bug.cgi?id=247943#c1

Expected result at https://dev.bikeapp.sk/apple-bugs/debug/ is:

- "Internet Status: OFFLINE" (which works fine)

- and "WebSocket {N} Event: CLOSE" (which does not work in latest Safari)

(order of OFFLINE and CLOSE after disconnecting does not matter)

[youenn fablet]

Trying on Ventura, what I see is the following:
- Stop Wifi, status goes to offline
Then:
- If I restart Wifi quickly enough, status goes to online and WebSocket connection is still live and working
- If I wait to restart Wifi for quite some time, I get the web socket connection failing at some point, most of the time at the time Wifi restarts. Maybe this is what you do not expect?

Trying in Monterey in Safari 15, I see roughly the same behavior.
I am not sure to see a regression here.

@Mike, Can you clarify?
Also, can you reproduce in Safari Tech Preview as well?

[Mike]

For better testing if connection is still alive even after internet disconnect and connect I have connected debug tool website https://dev.bikeapp.sk/apple-bugs/debug/ to new web socket server for debug purposes. Now server sends ping messages each 5 seconds to client and client responds with pong. When client sends manually ping using console ws.send("ping"), server responds with pong. We can see it in network console in message frames.

I have tested Monterey 12.6.1 (21G217) with Safari v15.6.1 and yes it has the same behavior as v16.1. So regression is already between Catalina's Safari v15.6.1 and Monterey's Safari v15.6.1. Please see diff between this screen https://dev.bikeapp.sk/apple-bugs/2/Screen%20Shot%202022-11-15%20at%2013.26.52.png and this one https://dev.bikeapp.sk/apple-bugs/2/Screen%20Shot%202022-11-16%20at%2014.28.28%20missing%20onclose%20event.png I did not notice that Safari v15.6.1 could be different on Catalina and Monterey and here https://dev.bikeapp.sk/apple-bugs/2/ I have originally tested just Catalina where WebSocket onclose if correctly fired when internet is disconnected. I just added screen from Monterey's Safari v15.6.1.

Why new behavior is problematic...

1.) Yes, when disconnected and connected wifi again websocket connection is still working. I have did not notice this because origin websocket server implements ping/pong to detect zombie connections. Clients not responding to server's ping are terminated (considered stale/broken/zombie).

2.) Yes when disconnected for short time WS connection is still alive after connecting back. But when disconnected and connected after in about 2 minutes still not receiving CLOSE event and websocket connection on client sides seems to be active but in the fact it is not. See screen https://dev.bikeapp.sk/apple-bugs/2/Screen%20Shot%202022-11-16%20at%2013.16.33%20with%20note.png when sending "ping" from client, server does not respond with pong so connection is broken and client does no fire CLOSE event (for long time, maybe forever - must test it). You can see ping attempts in screen but no "pong" answer from server. So this regression breaks reconnect mechanism. What time should client consider connection working after turning internet off/on? It is always different when testing - not reliable. Client does not know if connection is alive or not so it is not reliable now. In the past when CLOSE was fired with each interned disconnection it was better and reconnection could be implemented easily thanks to onclose event. Now connection looks like working but it is not. You can send how many messages want but server is not receiving it and client is not errored.

[Mike]

Notice that on iPhone when locked screen or minimized Safari with debug tool website https://dev.bikeapp.sk/apple-bugs/debug/ , WebSocket onclose is fired immediately. And user much more locks phone and much more minimizes app then turning internet off and on. So when onclose is fired with lock/minimize why could not be also fired on internet disconnection as in the past versions? What sense has this new behavior please? It keeps old WebSocket connection in readyState OPEN after internet disconnected but connection is not reliable when internet is connected again (connection could be not working after what amount of time after disconnect/connect?) so developer must anyway on internet disconnected close origin websocket connection manually and on internet connected open new one. This new behavior just breaks reconnect mechanism based on close event verified over the years in many webapps.

[youenn fablet]

We changed of backend and are now using NSURLSession WebSocket, which probably explains the difference of behaviour.

Trying on Safari, Chrome and Firefox the provided example, I can see Firefox proactively firing the close event.
Chrome and Safari do not seem to close it proactively though in my setup.

[youenn fablet]

> 2.) Yes when disconnected for short time WS connection is still alive after
> connecting back. But when disconnected and connected after in about 2
> minutes still not receiving CLOSE event and websocket connection on client
> sides seems to be active but in the fact it is not. See screen
> https://dev.bikeapp.sk/apple-bugs/2/Screen%20Shot%202022-11-16%20at%2013.16.
> 33%20with%20note.png when sending "ping" from client, server does not
> respond with pong so connection is broken and client does no fire CLOSE
> event (for long time, maybe forever - must test it). You can see ping
> attempts in screen but no "pong" answer from server. So this regression
> breaks reconnect mechanism.

I would guess the TCP connection is still live, maybe the issue is on the WebSocket server itself that think the connection is dead even though it is still good.
Is it a custom WebSocket server or a widely used one?

[Mike]

Server is one of most popular https://github.com/websockets/ws

[youenn fablet]

(In reply to Mike from comment #9)
> Server is one of most popular https://github.com/websockets/ws

@Mike, are you seeing the same issue in Chrome? If so, how do you handle it there?

[Mike]

We primary take care about mobile devices. Chrome and others on iOS is not widely used. Anyway Firefox and Chrome on iOS have the same issue as Safari nowadays because they use WebKit too. Chrome on Android fires WebSocket CLOSE event immediately when internet is disconnected. It is reliable because we have tested also ancient devices with ancient Chrome (e.g. since Android v4.2, Chrome v37) and also modern devices from different manufactures with latest Chrome and behavior is the same with no problem. Also Firefox on Android fires CLOSE event immediately. Problem is in iOS WebKit browsers where behavior has been changed as you wrote since NSURLSession WebSocket is used.

How to handle it? I think the only one solution is to manually close connection when offline event is fired window.addEventListener('offline'), () => { ws.close() }) and on online event create a new one window.addEventListener('offline'), () => { var ws = new WebSocket(...) }) I think each developer must do it now because as mentioned in screen https://dev.bikeapp.sk/apple-bugs/2/Screen%20Shot%202022-11-16%20at%2013.16.33%20with%20note.png websocket connection after internet off/on is not reliable. So question is what sense have to do not fire close event automatically? Is it possible to change NSURLSession WebSocket behavior (fire close event on internet disconnected)? As I already mentioned close event is fired when phone is locked or app minimized, so it is expected to be fired when internet is disconnected.

New test: https://dev.bikeapp.sk/apple-bugs/2/Screen%20Shot%202022-11-18%20at%2010.03.13%20with%20note.png
On macOS Firefox closes websocket connection 10 seconds after internet disconnection, Chrome after 10 minutes and Safari seems never. All these browsers on iOS never fires close event because they use the same WebKit. So I think changing NSURLSession WebSocket behavior have sense.

It is not desired to consider websocket connection on client side as working after so long time after internet is disconnected. Author of "ws" lib recommends here https://github.com/websockets/ws#how-to-detect-and-close-broken-connections to consider clients not responding to ping for more then 30 secs (+ latency) to be zombie and must be terminated. This MUST BE always implemented by each developer because zombie connections can spend server resources. We also implemented it (except debug tool website to demonstrate that Safari websocket connection is not alive after some time although it is still in readyState OPEN).

So it would be great to change NSURLSession WebSocket:

A) fire close event immediately after internet is disconnected (identical behavior as in older Safari versions),

B) or inspire by Chrome and Firefox (those on macOs which are not using WebKit) and close connection after some fixed time.

[Brent Fulgham]

The cause of this bug is outside of WebKit. Resolving this bug as MOVED, since the change needs to happen in an external framework. The relevant team is working on a fix now (tracked by <rdar://problem/102398431>).

[Brent Fulgham]

Actually, the bug was fixed in rdar://103927601 and shipped in macOS 13.3.1 and iOS 16.4.1.

[Mike]

@Brent Fulgham by my testing it is not fixed. Tested now in iOS 16.4.1 (20E252). Here https://dev.bikeapp.sk/apple-bugs/2/ sentence "WebSocket fired event onclose" still shows "no" after turning wifi off. See screens here: https://dev.bikeapp.sk/apple-bugs/2/still-not-fixed-in-v16.4.1.png I have been waiting for at least 15 minutes and "onclose" still not changed to "yes" as in another non-webkit browsers (Firefox, Chrome etc.). Could you test please this https://dev.bikeapp.sk/apple-bugs/2/ ?

As I mentioned earlier it is not desired websocket connection to pretend in opened state when there is no internet connection. After restoring internet connection websocket connection is anyway usually dead because of killing zombie connections by server (server can not keep zombie connections in memory to infinity). Zombie connections I mean connections which do not responds with "pong" to server's "ping" for long time. For more more about zombie connections see here https://github.com/websockets/ws#how-to-detect-and-close-broken-connections 

It would be great Safari's websockets to use behaviour as before switching to NSURLSession WebSocket and as another browsers behaves.

[Mike]

Created attachment 465882 [details]
Still not fixed in iOS v16.4.1

[Alex Christensen]

I investigated and filed rdar://108017117 with some additional info.

[Rich Evans]

This has been causing real issues with my site which uses slate-yjs for realtime collaboration and persistence.

When the network switches/drops all subsequent changes are lost without so much as warning being raised to the user that they are offline, and the connection would usually never recover causing total user data loss in some instances.

I was forced to reimplement a heartbeat protocol so the client can detect it is offline and recreate the websocket manually.

This was impacting iOS, iPadOS and Safari in MacOS.

[Chad]

For those trying to debug this issue and tracking the importance of this bug, it's also worth pointing out that this issue now occurs in Chrome and Edge as well. 

Basically, all websites and web apps that use websockets, such as your document editing and stock trading apps, will now lose data when your connection is disrupted. That is because the client doesn't know that the connection was interrupted so any mitigation that was previously in place is now broken. 

This impacts nearly every web user, please prioritize this bug with the highest level of importance for everyone's benefit!

[gordon lee]

I got the same problem when use hocuspocus/provider for collaboration edit docs which use websocket(https://github.com/websockets/ws) in iOS15.0 or iOS16.4.1 webview

those step can reproduce this bugs:
1. First open wifi & cellular;
2. Open my share collaboration edit that will use wifi priority connect to remote Websocket server;
3. Turn off the wifi; even the client close websocket by call ws.close() the websocket has no fire close event that caused the connection hang forever util  lock screen;

its most important for those apps which use WKWebView to ingest collaboration apps.

[Chad]

This issue appears to be fixed in Safari (it is still an issue in Chrome).

[Mike]

(In reply to Chad from comment #20)
> This issue appears to be fixed in Safari (it is still an issue in Chrome).

Yes, I can confirm that Safari 17.3 on latest iOS now fires CLOSE event before OFFLINE event as expected. Great! So it must be fixed somewhere in v17 but without mention in some of v17.x release notes here https://developer.apple.com/documentation/safari-release-notes Btw latest v16 Safari is not fixed - tested now. I have tested macOS Safari v17.1 and there it is not fixed (I do not have probably latest v17 now).

When you mention Chrome, note that CLOSE event was always working and also working now in latest Chrome working on Android. Only desktop Chrome version does not fire CLOSE event immediately but I think several minutes later (such as Firefox) - it is difference because Safari after bug introducing never fired CLOSE event - you could be offline several hours but websocket connection looked connecting/working!

[Ahmad Saleem]

As confirmed by the originator that this bug is now fixed.

Marking this as RESOLVED CONFIGURATION CHANGED.