Bug 227030 - REGRESSION (r275496): WebSocket Message too long when message is larger than 1mb
Summary: REGRESSION (r275496): WebSocket Message too long when message is larger than 1mb
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Page Loading (show other bugs)
Version: Safari Technology Preview
Hardware: iPhone / iPad Other
: P2 Normal
Assignee: youenn fablet
URL:
Keywords: InRadar
Depends on:
Blocks: 224102
  Show dependency treegraph
 
Reported: 2021-06-15 08:46 PDT by Licat
Modified: 2021-06-21 11:15 PDT (History)
7 users (show)

See Also:


Attachments
Steps to reproduce (459.37 KB, image/jpeg)
2021-06-15 08:46 PDT, Licat
no flags Details
Echo server with no fragmentation (924 bytes, application/gzip)
2021-06-15 22:19 PDT, Steve Dunham
no flags Details
Echo server with fragmentation (839 bytes, application/gzip)
2021-06-15 22:22 PDT, Steve Dunham
no flags Details
Patch (12.64 KB, patch)
2021-06-17 16:46 PDT, Alex Christensen
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (12.90 KB, patch)
2021-06-17 17:03 PDT, Alex Christensen
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (12.18 KB, patch)
2021-06-17 17:20 PDT, Alex Christensen
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (12.23 KB, patch)
2021-06-17 17:26 PDT, Alex Christensen
no flags Details | Formatted Diff | Diff
Patch (12.26 KB, patch)
2021-06-18 09:31 PDT, Alex Christensen
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Licat 2021-06-15 08:46:16 PDT
Created attachment 431441 [details]
Steps to reproduce

Overview: Starting in iOS 15 Beta, whenever a WebSocket receives a message of size 1,048,549 bytes or more, even if from multiple frames, it will terminate the connection with error 1009 (Message too big).
This wasn't the case in any previous builds.

Steps to reproduce:
- Create a WebSocket echo server using any language (A good sample can be found at https://www.npmjs.com/package/ws#simple-server)
- Using the remote debugger console, create a WebSocket and send a short message, observe that the message is sent back.
- Send a message of size 1,048,549 or larger, observe that the message is sent, and the connection is immediately terminated with "The operation couldn't be completed (kNWErrorDomainPOSIX error 40 - Message too long)"
- Observe from the server that the client disconnected with error 1009 (Message too long).

I find that 1mb is quite a low limit, if that's intended to limit memory consumption. For any use cases involving sending significant amount of data, this would need to be worked around by concatenating multiple messages, which will end up consuming a lot more memory anyway.
Comment 1 Licat 2021-06-15 09:04:55 PDT
Correction from last post: The message size should be 1,048,577 or larger, corresponding to 1024*1024+1, as opposed to 1,048,549
Comment 2 Radar WebKit Bug Importer 2021-06-15 17:26:08 PDT
<rdar://problem/79370994>
Comment 3 Alex Christensen 2021-06-15 17:57:40 PDT
I'm able to send and receive messages larger than 1MB.  This might need more investigation.  Licat, are you able to provide a self-contained example website where we can see the issue?
Comment 4 Licat 2021-06-15 20:07:25 PDT
I've hosted a temp site on aws at:

http://18.237.249.65:8888/

The script on the site sends a blob of 1048577 "x" characters, and the server echos it back.

I've actually gotten reports from others that this only happens on some devices. One of my contacts said that the bug only occurred on their iOS 15 device but not iPadOS 15 device, while another ran into this issue on an iPad.
Comment 5 Chris Dumez 2021-06-15 20:24:34 PDT
On my phone with iOS 15, I get “error: undefined”. I guess that means I reproduced?
Comment 6 Chris Dumez 2021-06-15 20:27:43 PDT
(In reply to Chris Dumez from comment #5)
> On my phone with iOS 15, I get “error: undefined”. I guess that means I
> reproduced?

I got the same error on an iPad Pro.
Comment 7 Licat 2021-06-15 20:34:14 PDT
A correct run should print the following:

CONNECTED
SENT: 1048577 bytes
RESPONSE: 1048577 bytes
DISCONNECTED

It would probably help to check Safari's remote inspector and have a look at the WebSocket frames. The behavior should be a similar to the attached screenshot.
Comment 8 Chris Dumez 2021-06-15 20:35:31 PDT
Could be an IPC issue.
Comment 9 Steve Dunham 2021-06-15 22:06:19 PDT
A few more details - I wrote a websocket echo server in Go (gorilla/websocket) to help Licat diagnose this. The client was an iPad Pro with iOS 15.0 (19A5261w). The attached jpeg reflects my testing.

In my initial test, I had the server relay the message as-is. If it was larger than 1048576 bytes, the server received the message, sent the response, and got an EPIPE.  Via the remote dev tools, MobileSafari reported the following and closed the connection:

[Error] WebSocket connection to 'ws://arbol.local:8080/echo' failed: The operation couldn’t be completed. (kNWErrorDomainPOSIX error 40 - Message too long)

I then adjusted the server code to fragment the message into maximum 512kb fragments. After sending the last fragment of a 1048577 byte message, the server received a 1009 error and MobileSafari didn't register any response or error (looking at the network pane in the remote dev tools). At that point, MobileSafari thought the connection was still open (readyState = 1) and let me send messages without error, even after I killed the server.

The same testing on dev console in Safari Technology Preview on Big Sur worked flawlessly. (The messages were received by the server, sent back to the browser, and logged.)
Comment 10 Steve Dunham 2021-06-15 22:19:50 PDT
Created attachment 431516 [details]
Echo server with no fragmentation

Attached is the echo server, without fragmentation, that I used in testing.  It can be started with "go run ." within the directory and will listen on port 8080.

The test was the sequence of manual javascript commands shown in the jpeg attached to this issue.
Comment 11 Steve Dunham 2021-06-15 22:22:17 PDT
Created attachment 431517 [details]
Echo server with fragmentation

This is the echo server, with fragmentation, that I used in testing.  It can be started with "go run ." within the directory and will listen on port 8080.
Comment 12 Chris Dumez 2021-06-16 08:52:39 PDT
Easily reproducible for me with http://18.237.249.65:8888/, even on macOS. I am bisecting to find the regression point.
Comment 13 youenn fablet 2021-06-16 09:14:58 PDT
Can you disable Nsurlsession web socket experimental feature and try reproing?
Comment 14 Chris Dumez 2021-06-16 09:16:36 PDT
(In reply to youenn fablet from comment #13)
> Can you disable Nsurlsession web socket experimental feature and try
> reproing?

Regression range:
https://trac.webkit.org/log/webkit/trunk?mode=follow_copy&rev=275500&stop_rev=275491

most likely a regression from trac.webkit.org/r275496.
Comment 15 Chris Dumez 2021-06-16 09:22:55 PDT
Confirmed regression from https://trac.webkit.org/changeset/275496.
Comment 16 Alex Christensen 2021-06-17 16:37:44 PDT
Thanks for providing that server.  The test server I was using was using deflate, so I was "receiving" over 1MB messages fine but only a few bytes were actually going over the network because it compressed so well.  Your server doesn't use deflate, so it hits this failure mode.

I've investigated this issue and found a fix with a test.
Comment 17 Alex Christensen 2021-06-17 16:46:33 PDT
Created attachment 431740 [details]
Patch
Comment 18 Alex Christensen 2021-06-17 17:03:28 PDT
Created attachment 431744 [details]
Patch
Comment 19 Chris Dumez 2021-06-17 17:11:40 PDT
Comment on attachment 431744 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=431744&action=review

> Tools/ChangeLog:17
> +2021-06-17  Alex Christensen  <achristensen@webkit.org>

double changelog.
Comment 20 Alex Christensen 2021-06-17 17:20:20 PDT
Created attachment 431746 [details]
Patch
Comment 21 Alex Christensen 2021-06-17 17:26:08 PDT
Created attachment 431747 [details]
Patch
Comment 22 Alex Christensen 2021-06-18 09:31:36 PDT
Created attachment 431778 [details]
Patch
Comment 23 youenn fablet 2021-06-19 01:05:44 PDT
Comment on attachment 431778 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=431778&action=review

> Source/WebKit/ChangeLog:14
> +        The test used to fail using our NSURLSession-based WebSocket implementation but now passes everywhere.

LayoutTests/imported/w3c/web-platform-tests/websockets/handlers/send-backpressure_wsh.py is triggering the bug for me.
Comment 24 EWS 2021-06-21 11:15:13 PDT
Committed r279069 (238989@main): <https://commits.webkit.org/238989@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 431778 [details].