Bug 176240 - Layout test http/tests/websocket/tests/hybi/handshake-ok-with-legacy-websocket-response-headers.html is flaky
Summary: Layout test http/tests/websocket/tests/hybi/handshake-ok-with-legacy-websocke...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: katherine_cheney
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2017-09-01 12:33 PDT by Matt Lewis
Modified: 2019-11-01 15:48 PDT (History)
10 users (show)

See Also:


Attachments
Patch (2.53 KB, patch)
2019-10-25 15:57 PDT, katherine_cheney
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from webkit-cq-01 for mac-highsierra (3.57 MB, application/zip)
2019-11-01 11:12 PDT, WebKit Commit Bot
no flags Details
Patch (2.46 KB, patch)
2019-11-01 11:21 PDT, katherine_cheney
commit-queue: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from webkit-cq-03 for mac-highsierra (3.53 MB, application/zip)
2019-11-01 15:26 PDT, WebKit Commit Bot
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Lewis 2017-09-01 12:33:00 PDT
According to the dashboard http/tests/websocket/tests/hybi/handshake-ok-with-legacy-websocket-response-headers.html is a flaky failure on iOS simulator WK2.

History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fhandshake-ok-with-legacy-websocket-response-headers.html

build:
https://build.webkit.org/results/Apple%20iOS%2010%20Simulator%20Release%20WK2%20(Tests)/r221482%20(4031)/results.html
https://build.webkit.org/builders/Apple%20iOS%2010%20Simulator%20Release%20WK2%20(Tests)/builds/4031

diff:

--- /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/http/tests/websocket/tests/hybi/handshake-ok-with-legacy-websocket-response-headers-expected.txt
+++ /Volumes/Data/slave/ios-simulator-10-release-tests-wk2/build/layout-test-results/http/tests/websocket/tests/hybi/handshake-ok-with-legacy-websocket-response-headers-actual.txt
@@ -1,3 +1,4 @@
+CONSOLE MESSAGE: WebSocket network error: The operation couldn’t be completed. Connection reset by peer
 Test whether WebSocket handshake is OK with legacy response headers 'WebSocket-Location' and 'WebSocket-Origin'.
 
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
Comment 1 Matt Lewis 2017-09-19 10:57:10 PDT
Marked as flaky on iOS WK2

https://trac.webkit.org/changeset/222213/webkit
Comment 2 Ryan Haddad 2018-10-08 17:41:57 PDT
This has also been seen on High Sierra Release WK2 with the same diff.

https://build.webkit.org/results/Apple%20High%20Sierra%20Release%20WK2%20(Tests)/r236943%20(7166)/results.html
Comment 3 Shawn Roberts 2019-01-25 09:39:42 PST
The following layout test is now flaky on Mojave Release. 

Previous flaky failures noted in iOS 10 Simulator WK2 debut and release, still happening as of today.

Command used to reproduce:

rwt --root testbuild-240473 http/tests/websocket/tests/hybi/handshake-ok-with-legacy-websocket-response-headers.html --iterations 500 -f
Comment 4 Shawn Roberts 2019-06-05 10:02:46 PDT
Updated expectations for Mojave in https://trac.webkit.org/changeset/246111/webkit
Comment 5 Radar WebKit Bug Importer 2019-06-05 10:02:47 PDT
<rdar://problem/51445000>
Comment 6 katherine_cheney 2019-10-25 15:57:38 PDT
Created attachment 381973 [details]
Patch
Comment 7 katherine_cheney 2019-10-31 16:26:23 PDT
Is anyone willing to r+? Can't reproduce this locally, it's not skipped in Test Expectations and it doesn't show up on the flakiness dashboard.
Comment 8 WebKit Commit Bot 2019-11-01 11:12:18 PDT
Comment on attachment 381973 [details]
Patch

Rejecting attachment 381973 [details] from commit-queue.

New failing tests:
requestidlecallback/requestidlecallback-document-gc.html
Full output: https://webkit-queues.webkit.org/results/13201513
Comment 9 WebKit Commit Bot 2019-11-01 11:12:20 PDT
Created attachment 382597 [details]
Archive of layout-test-results from webkit-cq-01 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the commit-queue.
Bot: webkit-cq-01  Port: mac-highsierra  Platform: Mac OS X 10.13.6
Comment 10 katherine_cheney 2019-11-01 11:21:30 PDT
Created attachment 382598 [details]
Patch
Comment 11 EWS 2019-11-01 11:32:55 PDT
Comment on attachment 382598 [details]
Patch

Rejecting attachment 382598 [details] from commit-queue.

Katherine_cheney@apple.com does not have committer permissions according to https://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/common/config/contributors.json.

- If you do not have committer rights please read http://webkit.org/coding/contributing.html for instructions on how to use bugzilla flags.

- If you have committer rights please correct the error in Tools/Scripts/webkitpy/common/config/contributors.json by adding yourself to the file (no review needed).  The commit-queue restarts itself every 2 hours.  After restart the commit-queue will correctly respect your committer rights.
Comment 12 WebKit Commit Bot 2019-11-01 15:26:00 PDT
Comment on attachment 382598 [details]
Patch

Rejecting attachment 382598 [details] from commit-queue.

New failing tests:
requestidlecallback/requestidlecallback-document-gc.html
Full output: https://webkit-queues.webkit.org/results/13202288
Comment 13 WebKit Commit Bot 2019-11-01 15:26:02 PDT
Created attachment 382640 [details]
Archive of layout-test-results from webkit-cq-03 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the commit-queue.
Bot: webkit-cq-03  Port: mac-highsierra  Platform: Mac OS X 10.13.6
Comment 14 David Kilzer (:ddkilzer) 2019-11-01 15:48:25 PDT
Committed r251949: <https://trac.webkit.org/changeset/251949>