Layout Test http/tests/websocket/tests/hybi/websocket-cookie-overwrite-behavior.html is a flaky failure

• Component: New Bugs
• Status: NEW ---

[Ryan Haddad]

The following layout test is flaky on High Sierra Debug WK1/WK2

http/tests/websocket/tests/hybi/websocket-cookie-overwrite-behavior.html

Probable cause:

Unknown.

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fwebsocket-cookie-overwrite-behavior.html

--- /Volumes/Data/slave/highsierra-debug-tests-wk1/build/layout-test-results/http/tests/websocket/tests/hybi/websocket-cookie-overwrite-behavior-expected.txt
+++ /Volumes/Data/slave/highsierra-debug-tests-wk1/build/layout-test-results/http/tests/websocket/tests/hybi/websocket-cookie-overwrite-behavior-actual.txt
@@ -1,14 +1,17 @@
+CONSOLE MESSAGE: WebSocket connection to 'ws://127.0.0.1:8880/websocket/tests/hybi/cookie?set' failed: Invalid HTTP version string: HTTP/1.0
+CONSOLE MESSAGE: WebSocket connection to 'ws://localhost:8880/websocket/tests/hybi/cookie?set' failed: Invalid HTTP version string: HTTP/1.0
 Tests WebSocket Set-Cookie overwriting behavior with respect to a document cookie.
 
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
 Same origin WebSocket:
-PASS cookieValue is "foo=bar"
+FAIL cookieValue should be foo=bar. Was foo=should_be_overwritten_by_websocket_set_cookie.
 
 Cross origin WebSocket:
 PASS cookieValue is ""
 PASS successfullyParsed is true
+Some tests failed.
 
 TEST COMPLETE

[Ryan Haddad]

This is a new test added with https://trac.webkit.org/changeset/230052/webkit

Dan, would you please take a look?

[Radar WebKit Bug Importer]

<rdar://problem/39145382>

[Daniel Bates]

<https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fwebsocket-cookie-overwrite-behavior.html> does not seem to be showing me the results when the test fails whenever I click on a red box and click the link of the form "Show results for WebKit rX" from the popup menu :/

[Daniel Bates]

Is this test still flaky. Looking at <https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fwebsocket%2Ftests%2Fhybi%2Fwebsocket-cookie-overwrite-behavior.html> it seems to be consistently passing since r230290 (04/04/2018) on both High Sierra Debug WK1/WK2.

[Alexey Proskuryakov]

It doesn't seem to be flaky on he bots. However, the circumstances are very suspicious, and I doubt that there was an actual progression.

This test was very flaky on slow bots, failing ~50% on the time. On some bots, the test abruptly ceased to be flaky around r230292. On others, it ceased to fail around r230210...230235. One suspicious thing is that the progression point are different; another is that there isn't anything there that looks like it could fix it.

There were many loader refactorings around this time. Does this test even fail in ToT if the code change in r230052 is reverted?