Bug 229459 – [ EWS Catalina ] http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger.html is flaky timing out / failing

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger.html

Is flaky timing out on macOS-Catalina-Debug-WK1-Tests-EWS

The flaky timeout and failure is seen on Big Sur wk1 Debug and Catalina Debug in the open source directory: https://results.webkit.org/?suite=layout-tests&test=http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger.html

Result page: https://ews-build.s3-us-west-2.amazonaws.com/macOS-Catalina-Debug-WK1-Tests-EWS/r436009-13490/results.html

Flakiness on EWS seems to have started at this build https://ews-build.webkit.org/#/builders/56/builds/13490

Diff from timeout and failure on the open source: 

--- /Volumes/Data/worker/catalina-debug-tests-wk1/build/layout-test-results/http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger-expected.txt
+++ /Volumes/Data/worker/catalina-debug-tests-wk1/build/layout-test-results/http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger-actual.txt
@@ -1,3 +1,4 @@
+FAIL: Timed out waiting for notifyDone to be called
 Tests sending and receiving WebSocket messages.
 
 
@@ -9,10 +10,10 @@
 PASS: Frame should be outgoing.
 PASS: Message is walltime.
 Pausing script execution with `debugger` statement.
-Resuming script execution.
 PASS: Resource size should double.
 PASS: Frame data should be 'Hello World! Привет Мир!'
 PASS: Frame should be text.
 PASS: Frame should be incoming.
 PASS: Frame walltime should be greater than the previous one.
+Resuming script execution.

[Radar WebKit Bug Importer]

<rdar://problem/82301943>

It seems like the test has been flaky since it was introduced here https://trac.webkit.org/changeset/281323/webkit

Updated test expectations https://trac.webkit.org/changeset/281507/webkit

[Patrick Angle]

I think the timing in this test is too tight since we need to receive the message from the test WebSocket server while paused for debugging in the test. I'll tweak the timing.

[Patrick Angle]

Created attachment 436470 [details]
Patch v1.0

[Patrick Angle]

Comment on attachment 436470 [details]
Patch v1.0

Manually verified locally that 1000 iterations pass with `run-webkit-tests --iterations=1000 --exit-after-n-crashes-or-timeouts=1 --exit-after-n-failures=1 LayoutTests/http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger.html`

[Devin Rousso]

Comment on attachment 436470 [details]
Patch v1.0

rs=me

[EWS]

Committed r281658 (241011@main): <https://commits.webkit.org/241011@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 436470 [details].