Bug 229459 - [ EWS Catalina ] http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger.html is flaky timing out / failing
Summary: [ EWS Catalina ] http/tests/websocket/tests/hybi/inspector/send-and-recieve-d...
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Mac (Intel) Unspecified
: P2 Normal
Assignee: Patrick Angle
Keywords: InRadar
Depends on:
Reported: 2021-08-24 11:35 PDT by ayumi_kojima
Modified: 2021-08-26 14:59 PDT (History)
6 users (show)

See Also:

Patch v1.0 (3.92 KB, patch)
2021-08-25 22:02 PDT, Patrick Angle
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description ayumi_kojima 2021-08-24 11:35:53 PDT

Is flaky timing out on macOS-Catalina-Debug-WK1-Tests-EWS

The flaky timeout and failure is seen on Big Sur wk1 Debug and Catalina Debug in the open source directory: https://results.webkit.org/?suite=layout-tests&test=http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger.html

Result page: https://ews-build.s3-us-west-2.amazonaws.com/macOS-Catalina-Debug-WK1-Tests-EWS/r436009-13490/results.html

Flakiness on EWS seems to have started at this build https://ews-build.webkit.org/#/builders/56/builds/13490

Diff from timeout and failure on the open source: 

--- /Volumes/Data/worker/catalina-debug-tests-wk1/build/layout-test-results/http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger-expected.txt
+++ /Volumes/Data/worker/catalina-debug-tests-wk1/build/layout-test-results/http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger-actual.txt
@@ -1,3 +1,4 @@
+FAIL: Timed out waiting for notifyDone to be called
 Tests sending and receiving WebSocket messages.
@@ -9,10 +10,10 @@
 PASS: Frame should be outgoing.
 PASS: Message is walltime.
 Pausing script execution with `debugger` statement.
-Resuming script execution.
 PASS: Resource size should double.
 PASS: Frame data should be 'Hello World! Привет Мир!'
 PASS: Frame should be text.
 PASS: Frame should be incoming.
 PASS: Frame walltime should be greater than the previous one.
+Resuming script execution.
Comment 1 Radar WebKit Bug Importer 2021-08-24 11:36:16 PDT
Comment 2 ayumi_kojima 2021-08-24 11:38:27 PDT
It seems like the test has been flaky since it was introduced here https://trac.webkit.org/changeset/281323/webkit
Comment 3 ayumi_kojima 2021-08-24 11:48:40 PDT
Updated test expectations https://trac.webkit.org/changeset/281507/webkit
Comment 4 Patrick Angle 2021-08-24 11:49:14 PDT
I think the timing in this test is too tight since we need to receive the message from the test WebSocket server while paused for debugging in the test. I'll tweak the timing.
Comment 5 Patrick Angle 2021-08-25 22:02:54 PDT
Created attachment 436470 [details]
Patch v1.0
Comment 6 Patrick Angle 2021-08-25 22:04:01 PDT
Comment on attachment 436470 [details]
Patch v1.0

Manually verified locally that 1000 iterations pass with `run-webkit-tests --iterations=1000 --exit-after-n-crashes-or-timeouts=1 --exit-after-n-failures=1 LayoutTests/http/tests/websocket/tests/hybi/inspector/send-and-recieve-debugger.html`
Comment 7 Devin Rousso 2021-08-26 14:37:53 PDT
Comment on attachment 436470 [details]
Patch v1.0

Comment 8 EWS 2021-08-26 14:59:28 PDT
Committed r281658 (241011@main): <https://commits.webkit.org/241011@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 436470 [details].