Bug 206503 – imported/w3c/web-platform-tests/html/dom/idlharness.worker.html is timing out on Catalina wk1 debug EWS

• Status: NEW
• Component: Tools / Tests

[Aakash Jain]

imported/w3c/web-platform-tests/html/dom/idlharness.worker.html is failing on Mojave wk1 debug EWS. This is slowing down mac-wk1-debug EWS since EWS has to re-run the test to rule out flakiness and then run tests on clean tree to check if the failure is pre-existing.

e.g.: 
https://ews-build.webkit.org/#/builders/32/builds/1017
https://ews-build.webkit.org/#/builders/32/builds/1013
https://ews-build.webkit.org/#/builders/32/builds/1011
https://ews-build.webkit.org/#/builders/32/builds/997
https://ews-build.webkit.org/#/builders/32/builds/991
https://ews-build.webkit.org/#/builders/32/builds/980

[Aakash Jain]

https://ews-build.webkit.org/results/macOS-Mojave-Debug-WK1-Tests-EWS/r388229-1017/results.html

--- /Volumes/Data/worker/macOS-Mojave-Debug-WK1-Tests-EWS/build/layout-test-results/imported/w3c/web-platform-tests/html/dom/idlharness.worker-expected.txt
+++ /Volumes/Data/worker/macOS-Mojave-Debug-WK1-Tests-EWS/build/layout-test-results/imported/w3c/web-platform-tests/html/dom/idlharness.worker-actual.txt
@@ -1,5 +1,7 @@
 
-PASS idl_test setup 
+Harness Error (TIMEOUT), message = null
+
+TIMEOUT idl_test setup Test timed out
 PASS idl_test validation 
 PASS Partial interface Document: original interface defined 
 PASS Partial interface Document: member names are unique 
@@ -76,717 +78,5 @@
 PASS Partial interface Window: original interface defined 
 PASS Partial interface Window: member names are unique 
 PASS Partial interface UIEvent: member names are unique 
-PASS Document includes GlobalEventHandlers: member names are unique 
-PASS Document includes DocumentAndElementEventHandlers: member names are unique 
-PASS Document includes NonElementParentNode: member names are unique 
...
...
...
-PASS MouseEvent interface: existence and properties of interface object 
+TIMEOUT Document includes GlobalEventHandlers: member names are unique Test timed out

[Aakash Jain]

Doesn't seems to happen on build.webkit.org https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fdom%2Fidlharness.worker.html

[Ryan Haddad]

Marked the test as failing to speed up EWS while we investigate in https://trac.webkit.org/changeset/254888/webkit

[Alexey Proskuryakov]

The test is very slow, easily taking 10-20 seconds in post-commit debug. So it's not very surprising that it's a flaky timeout on machines with more cores.

> Marked the test as failing to speed up EWS while we investigate in https://trac.webkit.org/changeset/254888/webkit

It seems like it doesn't need to be marked on release.

[Radar WebKit Bug Importer]

<rdar://problem/59008330>

[Ryan Haddad]

(In reply to Alexey Proskuryakov from comment #4)
> It seems like it doesn't need to be marked on release.
Limited it to debug in https://trac.webkit.org/changeset/255408/webkit

[Ryan Haddad]

Still seeing flakiness after migrating EWS to Catalina:
https://ews-build.s3-us-west-2.amazonaws.com/macOS-Catalina-Debug-WK1-Tests-EWS/r419103-338/results.html
--- /Volumes/Data/worker/macOS-Catalina-Debug-WK1-Tests-EWS/build/layout-test-results/imported/w3c/web-platform-tests/html/dom/idlharness.worker-expected.txt
+++ /Volumes/Data/worker/macOS-Catalina-Debug-WK1-Tests-EWS/build/layout-test-results/imported/w3c/web-platform-tests/html/dom/idlharness.worker-actual.txt
@@ -1,4 +1,6 @@
 Blocked access to external URL http://invalid/
+
+Harness Error (TIMEOUT), message = null
 
 PASS idl_test setup
 PASS idl_test validation

[Robert Jenner]

Created attachment 420150 [details]
Update TestExpectations

[Ryan Haddad]

Comment on attachment 420150 [details]
Update TestExpectations

Clearing flags on attachment: 420150

Committed r272817 (234053@main): <https://commits.webkit.org/234053@main>