Bug 229634

Summary: ews might mark build as successful if JSC tests fail to run
Product: WebKit Reporter: Aakash Jain <aakash_jain>
Component: Tools / TestsAssignee: Aakash Jain <aakash_jain>
Severity: Normal CC: aakash_jain, angelos, aperez, ap, clopez, dpino, jbedard, ryanhaddad, sokenso66, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=215944
Description Flags
Patch none

Description Aakash Jain 2021-08-27 16:34:39 PDT
EWS might mark build as successful in certain conditions if JSC tests fail to run (either due to some infrastructure issue or the patch breaking the tests so that they can't generate json results). For e.g. in https://ews-build.webkit.org/#/builders/1/builds/44335 tests failed to run properly with the patch. However the build was marked successful. Similarly, in https://ews-build.webkit.org/#/builders/46/builds/15944 tests were hanging (and were killed by buildbot after timeout), however the build was marked successful.

Similar to Bug 215944 for layout-tests.
Comment 1 Aakash Jain 2021-08-30 03:36:13 PDT
Created attachment 436762 [details]
Comment 2 EWS 2021-08-30 09:02:21 PDT
Committed r281761 (241102@main): <https://commits.webkit.org/241102@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 436762 [details].
Comment 3 Radar WebKit Bug Importer 2021-08-30 09:03:18 PDT
Comment 4 Aakash Jain 2021-08-30 09:16:12 PDT
Restarted buildbot to pick up this change.
Comment 6 Aakash Jain 2021-09-10 08:27:33 PDT
*** Bug 219543 has been marked as a duplicate of this bug. ***