Bug 229533 - [EWS] compile-webkit-without-patch failed, but bot ran layout tests anyway and blamed the patch for a pre-existing failure
Summary: [EWS] compile-webkit-without-patch failed, but bot ran layout tests anyway an...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Aakash Jain
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-08-25 20:27 PDT by Ryan Haddad
Modified: 2021-09-03 06:23 PDT (History)
4 users (show)

See Also:


Attachments
Patch (4.86 KB, patch)
2021-08-26 13:38 PDT, Aakash Jain
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2021-08-25 20:27:32 PDT
As you can see in https://ews-build.webkit.org/#/builders/60/builds/10553, the layout-tests and re-run-layout-tests both failed. The bot then proceeded to the compile-webkit-without-patch step and failed to compile. 

Surprisingly, the bot still proceeded to the run-layout-tests-without-patch step, which failed with the following:

16:30:10.005 10889 WebKitTestRunner was not found at /Volumes/Data/worker/macOS-AppleSilicon-Big-Sur-Debug-WK2-Tests-EWS/build/WebKitBuild/Debug/WebKitTestRunner

I would expect that the EWS run would be considered inconclusive or invalid, but instead it blamed the patch under test for introducing a new test failure that really should have been considered pre-existing.

Other examples from around the same time:
https://ews-build.webkit.org/#/builders/60/builds/10560
https://ews-build.webkit.org/#/builders/60/builds/10561
https://ews-build.webkit.org/#/builders/60/builds/10562
https://ews-build.webkit.org/#/builders/60/builds/10563
Comment 1 Radar WebKit Bug Importer 2021-08-25 20:28:02 PDT
<rdar://problem/82371726>
Comment 2 Aakash Jain 2021-08-26 13:38:25 PDT
Created attachment 436560 [details]
Patch
Comment 3 Aakash Jain 2021-08-26 13:39:53 PDT
Tested on uat instance, e.g.: https://ews-build.webkit-uat.org/#/builders/34/builds/40260
Comment 4 EWS 2021-08-26 15:07:20 PDT
Committed r281660 (241013@main): <https://commits.webkit.org/241013@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 436560 [details].
Comment 5 Aakash Jain 2021-08-27 03:34:32 PDT
Restarted buildbot to pick up this change.
Comment 6 Aakash Jain 2021-08-27 04:20:14 PDT Comment hidden (obsolete)