Bug 203756 - [ews] Perform validation of patch before retrying API and layout tests
Summary: [ews] Perform validation of patch before retrying API and layout tests
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Aakash Jain
Keywords: InRadar
Depends on:
Reported: 2019-11-01 13:44 PDT by Aakash Jain
Modified: 2019-11-15 04:11 PST (History)
5 users (show)

See Also:

Patch (7.15 KB, patch)
2019-11-01 14:22 PDT, Aakash Jain
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Aakash Jain 2019-11-01 13:44:15 PDT
Some tests (like API and layout-tests) takes a while to run (15-50 mins depending on test-suite). In case of failure, we retry the tests, and if failures are noticed in retry as well, then we un-apply the patch, build the ToT and run the tests again. All these steps might take a while to complete. 

Meanwhile the patch might get obsolete or r-. EWS should not continue to process such patches, as it would be wasteful. We should check the patch status before running any time-consuming step (like running tests, or building without patch). 

This is similar to old EWS behavior in: https://bugs.webkit.org/show_bug.cgi?id=160739

Note that if the bug is closed, we should still continue to process the patch, since re-trying tests indicates that the patch might have broken something, and having the EWS results would be valuable even after patch is landed.

Also note that this doesn't not terminate the on-going steps, which Bug 201924 aims to achieve.
Comment 1 Aakash Jain 2019-11-01 14:22:33 PDT
Created attachment 382627 [details]
Comment 2 Aakash Jain 2019-11-01 14:29:02 PDT
Sample runs:
API-tests: https://ews-build.webkit-uat.org/#/builders/4/builds/78

Layout-tests: https://ews-build.webkit-uat.org/#/builders/4/builds/78 

(Note that in above builds, for testing, I skipped the obsolete check in first few validate-patch steps).
Comment 3 WebKit Commit Bot 2019-11-04 20:24:48 PST
Comment on attachment 382627 [details]

Clearing flags on attachment: 382627

Committed r252037: <https://trac.webkit.org/changeset/252037>
Comment 4 WebKit Commit Bot 2019-11-04 20:24:50 PST
All reviewed patches have been landed.  Closing bug.
Comment 5 Radar WebKit Bug Importer 2019-11-04 20:25:17 PST