Bug 203756

Summary: [ews] Perform validation of patch before retrying API and layout tests
Product: WebKit Reporter: Aakash Jain <aakash_jain>
Component: Tools / TestsAssignee: Aakash Jain <aakash_jain>
Status: RESOLVED FIXED    
Severity: Normal CC: aakash_jain, ap, commit-queue, jbedard, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: Other   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=201924
https://bugs.webkit.org/show_bug.cgi?id=160739
https://bugs.webkit.org/show_bug.cgi?id=204222
Attachments:
Description Flags
Patch none

Description Aakash Jain 2019-11-01 13:44:15 PDT
Some tests (like API and layout-tests) takes a while to run (15-50 mins depending on test-suite). In case of failure, we retry the tests, and if failures are noticed in retry as well, then we un-apply the patch, build the ToT and run the tests again. All these steps might take a while to complete. 

Meanwhile the patch might get obsolete or r-. EWS should not continue to process such patches, as it would be wasteful. We should check the patch status before running any time-consuming step (like running tests, or building without patch). 

This is similar to old EWS behavior in: https://bugs.webkit.org/show_bug.cgi?id=160739


Note that if the bug is closed, we should still continue to process the patch, since re-trying tests indicates that the patch might have broken something, and having the EWS results would be valuable even after patch is landed.

Also note that this doesn't not terminate the on-going steps, which Bug 201924 aims to achieve.
Comment 1 Aakash Jain 2019-11-01 14:22:33 PDT
Created attachment 382627 [details]
Patch
Comment 2 Aakash Jain 2019-11-01 14:29:02 PDT
Sample runs:
API-tests: https://ews-build.webkit-uat.org/#/builders/4/builds/78

Layout-tests: https://ews-build.webkit-uat.org/#/builders/4/builds/78 

(Note that in above builds, for testing, I skipped the obsolete check in first few validate-patch steps).
Comment 3 WebKit Commit Bot 2019-11-04 20:24:48 PST
Comment on attachment 382627 [details]
Patch

Clearing flags on attachment: 382627

Committed r252037: <https://trac.webkit.org/changeset/252037>
Comment 4 WebKit Commit Bot 2019-11-04 20:24:50 PST
All reviewed patches have been landed.  Closing bug.
Comment 5 Radar WebKit Bug Importer 2019-11-04 20:25:17 PST
<rdar://problem/56891516>