| Summary: | EWS bot should fail if a new test is missing its result | ||||||
|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Sam Weinig <sam> | ||||
| Component: | Tools / Tests | Assignee: | Aakash Jain <aakash_jain> | ||||
| Status: | RESOLVED FIXED | ||||||
| Severity: | Normal | CC: | aakash_jain, ap, jbedard, lforschler, megan_gardner, ryanhaddad, simon.fraser, webkit-bot-watchers-bugzilla, webkit-bug-importer | ||||
| Priority: | P2 | Keywords: | InRadar | ||||
| Version: | WebKit Nightly Build | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Attachments: |
|
||||||
|
Description
Sam Weinig
2017-10-01 12:44:42 PDT
*** Bug 181940 has been marked as a duplicate of this bug. *** Noticed this in: https://ews-build.webkit.org/#/builders/24/builds/28039. Patch 411263 added two tests with missing results. layout-tests step indicated '2 missing results' but did not mark the step/build as failed. The fix is to simply mark the build step as failed, that would cause layout-tests to be retried, and then retry on clean build, and since clean build would pass, it would cause the build to be marked as failed. Created attachment 411436 [details]
Patch
Comment on attachment 411436 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=411436&action=review > Tools/BuildSlaveSupport/ews-build/steps.py:2000 > + elif line.find('missing results') >= 0: Added this elif explicitly for readability. Can remove this as well if reviewers feel strongly about it. Comment on attachment 411436 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=411436&action=review >> Tools/BuildSlaveSupport/ews-build/steps.py:2000 >> + elif line.find('missing results') >= 0: > > Added this elif explicitly for readability. Can remove this as well if reviewers feel strongly about it. I'm fine with the elif, but it seems like the failure condition should be first. (since in the event a line contained both 'flakes' and 'missing results', we would want to mark the build as failed). I know this concern is more about purity than practicality at the moment, though. (In reply to Jonathan Bedard from comment #6) > since in the event a line contained both 'flakes' and 'missing results' 'flakes' and 'missing results' words aren't being checked literally, they store output of regular expressions matching from https://trac.webkit.org/browser/webkit/trunk/Tools/BuildSlaveSupport/ews-build/steps.py#L1949 Committed r268526: <https://trac.webkit.org/changeset/268526> All reviewed patches have been landed. Closing bug and clearing flags on attachment 411436 [details]. Restarted EWS server to pick up this change. |