Bug 209226

Summary: [ews] commit-queue should comment on bug in case of test failures
Product: WebKit Reporter: Aakash Jain <aakash_jain>
Component: Tools / TestsAssignee: Aakash Jain <aakash_jain>
Status: RESOLVED FIXED    
Severity: Normal CC: aakash_jain, ap, jbedard, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: Other   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=209227
Bug Depends on:    
Bug Blocks: 201934    
Attachments:
Description Flags
Patch jbedard: review+

Description Aakash Jain 2020-03-18 06:45:02 PDT
[ews] commit-queue should comment on bug and set cq- flag in case of test failures.
Comment 1 Aakash Jain 2020-03-18 07:01:03 PDT
Created attachment 393845 [details]
Patch
Comment 2 Aakash Jain 2020-03-18 07:28:02 PDT
Sample run: https://ews-build.webkit-uat.org/#/builders/26/builds/1593
Comment 3 Jonathan Bedard 2020-03-18 09:45:59 PDT
Comment on attachment 393845 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=393845&action=review

> Tools/BuildSlaveSupport/ews-build/steps.py:1830
> +            self.build.addStepsAfterCurrentStep([CommentOnBug(), SetCommitQueueMinusFlagOnPatch()])

Is this the current behavior? If I recall, this logic doesn't apply to flaky tests, but I can see some circumstances where this behavior is undesirable.
Comment 4 Aakash Jain 2020-03-18 11:09:06 PDT
(In reply to Jonathan Bedard from comment #3)
> Is this the current behavior? If I recall, this logic doesn't apply to flaky tests.
Yes, I think so. The comment and cq- would be when a test failure is confirmed (test consistently failed in first and second run, and passed in clean-tree-run). Test which were found to be flaky during the run would not affect the result.
Comment 5 Aakash Jain 2020-03-18 11:15:37 PDT
Committed r258652: <https://trac.webkit.org/changeset/258652>
Comment 6 Radar WebKit Bug Importer 2020-03-18 11:16:19 PDT
<rdar://problem/60592347>
Comment 7 Aakash Jain 2020-03-19 15:04:58 PDT
This change seems to be working fine.

Patch with test failure: https://ews-build.webkit.org/#/builders/28/builds/63

Flaky test: https://ews-build.webkit.org/#/builders/28/builds/66