Bug 215231 - [ews] Add method to send email notifications to patch author for layout test failures
Summary: [ews] Add method to send email notifications to patch author for layout test ...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Aakash Jain
URL:
Keywords: InRadar
Depends on:
Blocks: 215220
  Show dependency treegraph
 
Reported: 2020-08-06 11:45 PDT by Aakash Jain
Modified: 2020-08-24 12:12 PDT (History)
8 users (show)

See Also:


Attachments
Patch (2.48 KB, patch)
2020-08-06 11:48 PDT, Aakash Jain
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Aakash Jain 2020-08-06 11:45:12 PDT
Add method to send email notifications to patch author for layout test failures on their patch in EWS.
Comment 1 Aakash Jain 2020-08-06 11:48:57 PDT
Created attachment 406097 [details]
Patch
Comment 2 Jonathan Bedard 2020-08-06 16:22:17 PDT
Comment on attachment 406097 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=406097&action=review

> Tools/BuildSlaveSupport/ews-build/steps.py:2108
> +    def send_email_for_new_test_failures(self, test_names):

I'm a bit more skeptical of this one, although we aren't using it yet, so I'll r+ it.

We tend to have way more false-positives for test failures in EWS then we do false-positives for build failures.
Comment 3 Aakash Jain 2020-08-07 04:24:14 PDT
(In reply to Jonathan Bedard from comment #2)
> We tend to have way more false-positives for test failures in EWS then we do false-positives for build failures.
Agree. I believe false-positives for test failures in EWS are mostly because of flaky tests. We would have to try to reduce flaky tests separately (few ideas: by fixing flaky tests, adding stress tests EWS, using information from results database API etc), or have to live with some amount of flaky tests.

Engineer would still find these notifications useful. Tester queues takes longer than builder queues, and so email notifications for tester queues would be quite useful, so that engineers don't have to keep polling Bugzilla. For e.g.: In Bug 200399 Daniel Bates requested these email notification and provided an example of ios-wk2 queue (which is a tester queue).
Comment 4 EWS 2020-08-07 04:31:20 PDT
Committed r265368: <https://trac.webkit.org/changeset/265368>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 406097 [details].
Comment 5 Radar WebKit Bug Importer 2020-08-07 04:32:34 PDT
<rdar://problem/66674272>