Bug 215231 – [ews] Add method to send email notifications to patch author for layout test failures

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Aakash Jain]

Add method to send email notifications to patch author for layout test failures on their patch in EWS.

[Aakash Jain]

Created attachment 406097 [details]
Patch

[Jonathan Bedard]

Comment on attachment 406097 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=406097&action=review

> Tools/BuildSlaveSupport/ews-build/steps.py:2108
> +    def send_email_for_new_test_failures(self, test_names):

I'm a bit more skeptical of this one, although we aren't using it yet, so I'll r+ it.

We tend to have way more false-positives for test failures in EWS then we do false-positives for build failures.

[Aakash Jain]

(In reply to Jonathan Bedard from comment #2)
> We tend to have way more false-positives for test failures in EWS then we do false-positives for build failures.
Agree. I believe false-positives for test failures in EWS are mostly because of flaky tests. We would have to try to reduce flaky tests separately (few ideas: by fixing flaky tests, adding stress tests EWS, using information from results database API etc), or have to live with some amount of flaky tests.

Engineer would still find these notifications useful. Tester queues takes longer than builder queues, and so email notifications for tester queues would be quite useful, so that engineers don't have to keep polling Bugzilla. For e.g.: In Bug 200399 Daniel Bates requested these email notification and provided an example of ios-wk2 queue (which is a tester queue).

[EWS]

Committed r265368: <https://trac.webkit.org/changeset/265368>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 406097 [details].

[Radar WebKit Bug Importer]

<rdar://problem/66674272>