215231 2020-08-06 11:45:12 -0700 [ews] Add method to send email notifications to patch author for layout test failures 2020-08-24 12:12:30 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=215219 https://bugs.webkit.org/show_bug.cgi?id=215776 InRadar P2 Normal --- 215220 1 aakash_jain aakash_jain aakash_jain ap cdumez dbates hi jbedard ryanhaddad webkit-bug-importer oldest_to_newest 1678329 0 aakash_jain 2020-08-06 11:45:12 -0700 Add method to send email notifications to patch author for layout test failures on their patch in EWS. 1678331 1 406097 aakash_jain 2020-08-06 11:48:57 -0700 Created attachment 406097 Patch 1678452 2 406097 jbedard 2020-08-06 16:22:17 -0700 Comment on attachment 406097 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=406097&action=review > Tools/BuildSlaveSupport/ews-build/steps.py:2108 > + def send_email_for_new_test_failures(self, test_names): I'm a bit more skeptical of this one, although we aren't using it yet, so I'll r+ it. We tend to have way more false-positives for test failures in EWS then we do false-positives for build failures. 1678548 3 aakash_jain 2020-08-07 04:24:14 -0700 (In reply to Jonathan Bedard from comment #2) > We tend to have way more false-positives for test failures in EWS then we do false-positives for build failures. Agree. I believe false-positives for test failures in EWS are mostly because of flaky tests. We would have to try to reduce flaky tests separately (few ideas: by fixing flaky tests, adding stress tests EWS, using information from results database API etc), or have to live with some amount of flaky tests. Engineer would still find these notifications useful. Tester queues takes longer than builder queues, and so email notifications for tester queues would be quite useful, so that engineers don't have to keep polling Bugzilla. For e.g.: In Bug 200399 Daniel Bates requested these email notification and provided an example of ios-wk2 queue (which is a tester queue). 1678549 4 ews-feeder 2020-08-07 04:31:20 -0700 Committed r265368: <https://trac.webkit.org/changeset/265368> All reviewed patches have been landed. Closing bug and clearing flags on attachment 406097. 1678550 5 webkit-bug-importer 2020-08-07 04:32:34 -0700 <rdar://problem/66674272> 406097 2020-08-06 11:48:57 -0700 2020-08-07 04:31:21 -0700 Patch bug-215231-20200806144856.patch text/plain 2536 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI2NTM0MSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDEzIEBACisyMDIwLTA4LTA2ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbZXdzXSBBZGQgbWV0aG9kIHRvIHNlbmQgZW1haWwgbm90aWZpY2F0aW9u cyB0byBwYXRjaCBhdXRob3IgZm9yIGxheW91dCB0ZXN0IGZhaWx1cmVzCisgICAgICAgIGh0dHBz Oi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0yMTUyMzEKKworICAgICAgICBSZXZp ZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICAqIEJ1aWxkU2xhdmVTdXBwb3J0L2V3 cy1idWlsZC9zdGVwcy5weToKKyAgICAgICAgKEFuYWx5emVMYXlvdXRUZXN0c1Jlc3VsdHMuc2Vu ZF9lbWFpbF9mb3JfbmV3X3Rlc3RfZmFpbHVyZXMpOgorCiAyMDIwLTA4LTA2ICBBYWthc2ggSmFp biAgPGFha2FzaF9qYWluQGFwcGxlLmNvbT4KIAogICAgICAgICBbZXdzXSBBZGQgbWV0aG9kIHRv IHNlbmQgZW1haWwgbm90aWZpY2F0aW9ucyB0byBwYXRjaCBhdXRob3IgZm9yIGJ1aWxkIGZhaWx1 cmUKSW5kZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weQo9PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09Ci0tLSBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHMucHkJKHJl dmlzaW9uIDI2NTM0MSkKKysrIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVw cy5weQkod29ya2luZyBjb3B5KQpAQCAtMjEwNSw2ICsyMTA1LDI1IEBAIGNsYXNzIEFuYWx5emVM YXlvdXRUZXN0c1Jlc3VsdHMoYnVpbGRzdGUKICAgICAgICAgZXhjZXB0IEV4Y2VwdGlvbiBhcyBl OgogICAgICAgICAgICAgcHJpbnQoJ0Vycm9yIGluIHNlbmRpbmcgZW1haWwgZm9yIHByZS1leGlz dGluZyBmYWlsdXJlOiB7fScuZm9ybWF0KGUpKQogCisgICAgZGVmIHNlbmRfZW1haWxfZm9yX25l d190ZXN0X2ZhaWx1cmVzKHNlbGYsIHRlc3RfbmFtZXMpOgorICAgICAgICB0cnk6CisgICAgICAg ICAgICBidWlsZGVyX25hbWUgPSBzZWxmLmdldFByb3BlcnR5KCdidWlsZGVybmFtZScsICcnKQor ICAgICAgICAgICAgYnVnX3RpdGxlID0gc2VsZi5nZXRQcm9wZXJ0eSgnYnVnX3RpdGxlJywgJycp CisgICAgICAgICAgICB3b3JrZXJfbmFtZSA9IHNlbGYuZ2V0UHJvcGVydHkoJ3dvcmtlcm5hbWUn LCAnJykKKyAgICAgICAgICAgIHBhdGNoX2lkID0gc2VsZi5nZXRQcm9wZXJ0eSgncGF0Y2hfaWQn LCAnJykKKyAgICAgICAgICAgIHBhdGNoX2F1dGhvciA9IHNlbGYuZ2V0UHJvcGVydHkoJ3BhdGNo X2F1dGhvcicsICcnKQorICAgICAgICAgICAgYnVpbGRfdXJsID0gJ3t9Iy9idWlsZGVycy97fS9i dWlsZHMve30nLmZvcm1hdChzZWxmLm1hc3Rlci5jb25maWcuYnVpbGRib3RVUkwsIHNlbGYuYnVp bGQuX2J1aWxkZXJpZCwgc2VsZi5idWlsZC5udW1iZXIpCisgICAgICAgICAgICB0ZXN0X25hbWVz X3N0cmluZyA9ICctICcgKyAnXG4tICcuam9pbih0ZXN0X25hbWVzKQorCisgICAgICAgICAgICBl bWFpbF9zdWJqZWN0ID0gJ0xheW91dCB0ZXN0IGZhaWx1cmUgZm9yIFBhdGNoIHt9OiB7fSAnLmZv cm1hdChwYXRjaF9pZCwgYnVnX3RpdGxlKQorICAgICAgICAgICAgZW1haWxfdGV4dCA9ICdFV1Mg aGFzIGRldGVjdGVkIHRlc3QgZmFpbHVyZSBvbiB7fSB3aGlsZSB0ZXN0aW5nIFBhdGNoIHt9Licu Zm9ybWF0KGJ1aWxkZXJfbmFtZSwgcGF0Y2hfaWQpCisgICAgICAgICAgICBlbWFpbF90ZXh0ICs9 ICdcblxuRnVsbCBkZXRhaWxzIGFyZSBhdmFpbGFibGUgYXQ6IHt9XG5cblBhdGNoIGF1dGhvcjog e30nLmZvcm1hdChidWlsZF91cmwsIHBhdGNoX2F1dGhvcikKKyAgICAgICAgICAgIGVtYWlsX3Rl eHQgKz0gJ1xuXG5MYXlvdXQgdGVzdCBmYWlsdXJlOlxuXG57fScuZm9ybWF0KHRlc3RfbmFtZXNf c3RyaW5nKQorICAgICAgICAgICAgZW1haWxfdGV4dCArPSAnXG5cblRvIHVuc3Vic2NyaWJsZSBm cm9tIHRoZXNlIG5vdGlmaWNhdGlvbnMgb3IgdG8gcHJvdmlkZSBhbnkgZmVlZGJhY2sgcGxlYXNl IGVtYWlsIGFha2FzaF9qYWluQGFwcGxlLmNvbScKKyAgICAgICAgICAgIHNlbmRfZW1haWwoW3Bh dGNoX2F1dGhvcl0sIGVtYWlsX3N1YmplY3QsIGVtYWlsX3RleHQpCisgICAgICAgIGV4Y2VwdCBF eGNlcHRpb24gYXMgZToKKyAgICAgICAgICAgIHByaW50KCdFcnJvciBpbiBzZW5kaW5nIGVtYWls IGZvciBuZXcgbGF5b3V0IHRlc3QgZmFpbHVyZXM6IHt9Jy5mb3JtYXQoZSkpCisKICAgICBkZWYg X3JlcG9ydF9mbGFreV90ZXN0cyhzZWxmLCBmbGFreV90ZXN0cyk6CiAgICAgICAgICNUT0RPOiBp bXBsZW1lbnQgdGhpcwogICAgICAgICBwYXNzCg==