215949 2020-08-28 13:34:35 -0700 [ews] Notify bot watchers about pre-existing and flaky test failures for API tests 2020-09-04 11:52:19 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=215948 https://bugs.webkit.org/show_bug.cgi?id=207475 InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain ap jbedard ryanhaddad webkit-bug-importer oldest_to_newest 1683756 0 aakash_jain 2020-08-28 13:34:35 -0700 Notify bot watchers about pre-existing and flaky test failures for API tests found in EWS. 1685602 1 407957 aakash_jain 2020-09-04 04:36:45 -0700 Created attachment 407957 Patch 1685769 2 407957 jbedard 2020-09-04 11:18:04 -0700 Comment on attachment 407957 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=407957&action=review > Tools/BuildSlaveSupport/ews-build/steps.py:2606 > + def send_email_for_flaky_failure(self, test_name): I'm find with this as-is, but it does seem like we're duplicating quite a bit of code for layout test emails. Seems like we might want a base "AnalyzeTestResults" step with a "send_email_for_flaky_failure(self, test_name):" function that pulls from a "suite" variable 1685791 3 aakash_jain 2020-09-04 11:43:35 -0700 Yeah, will do that in a separate patch. 1685795 4 ews-feeder 2020-09-04 11:51:03 -0700 Committed r266626: <https://trac.webkit.org/changeset/266626> All reviewed patches have been landed. Closing bug and clearing flags on attachment 407957. 1685798 5 webkit-bug-importer 2020-09-04 11:52:19 -0700 <rdar://problem/68358168> 407957 2020-09-04 04:36:45 -0700 2020-09-04 11:51:04 -0700 Patch bug-215949-20200904073644.patch text/plain 4015 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI2NjU5NikKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE2IEBACisyMDIwLTA5LTA0ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbZXdzXSBOb3RpZnkgYm90IHdhdGNoZXJzIGFib3V0IHByZS1leGlzdGlu ZyBhbmQgZmxha3kgdGVzdCBmYWlsdXJlcyBmb3IgQVBJIHRlc3RzCisgICAgICAgIGh0dHBzOi8v YnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0yMTU5NDkKKworICAgICAgICBSZXZpZXdl ZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICAqIEJ1aWxkU2xhdmVTdXBwb3J0L2V3cy1i dWlsZC9zdGVwcy5weToKKyAgICAgICAgKEFuYWx5emVBUElUZXN0c1Jlc3VsdHMuYW5hbHl6ZVJl c3VsdHMpOgorICAgICAgICAoQW5hbHl6ZUFQSVRlc3RzUmVzdWx0cy5zZW5kX2VtYWlsX2Zvcl9m bGFreV9mYWlsdXJlKToKKyAgICAgICAgKEFuYWx5emVBUElUZXN0c1Jlc3VsdHMpOgorICAgICAg ICAoQW5hbHl6ZUFQSVRlc3RzUmVzdWx0cy5zZW5kX2VtYWlsX2Zvcl9wcmVfZXhpc3RpbmdfZmFp bHVyZSk6CisKIDIwMjAtMDktMDQgIENhcmxvcyBHYXJjaWEgQ2FtcG9zICA8Y2dhcmNpYUBpZ2Fs aWEuY29tPgogCiAgICAgICAgIFtHTElCXSBTdG9wIHVzaW5nIGZpcmVmb3ggdXNlciBhZ2VudCBx dWlyayBmb3IgZ29vZ2xlIGRvY3MKSW5kZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1i dWlsZC9zdGVwcy5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3Mt YnVpbGQvc3RlcHMucHkJKHJldmlzaW9uIDI2NjU5NikKKysrIFRvb2xzL0J1aWxkU2xhdmVTdXBw b3J0L2V3cy1idWlsZC9zdGVwcy5weQkod29ya2luZyBjb3B5KQpAQCAtMjU1NiwxMCArMjU1Niwx NCBAQCBjbGFzcyBBbmFseXplQVBJVGVzdHNSZXN1bHRzKGJ1aWxkc3RlcC5CCiAgICAgICAgICAg ICBtZXNzYWdlID0gJycKICAgICAgICAgICAgIGlmIGNsZWFuX3RyZWVfZmFpbHVyZXM6CiAgICAg ICAgICAgICAgICAgbWVzc2FnZSA9ICdGb3VuZCB7fSBwcmUtZXhpc3RpbmcgQVBJIHRlc3QgZmFp bHVyZXt9OiB7fScuZm9ybWF0KGxlbihjbGVhbl90cmVlX2ZhaWx1cmVzKSwgcGx1cmFsU3VmZml4 LCBjbGVhbl90cmVlX2ZhaWx1cmVzX3N0cmluZykKKyAgICAgICAgICAgICAgICBmb3IgY2xlYW5f dHJlZV9mYWlsdXJlIGluIGNsZWFuX3RyZWVfZmFpbHVyZXM6CisgICAgICAgICAgICAgICAgICAg IHNlbGYuc2VuZF9lbWFpbF9mb3JfcHJlX2V4aXN0aW5nX2ZhaWx1cmUoY2xlYW5fdHJlZV9mYWls dXJlKQogICAgICAgICAgICAgaWYgbGVuKGNsZWFuX3RyZWVfZmFpbHVyZXMpID4gc2VsZi5OVU1f QVBJX0ZBSUxVUkVTX1RPX0RJU1BMQVk6CiAgICAgICAgICAgICAgICAgbWVzc2FnZSArPSAnIC4u LicKICAgICAgICAgICAgIGlmIGZsYWt5X2ZhaWx1cmVzOgogICAgICAgICAgICAgICAgIG1lc3Nh Z2UgKz0gJyBGb3VuZCBmbGFreSB0ZXN0czoge30nLmZvcm1hdChmbGFreV9mYWlsdXJlc19zdHJp bmcpCisgICAgICAgICAgICAgICAgZm9yIGZsYWt5X2ZhaWx1cmUgaW4gZmxha3lfZmFpbHVyZXM6 CisgICAgICAgICAgICAgICAgICAgIHNlbGYuc2VuZF9lbWFpbF9mb3JfZmxha3lfZmFpbHVyZShm bGFreV9mYWlsdXJlKQogICAgICAgICAgICAgc2VsZi5idWlsZC5idWlsZEZpbmlzaGVkKFttZXNz YWdlXSwgU1VDQ0VTUykKIAogICAgIEBkZWZlci5pbmxpbmVDYWxsYmFja3MKQEAgLTI1OTksNiAr MjYwMywzMSBAQCBjbGFzcyBBbmFseXplQVBJVGVzdHNSZXN1bHRzKGJ1aWxkc3RlcC5CCiAgICAg ICAgIGV4Y2VwdCBFeGNlcHRpb24gYXMgZXg6CiAgICAgICAgICAgICBzZWxmLl9hZGRUb0xvZygn c3RkZXJyJywgJ0VSUk9SOiB1bmFibGUgdG8gcGFyc2UgZGF0YSwgZXhjZXB0aW9uOiB7fScuZm9y bWF0KGV4KSkKIAorICAgIGRlZiBzZW5kX2VtYWlsX2Zvcl9mbGFreV9mYWlsdXJlKHNlbGYsIHRl c3RfbmFtZSk6CisgICAgICAgIHRyeToKKyAgICAgICAgICAgIGJ1aWxkZXJfbmFtZSA9IHNlbGYu Z2V0UHJvcGVydHkoJ2J1aWxkZXJuYW1lJywgJycpCisgICAgICAgICAgICB3b3JrZXJfbmFtZSA9 IHNlbGYuZ2V0UHJvcGVydHkoJ3dvcmtlcm5hbWUnLCAnJykKKyAgICAgICAgICAgIGJ1aWxkX3Vy bCA9ICd7fSMvYnVpbGRlcnMve30vYnVpbGRzL3t9Jy5mb3JtYXQoc2VsZi5tYXN0ZXIuY29uZmln LmJ1aWxkYm90VVJMLCBzZWxmLmJ1aWxkLl9idWlsZGVyaWQsIHNlbGYuYnVpbGQubnVtYmVyKQor ICAgICAgICAgICAgaGlzdG9yeV91cmwgPSAne30/c3VpdGU9YXBpLXRlc3RzJnRlc3Q9e30nLmZv cm1hdChSRVNVTFRTX0RCX1VSTCwgdGVzdF9uYW1lKQorCisgICAgICAgICAgICBlbWFpbF9zdWJq ZWN0ID0gdSdGbGFreSB0ZXN0OiB7fScuZm9ybWF0KHRlc3RfbmFtZSkKKyAgICAgICAgICAgIGVt YWlsX3RleHQgPSAnRmxha3kgdGVzdDoge31cblxuQnVpbGQ6IHt9XG5cbkJ1aWxkZXI6IHt9XG5c bldvcmtlcjoge31cblxuSGlzdG9yeToge30nLmZvcm1hdCh0ZXN0X25hbWUsIGJ1aWxkX3VybCwg YnVpbGRlcl9uYW1lLCB3b3JrZXJfbmFtZSwgaGlzdG9yeV91cmwpCisgICAgICAgICAgICBzZW5k X2VtYWlsX3RvX2JvdF93YXRjaGVycyhlbWFpbF9zdWJqZWN0LCBlbWFpbF90ZXh0LCAnZmxha3kt e30nLmZvcm1hdCh0ZXN0X25hbWUpKQorICAgICAgICBleGNlcHQgRXhjZXB0aW9uIGFzIGU6Cisg ICAgICAgICAgICBwcmludCgnRXJyb3IgaW4gc2VuZGluZyBlbWFpbCBmb3IgZmxha3kgZmFpbHVy ZToge30nLmZvcm1hdChlKSkKKworICAgIGRlZiBzZW5kX2VtYWlsX2Zvcl9wcmVfZXhpc3Rpbmdf ZmFpbHVyZShzZWxmLCB0ZXN0X25hbWUpOgorICAgICAgICB0cnk6CisgICAgICAgICAgICBidWls ZGVyX25hbWUgPSBzZWxmLmdldFByb3BlcnR5KCdidWlsZGVybmFtZScsICcnKQorICAgICAgICAg ICAgd29ya2VyX25hbWUgPSBzZWxmLmdldFByb3BlcnR5KCd3b3JrZXJuYW1lJywgJycpCisgICAg ICAgICAgICBidWlsZF91cmwgPSAne30jL2J1aWxkZXJzL3t9L2J1aWxkcy97fScuZm9ybWF0KHNl bGYubWFzdGVyLmNvbmZpZy5idWlsZGJvdFVSTCwgc2VsZi5idWlsZC5fYnVpbGRlcmlkLCBzZWxm LmJ1aWxkLm51bWJlcikKKyAgICAgICAgICAgIGhpc3RvcnlfdXJsID0gJ3t9P3N1aXRlPWFwaS10 ZXN0cyZ0ZXN0PXt9Jy5mb3JtYXQoUkVTVUxUU19EQl9VUkwsIHRlc3RfbmFtZSkKKworICAgICAg ICAgICAgZW1haWxfc3ViamVjdCA9IHUnUHJlLWV4aXN0aW5nIHRlc3QgZmFpbHVyZToge30nLmZv cm1hdCh0ZXN0X25hbWUpCisgICAgICAgICAgICBlbWFpbF90ZXh0ID0gJ1Rlc3Qge30gZmFpbGVk IG9uIGNsZWFuIHRyZWUgcnVuIGluIHt9LlxuXG5CdWlsZGVyOiB7fVxuXG5Xb3JrZXI6IHt9XG5c bkhpc3Rvcnk6IHt9Jy5mb3JtYXQodGVzdF9uYW1lLCBidWlsZF91cmwsIGJ1aWxkZXJfbmFtZSwg d29ya2VyX25hbWUsIGhpc3RvcnlfdXJsKQorICAgICAgICAgICAgc2VuZF9lbWFpbF90b19ib3Rf d2F0Y2hlcnMoZW1haWxfc3ViamVjdCwgZW1haWxfdGV4dCwgJ3ByZWV4aXN0aW5nLXt9Jy5mb3Jt YXQodGVzdF9uYW1lKSkKKyAgICAgICAgZXhjZXB0IEV4Y2VwdGlvbiBhcyBlOgorICAgICAgICAg ICAgcHJpbnQoJ0Vycm9yIGluIHNlbmRpbmcgZW1haWwgZm9yIHByZS1leGlzdGluZyBmYWlsdXJl OiB7fScuZm9ybWF0KGUpKQogCiBjbGFzcyBBcmNoaXZlVGVzdFJlc3VsdHMoc2hlbGwuU2hlbGxD b21tYW5kKToKICAgICBjb21tYW5kID0gWydweXRob24nLCAnVG9vbHMvQnVpbGRTbGF2ZVN1cHBv cnQvdGVzdC1yZXN1bHQtYXJjaGl2ZScsCg==