196947 2019-04-15 16:08:41 -0700 ews-build should clearly indicate flaky test failures 2019-04-15 16:37:18 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain ap dewei_zhu lforschler webkit-bug-importer oldest_to_newest 1527493 0 aakash_jain 2019-04-15 16:08:41 -0700 API tests has various flaky failures. Since ews-build runs API tests multiple times, it can notice flaky tests. ews-build should clearly indicate flaky test in the UI when it identifies a flaky test. This would help in easily identifying flaky tests and getting them fixed. 1527496 1 367473 aakash_jain 2019-04-15 16:09:49 -0700 Created attachment 367473 Patch 1527509 2 367473 lforschler 2019-04-15 16:33:15 -0700 Comment on attachment 367473 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=367473&action=review r=me after checking the first/second variable name. > Tools/BuildSlaveSupport/ews-build/steps.py:802 > + self._addToLog('stderr', '\nFailures in API Test second run: {}'.format(first_run_failures)) I think this should be second_run_failures. 1527510 3 aakash_jain 2019-04-15 16:36:28 -0700 Committed r244306: <https://trac.webkit.org/changeset/244306> 1527511 4 webkit-bug-importer 2019-04-15 16:37:18 -0700 <rdar://problem/49923241> 367473 2019-04-15 16:09:49 -0700 2019-04-15 16:33:15 -0700 Patch bug-196947-20190415191106.patch text/plain 2912 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI0NDMwNCkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDEzIEBACisyMDE5LTA0LTE1ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBld3MtYnVpbGQgc2hvdWxkIGNsZWFybHkgaW5kaWNhdGUgZmxha3kgdGVz dCBmYWlsdXJlcworICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/ aWQ9MTk2OTQ3CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAg ICAgKiBCdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHMucHk6CisgICAgICAgIChBbmFs eXplQVBJVGVzdHNSZXN1bHRzLmFuYWx5emVSZXN1bHRzKToKKwogMjAxOS0wNC0xNSAgQnJlbnQg RnVsZ2hhbSAgPGJmdWxnaGFtQGFwcGxlLmNvbT4KIAogICAgICAgICBJbmplY3RlZEJ1bmRsZSBw YXJhbWV0ZXJzIG9mdGVuIG5lZWQgaW5pdGlhbGl6YXRpb24gZnVuY3Rpb24gY2FsbGVkIGJlZm9y ZSB1bmFyY2hpdmluZwpJbmRleDogVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxkL3N0 ZXBzLnB5Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT0KLS0tIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9z dGVwcy5weQkocmV2aXNpb24gMjQ0MjgzKQorKysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdz LWJ1aWxkL3N0ZXBzLnB5CSh3b3JraW5nIGNvcHkpCkBAIC03OTIsMTMgKzc5MiwxNyBAQCBjbGFz cyBBbmFseXplQVBJVGVzdHNSZXN1bHRzKGJ1aWxkc3RlcC5CCiAgICAgICAgIHNlY29uZF9ydW5f ZmFpbHVyZXMgPSBnZXRBUElUZXN0RmFpbHVyZXMoc2Vjb25kX3J1bl9yZXN1bHRzKQogICAgICAg ICBjbGVhbl90cmVlX2ZhaWx1cmVzID0gZ2V0QVBJVGVzdEZhaWx1cmVzKGNsZWFuX3RyZWVfcmVz dWx0cykKIAotICAgICAgICBzZWxmLl9hZGRUb0xvZygnc3RkZXJyJywgJ1xuRmFpbHVyZXMgaW4g QVBJIFRlc3QgZmlyc3QgcnVuOiB7fScuZm9ybWF0KGZpcnN0X3J1bl9mYWlsdXJlcykpCi0gICAg ICAgIHNlbGYuX2FkZFRvTG9nKCdzdGRlcnInLCAnXG5GYWlsdXJlcyBpbiBBUEkgVGVzdCBzZWNv bmQgcnVuOiB7fScuZm9ybWF0KGZpcnN0X3J1bl9mYWlsdXJlcykpCi0gICAgICAgIHNlbGYuX2Fk ZFRvTG9nKCdzdGRlcnInLCAnXG5GYWlsdXJlcyBpbiBBUEkgVGVzdCBvbiBjbGVhbiB0cmVlOiB7 fScuZm9ybWF0KGNsZWFuX3RyZWVfZmFpbHVyZXMpKQogICAgICAgICBmYWlsdXJlc193aXRoX3Bh dGNoID0gZmlyc3RfcnVuX2ZhaWx1cmVzLmludGVyc2VjdGlvbihzZWNvbmRfcnVuX2ZhaWx1cmVz KQorICAgICAgICBmbGFreV9mYWlsdXJlcyA9IGZpcnN0X3J1bl9mYWlsdXJlcy51bmlvbihzZWNv bmRfcnVuX2ZhaWx1cmVzKSAtIGZpcnN0X3J1bl9mYWlsdXJlcy5pbnRlcnNlY3Rpb24oc2Vjb25k X3J1bl9mYWlsdXJlcykKKyAgICAgICAgZmxha3lfZmFpbHVyZXNfc3RyaW5nID0gJywgJy5qb2lu KFtmYWlsdXJlX25hbWUucmVwbGFjZSgnVGVzdFdlYktpdEFQSS4nLCAnJykgZm9yIGZhaWx1cmVf bmFtZSBpbiBmbGFreV9mYWlsdXJlc10pCiAgICAgICAgIG5ld19mYWlsdXJlcyA9IGZhaWx1cmVz X3dpdGhfcGF0Y2ggLSBjbGVhbl90cmVlX2ZhaWx1cmVzCiAgICAgICAgIG5ld19mYWlsdXJlc19z dHJpbmcgPSAnLCAnLmpvaW4oW2ZhaWx1cmVfbmFtZS5yZXBsYWNlKCdUZXN0V2ViS2l0QVBJLics ICcnKSBmb3IgZmFpbHVyZV9uYW1lIGluIG5ld19mYWlsdXJlc10pCiAKKyAgICAgICAgc2VsZi5f YWRkVG9Mb2coJ3N0ZGVycicsICdcbkZhaWx1cmVzIGluIEFQSSBUZXN0IGZpcnN0IHJ1bjoge30n LmZvcm1hdChmaXJzdF9ydW5fZmFpbHVyZXMpKQorICAgICAgICBzZWxmLl9hZGRUb0xvZygnc3Rk ZXJyJywgJ1xuRmFpbHVyZXMgaW4gQVBJIFRlc3Qgc2Vjb25kIHJ1bjoge30nLmZvcm1hdChmaXJz dF9ydW5fZmFpbHVyZXMpKQorICAgICAgICBzZWxmLl9hZGRUb0xvZygnc3RkZXJyJywgJ1xuRmxh a3kgVGVzdHM6IHt9Jy5mb3JtYXQoZmxha3lfZmFpbHVyZXMpKQorICAgICAgICBzZWxmLl9hZGRU b0xvZygnc3RkZXJyJywgJ1xuRmFpbHVyZXMgaW4gQVBJIFRlc3Qgb24gY2xlYW4gdHJlZToge30n LmZvcm1hdChjbGVhbl90cmVlX2ZhaWx1cmVzKSkKKwogICAgICAgICBpZiBuZXdfZmFpbHVyZXM6 CiAgICAgICAgICAgICBzZWxmLl9hZGRUb0xvZygnc3RkZXJyJywgJ1xuTmV3IGZhaWx1cmVzOiB7 fVxuJy5mb3JtYXQobmV3X2ZhaWx1cmVzKSkKICAgICAgICAgICAgIHNlbGYuZmluaXNoZWQoRkFJ TFVSRSkKQEAgLTgxMiw2ICs4MTYsOCBAQCBjbGFzcyBBbmFseXplQVBJVGVzdHNSZXN1bHRzKGJ1 aWxkc3RlcC5CCiAgICAgICAgICAgICBzZWxmLmJ1aWxkLnJlc3VsdHMgPSBTVUNDRVNTCiAgICAg ICAgICAgICBzZWxmLmRlc2NyaXB0aW9uRG9uZSA9ICdQYXNzZWQgQVBJIHRlc3RzJwogICAgICAg ICAgICAgbWVzc2FnZSA9ICdGb3VuZCB7fSBwcmUtZXhpc3RpbmcgQVBJIHRlc3RzIGZhaWx1cmVz Jy5mb3JtYXQobGVuKGNsZWFuX3RyZWVfZmFpbHVyZXMpKQorICAgICAgICAgICAgaWYgZmxha3lf ZmFpbHVyZXM6CisgICAgICAgICAgICAgICAgbWVzc2FnZSArPSAnLiBGbGFreSB0ZXN0czoge30n LmZvcm1hdChmbGFreV9mYWlsdXJlc19zdHJpbmcpCiAgICAgICAgICAgICBzZWxmLmJ1aWxkLmJ1 aWxkRmluaXNoZWQoW21lc3NhZ2VdLCBTVUNDRVNTKQogCiAgICAgQGRlZmVyLmlubGluZUNhbGxi YWNrcwo=