218618 2020-11-05 08:34:33 -0800 [ews] Patch which break layout-test shouldn't cause infinite RETRY loop 2024-02-01 05:48:19 -0800 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=215944 https://bugs.webkit.org/show_bug.cgi?id=268548 InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain angelos ap dean_johnson dewei_zhu jbedard ryanhaddad webkit-bug-importer oldest_to_newest 1704874 0 aakash_jain 2020-11-05 08:34:33 -0800 Currently we retry layout-tests when there is an exception in running the tests (exit code 254). We do it to handle any unexpected infrastructure issues (like simulators not booting up properly). We keep retrying until the issue is fixed. However, because of this infinite retry, patches which actually break layout-test are also retried, and go into an infinite retry loop (until the patch is obsolete, r-, bug is closed, or someone manually stops the build). e.g.: https://bugs.webkit.org/show_bug.cgi?id=218352#c2 caused continuos retries (https://ews-build.webkit.org/#/builders/30/builds/21779, https://ews-build.webkit.org/#/builders/30/builds/21778, https://ews-build.webkit.org/#/builders/30/builds/21777 etc.) We shouldn't automatically retry immediately when layout-tests unexpectedly fail. We should re-run the tests on clean tree as well. If the clean tree passes, it indicates that the issue is with the patch, and the build should be marked as FAILURE (and red ews bubble). Otherwise (if clean-tree results are also failure) we should retry the build. 1704877 1 aakash_jain 2020-11-05 08:46:43 -0800 This is also a problem (infinite retry) when a patch breaks layout-tests in some other manner (without exit code 254). e.g.: patch in https://bugs.webkit.org/show_bug.cgi?id=218306#c4 caused continuous retries (https://ews-build.webkit.org/#/builders/51/builds/672, https://ews-build.webkit.org/#/builders/51/builds/680) 1704878 2 413305 aakash_jain 2020-11-05 08:46:52 -0800 Created attachment 413305 Patch 1704978 3 ews-feeder 2020-11-05 12:40:37 -0800 Committed r269467: <https://trac.webkit.org/changeset/269467> All reviewed patches have been landed. Closing bug and clearing flags on attachment 413305. 1704979 4 webkit-bug-importer 2020-11-05 12:41:19 -0800 <rdar://problem/71089620> 413305 2020-11-05 08:46:52 -0800 2020-11-05 12:40:37 -0800 Patch bug-218618-20201105114651.patch text/plain 5759 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI2OTQzMykKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE4IEBACisyMDIwLTExLTA1ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbZXdzXSBQYXRjaCB3aGljaCBicmVhayBsYXlvdXQtdGVzdCBzaG91bGRu J3QgY2F1c2UgaW5maW5pdGUgUkVUUlkgbG9vcAorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0 Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MjE4NjE4CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZ IChPT1BTISkuCisKKyAgICAgICAgKiBDSVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5OgorICAg ICAgICAoUnVuV2ViS2l0VGVzdHMuZXZhbHVhdGVSZXN1bHQpOiBEbyBub3QgcmV0cnkgd2hlbiBs YXlvdXQtdGVzdHMgZmFpbCB3aXRoIGV4aXQgY29kZSAyNTQuIEluc3RlYWQgZmFpbCB0aGUgYnVp bGQtc3RlcCwgd2hpY2gKKyAgICAgICAgd2lsbCB0cmlnZ2VyIHJlLXJ1biBhbmQgcnVuLWxheW91 dC10ZXN0cy13aXRob3V0LXBhdGNoIHN0ZXBzLgorICAgICAgICAoUnVuV2ViS2l0VGVzdHNXaXRo b3V0UGF0Y2guZXZhbHVhdGVDb21tYW5kKTogU2V0IGNsZWFuX3RyZWVfcnVuX3N0YXR1cyBidWls ZCBwcm9wZXJ0eS4KKyAgICAgICAgKEFuYWx5emVMYXlvdXRUZXN0c1Jlc3VsdHMucmVwb3J0X2Zh aWx1cmUpOgorICAgICAgICAoQW5hbHl6ZUxheW91dFRlc3RzUmVzdWx0cy5zdGFydCk6IElmIGNs ZWFuIHRyZWUgcnVuIHBhc3NlcywgbWFyayB0aGUgYnVpbGQgYXMgZmFpbHVyZS4KKyAgICAgICAg KiBDSVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzX3VuaXR0ZXN0LnB5OiBBZGRlZC91cGRhdGVkIHVu aXQtdGVzdHMuCisKIDIwMjAtMTEtMDUgIFBhdHJpY2sgQW5nbGUgIDxwYW5nbGVAYXBwbGUuY29t PgogCiAgICAgICAgIENoYW5nZSBteSAoUGF0cmljayBBbmdsZSdzKSBzdGF0dXMgdG8gY29tbWl0 dGVyLgpJbmRleDogVG9vbHMvQ0lTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weQo9PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 Ci0tLSBUb29scy9DSVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5CShyZXZpc2lvbiAyNjk0MzMp CisrKyBUb29scy9DSVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5CSh3b3JraW5nIGNvcHkpCkBA IC0yMDExLDExICsyMDExLDYgQEAgY2xhc3MgUnVuV2ViS2l0VGVzdHMoc2hlbGwuVGVzdCk6CiAg ICAgICAgICAgICAgICAgZWxzZToKICAgICAgICAgICAgICAgICAgICAgcmV0dXJuIEZBSUxVUkUK IAotICAgICAgICAjIFJldHVybiBjb2RlIGZyb20gVG9vbHMvU2NyaXB0cy9sYXlvdXRfdGVzdHMv cnVuX3dlYmtpdF90ZXN0cy5weS4KLSAgICAgICAgIyBUaGlzIG1lYW5zIHRoYXQgYW4gZXhjZXB0 aW9uIHdhcyByYWlzZWQgd2hlbiBydW5uaW5nIHJ1bi13ZWJraXQtdGVzdHMgYW5kCi0gICAgICAg ICMgd2FzIG5ldmVyIGhhbmRsZWQuCi0gICAgICAgIGlmIGNtZC5yYyA9PSAyNTQ6Ci0gICAgICAg ICAgICByZXR1cm4gUkVUUlkKICAgICAgICAgaWYgY21kLnJjICE9IDA6CiAgICAgICAgICAgICBy ZXR1cm4gRkFJTFVSRQogCkBAIC0yMTIzLDYgKzIxMTgsNyBAQCBjbGFzcyBSdW5XZWJLaXRUZXN0 c1dpdGhvdXRQYXRjaChSdW5XZWJLCiAgICAgZGVmIGV2YWx1YXRlQ29tbWFuZChzZWxmLCBjbWQp OgogICAgICAgICByYyA9IHNoZWxsLlRlc3QuZXZhbHVhdGVDb21tYW5kKHNlbGYsIGNtZCkKICAg ICAgICAgc2VsZi5idWlsZC5hZGRTdGVwc0FmdGVyQ3VycmVudFN0ZXAoW0FyY2hpdmVUZXN0UmVz dWx0cygpLCBVcGxvYWRUZXN0UmVzdWx0cyhpZGVudGlmaWVyPSdjbGVhbi10cmVlJyksIEV4dHJh Y3RUZXN0UmVzdWx0cyhpZGVudGlmaWVyPSdjbGVhbi10cmVlJyksIEFuYWx5emVMYXlvdXRUZXN0 c1Jlc3VsdHMoKV0pCisgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ2NsZWFuX3RyZWVfcnVuX3N0 YXR1cycsIHJjKQogICAgICAgICByZXR1cm4gcmMKIAogICAgIGRlZiBjb21tYW5kQ29tcGxldGUo c2VsZiwgY21kKToKQEAgLTIxNDksMTMgKzIxNDUsMTYgQEAgY2xhc3MgQW5hbHl6ZUxheW91dFRl c3RzUmVzdWx0cyhidWlsZHN0ZQogICAgIGRlZiByZXBvcnRfZmFpbHVyZShzZWxmLCBuZXdfZmFp bHVyZXMpOgogICAgICAgICBzZWxmLmZpbmlzaGVkKEZBSUxVUkUpCiAgICAgICAgIHNlbGYuYnVp bGQucmVzdWx0cyA9IEZBSUxVUkUKLSAgICAgICAgcGx1cmFsU3VmZml4ID0gJ3MnIGlmIGxlbihu ZXdfZmFpbHVyZXMpID4gMSBlbHNlICcnCi0gICAgICAgIG5ld19mYWlsdXJlc19zdHJpbmcgPSAn LCAnLmpvaW4oc29ydGVkKG5ld19mYWlsdXJlcylbOnNlbGYuTlVNX0ZBSUxVUkVTX1RPX0RJU1BM QVldKQotICAgICAgICBtZXNzYWdlID0gJ0ZvdW5kIHt9IG5ldyB0ZXN0IGZhaWx1cmV7fToge30n LmZvcm1hdChsZW4obmV3X2ZhaWx1cmVzKSwgcGx1cmFsU3VmZml4LCBuZXdfZmFpbHVyZXNfc3Ry aW5nKQotICAgICAgICBpZiBsZW4obmV3X2ZhaWx1cmVzKSA+IHNlbGYuTlVNX0ZBSUxVUkVTX1RP X0RJU1BMQVk6Ci0gICAgICAgICAgICBtZXNzYWdlICs9ICcgLi4uJworICAgICAgICBpZiBub3Qg bmV3X2ZhaWx1cmVzOgorICAgICAgICAgICAgbWVzc2FnZSA9ICdGb3VuZCB1bmV4cGVjdGVkIGZh aWx1cmUgd2l0aCBwYXRjaCcKKyAgICAgICAgZWxzZToKKyAgICAgICAgICAgIHBsdXJhbFN1ZmZp eCA9ICdzJyBpZiBsZW4obmV3X2ZhaWx1cmVzKSA+IDEgZWxzZSAnJworICAgICAgICAgICAgbmV3 X2ZhaWx1cmVzX3N0cmluZyA9ICcsICcuam9pbihzb3J0ZWQobmV3X2ZhaWx1cmVzKVs6c2VsZi5O VU1fRkFJTFVSRVNfVE9fRElTUExBWV0pCisgICAgICAgICAgICBtZXNzYWdlID0gJ0ZvdW5kIHt9 IG5ldyB0ZXN0IGZhaWx1cmV7fToge30nLmZvcm1hdChsZW4obmV3X2ZhaWx1cmVzKSwgcGx1cmFs U3VmZml4LCBuZXdfZmFpbHVyZXNfc3RyaW5nKQorICAgICAgICAgICAgaWYgbGVuKG5ld19mYWls dXJlcykgPiBzZWxmLk5VTV9GQUlMVVJFU19UT19ESVNQTEFZOgorICAgICAgICAgICAgICAgIG1l c3NhZ2UgKz0gJyAuLi4nCisgICAgICAgICAgICBzZWxmLnNlbmRfZW1haWxfZm9yX25ld190ZXN0 X2ZhaWx1cmVzKG5ld19mYWlsdXJlcykKICAgICAgICAgc2VsZi5kZXNjcmlwdGlvbkRvbmUgPSBt ZXNzYWdlCi0gICAgICAgIHNlbGYuc2VuZF9lbWFpbF9mb3JfbmV3X3Rlc3RfZmFpbHVyZXMobmV3 X2ZhaWx1cmVzKQogICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdidWlsZF9maW5pc2hfc3VtbWFy eScsIG1lc3NhZ2UpCiAKICAgICAgICAgaWYgc2VsZi5nZXRQcm9wZXJ0eSgnYnVpbGRlcm5hbWUn LCAnJykubG93ZXIoKSA9PSAnY29tbWl0LXF1ZXVlJzoKQEAgLTIyODIsNiArMjI4MSw5IEBAIGNs YXNzIEFuYWx5emVMYXlvdXRUZXN0c1Jlc3VsdHMoYnVpbGRzdGUKICAgICAgICAgaWYgKG5vdCBm aXJzdF9yZXN1bHRzX2ZhaWxpbmdfdGVzdHMpIGFuZCAobm90IHNlY29uZF9yZXN1bHRzX2ZhaWxp bmdfdGVzdHMpOgogICAgICAgICAgICAgIyBJZiB3ZSd2ZSBtYWRlIGl0IGhlcmUsIHRoZW4gbGF5 b3V0LXRlc3RzIGFuZCByZS1ydW4tbGF5b3V0LXRlc3RzIGZhaWxlZCwgd2hpY2ggbWVhbnMKICAg ICAgICAgICAgICMgdGhlcmUgc2hvdWxkIGhhdmUgYmVlbiBzb21lIHRlc3QgZmFpbHVyZXMuIE90 aGVyd2lzZSB0aGVyZSBpcyBzb21lIHVuZXhwZWN0ZWQgaXNzdWUuCisgICAgICAgICAgICBjbGVh bl90cmVlX3J1bl9zdGF0dXMgPSBzZWxmLmdldFByb3BlcnR5KCdjbGVhbl90cmVlX3J1bl9zdGF0 dXMnLCBGQUlMVVJFKQorICAgICAgICAgICAgaWYgY2xlYW5fdHJlZV9ydW5fc3RhdHVzID09IFNV Q0NFU1M6CisgICAgICAgICAgICAgICAgcmV0dXJuIHNlbGYucmVwb3J0X2ZhaWx1cmUoc2V0KCkp CiAgICAgICAgICAgICAjIFRPRE86IGVtYWlsIEVXUyBhZG1pbnMKICAgICAgICAgICAgIHJldHVy biBzZWxmLnJldHJ5X2J1aWxkKCdVbmV4cGVjdGVkIGluZnJhc3RydWN0dXJlIGlzc3VlLCByZXRy eWluZyBidWlsZCcpCiAKSW5kZXg6IFRvb2xzL0NJU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHNfdW5p dHRlc3QucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQ0lTdXBwb3J0L2V3cy1idWlsZC9zdGVwc191 bml0dGVzdC5weQkocmV2aXNpb24gMjY5NDMzKQorKysgVG9vbHMvQ0lTdXBwb3J0L2V3cy1idWls ZC9zdGVwc191bml0dGVzdC5weQkod29ya2luZyBjb3B5KQpAQCAtMTc3Niw3ICsxNzc2LDcgQEAg dHMiLCJ2ZXJzaW9uIjo0LCJudW1fcGFzc2VzIjo0MjE1OCwicGl4ZQogICAgICAgICAgICAgKyBF eHBlY3RTaGVsbC5sb2coJ3N0ZGlvJywgc3Rkb3V0PSdVbmV4cGVjdGVkIGVycm9yLicpCiAgICAg ICAgICAgICArIDI1NCwKICAgICAgICAgKQotICAgICAgICBzZWxmLmV4cGVjdE91dGNvbWUocmVz dWx0PVJFVFJZLCBzdGF0ZV9zdHJpbmc9J2xheW91dC10ZXN0cyAocmV0cnkpJykKKyAgICAgICAg c2VsZi5leHBlY3RPdXRjb21lKHJlc3VsdD1GQUlMVVJFLCBzdGF0ZV9zdHJpbmc9J2xheW91dC10 ZXN0cyAoZmFpbHVyZSknKQogICAgICAgICByZXR1cm4gc2VsZi5ydW5TdGVwKCkKIAogICAgIGRl ZiB0ZXN0X2ZhaWx1cmUoc2VsZik6CkBAIC0yMTQ0LDkgKzIxNDQsMTkgQEAgY2xhc3MgVGVzdEFu YWx5emVMYXlvdXRUZXN0c1Jlc3VsdHMoQnVpbAogICAgICAgICBzZWxmLmNvbmZpZ3VyZVN0ZXAo KQogICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdmaXJzdF9ydW5fZmFpbHVyZXMnLCBbXSkKICAg ICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnc2Vjb25kX3J1bl9mYWlsdXJlcycsIFtdKQorICAgICAg ICBzZWxmLnNldFByb3BlcnR5KCdjbGVhbl90cmVlX3J1bl9zdGF0dXMnLCBGQUlMVVJFKQogICAg ICAgICBzZWxmLmV4cGVjdE91dGNvbWUocmVzdWx0PVJFVFJZLCBzdGF0ZV9zdHJpbmc9J1VuZXhw ZWN0ZWQgaW5mcmFzdHJ1Y3R1cmUgaXNzdWUsIHJldHJ5aW5nIGJ1aWxkIChyZXRyeSknKQogICAg ICAgICByZXR1cm4gc2VsZi5ydW5TdGVwKCkKIAorICAgIGRlZiB0ZXN0X3BhdGNoX2JyZWFrc19s YXlvdXRfdGVzdHMoc2VsZik6CisgICAgICAgIHNlbGYuY29uZmlndXJlU3RlcCgpCisgICAgICAg IHNlbGYuc2V0UHJvcGVydHkoJ2ZpcnN0X3J1bl9mYWlsdXJlcycsIFtdKQorICAgICAgICBzZWxm LnNldFByb3BlcnR5KCdzZWNvbmRfcnVuX2ZhaWx1cmVzJywgW10pCisgICAgICAgIHNlbGYuc2V0 UHJvcGVydHkoJ2NsZWFuX3RyZWVfcnVuX2ZhaWx1cmVzJywgW10pCisgICAgICAgIHNlbGYuc2V0 UHJvcGVydHkoJ2NsZWFuX3RyZWVfcnVuX3N0YXR1cycsIFNVQ0NFU1MpCisgICAgICAgIHNlbGYu ZXhwZWN0T3V0Y29tZShyZXN1bHQ9RkFJTFVSRSwgc3RhdGVfc3RyaW5nPSdGb3VuZCB1bmV4cGVj dGVkIGZhaWx1cmUgd2l0aCBwYXRjaCAoZmFpbHVyZSknKQorICAgICAgICByZXR1cm4gc2VsZi5y dW5TdGVwKCkKKwogCiBjbGFzcyBUZXN0Q2hlY2tPdXRTcGVjaWZpY1JldmlzaW9uKEJ1aWxkU3Rl cE1peGluQWRkaXRpb25zLCB1bml0dGVzdC5UZXN0Q2FzZSk6CiAgICAgZGVmIHNldFVwKHNlbGYp Ogo=