215944 2020-08-28 12:41:14 -0700 ews might mark build as successful if tests fail to run 2021-08-30 03:35:26 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=218618 https://bugs.webkit.org/show_bug.cgi?id=219543 https://bugs.webkit.org/show_bug.cgi?id=229634 InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain ap jbedard ryanhaddad webkit-bug-importer oldest_to_newest 1683741 0 aakash_jain 2020-08-28 12:41:14 -0700 EWS might mark build as successful in certain conditions if tests fail to run due to some infrastructure issue (e.g.: test hanging). For e.g. in https://ews-build.webkit.org/#/builders/24/builds/24143 tests were getting stuck (and killed by buildbot after 20 minutes). However the build was marked successful. Note that in case the tests fails with an exit code 254, we already have logic to retry the build. However, in case the tests get stuck, and were killed by buildbot after 20 minutes, our current logic doesn't handle that well. 1685601 1 407956 aakash_jain 2020-09-04 04:28:33 -0700 Created attachment 407956 Patch 1685678 2 407956 jbedard 2020-09-04 08:56:27 -0700 Comment on attachment 407956 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=407956&action=review > Tools/BuildSlaveSupport/ews-build/steps.py:2219 > + return self.retry_build('Unexpected infrastructure issue, retrying build') Won't this put us into an infinite retry loop? I recall there being a bug about getting out of an infinite retry loop by updating the revision we're testing with.....have we landed that yet? 1685694 3 aakash_jain 2020-09-04 09:29:41 -0700 (In reply to Jonathan Bedard from comment #2) > Won't this put us into an infinite retry loop? I recall there being a bug about getting out of an infinite retry loop by updating the revision we're testing with.....have we landed that yet? You are right, depending on what caused the issue, it might create infinite retry loop (e.g.: bad commit causing layout-test to hang would create infinite retry loop, whereas bot in a bad state causing tests to hang would cause retries only until the bot is fixed). Bug 203698 which has the fix to automatically get out of retries isn't landed yet (need some more testing for edge cases). That patch would update retry_build() method, which is used by this patch. So when we land that patch, the fix will be applicable for this case as well. We can wait to land this patch after Bug 203698, or can land this now as well. In theory this patch is still an improvement, since without this, ews might show misleading green status-bubbles, which is probably worst than no results. 1685715 4 jbedard 2020-09-04 09:51:08 -0700 (In reply to Aakash Jain from comment #3) > (In reply to Jonathan Bedard from comment #2) > > Won't this put us into an infinite retry loop? I recall there being a bug about getting out of an infinite retry loop by updating the revision we're testing with.....have we landed that yet? > You are right, depending on what caused the issue, it might create infinite > retry loop (e.g.: bad commit causing layout-test to hang would create > infinite retry loop, whereas bot in a bad state causing tests to hang would > cause retries only until the bot is fixed). > > Bug 203698 which has the fix to automatically get out of retries isn't > landed yet (need some more testing for edge cases). That patch would update > retry_build() method, which is used by this patch. So when we land that > patch, the fix will be applicable for this case as well. > > We can wait to land this patch after Bug 203698, or can land this now as > well. In theory this patch is still an improvement, since without this, ews > might show misleading green status-bubbles, which is probably worst than no > results. How far are we from getting bug 203698 landed? I'm ok landing this one now, but only if we get bug 203698 landed in the next week. It seems like this change will make the case covered by bug 203698 more common. 1685762 5 aakash_jain 2020-09-04 11:08:04 -0700 (In reply to Jonathan Bedard from comment #4) > How far are we from getting bug 203698 landed? I'm ok landing this one now, but only if we get bug 203698 landed in the next week. It seems like this change will make the case covered by bug 203698 more common. I think next week is reasonable for Bug 203698. Can you please review that patch again as well? 1685818 6 webkit-bug-importer 2020-09-04 12:42:14 -0700 <rdar://problem/68360846> 1687127 7 ews-feeder 2020-09-09 14:30:40 -0700 Committed r266799: <https://trac.webkit.org/changeset/266799> All reviewed patches have been landed. Closing bug and clearing flags on attachment 407956. 407956 2020-09-04 04:28:33 -0700 2020-09-09 14:30:41 -0700 Patch bug-215944-20200904072832.patch text/plain 3465 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI2NjU5NikKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE2IEBACisyMDIwLTA5LTA0ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBld3MgbWlnaHQgbWFyayBidWlsZCBhcyBzdWNjZXNzZnVsIGlmIHRlc3Rz IGZhaWwgdG8gcnVuCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNn aT9pZD0yMTU5NDQKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAg ICAgICAqIEJ1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weToKKyAgICAgICAgKEFu YWx5emVMYXlvdXRUZXN0c1Jlc3VsdHMucmV0cnlfYnVpbGQpOgorICAgICAgICAoQW5hbHl6ZUxh eW91dFRlc3RzUmVzdWx0cy5zdGFydCk6CisgICAgICAgICogQnVpbGRTbGF2ZVN1cHBvcnQvZXdz LWJ1aWxkL3N0ZXBzX3VuaXR0ZXN0LnB5OgorICAgICAgICAoVGVzdEFuYWx5emVMYXlvdXRUZXN0 c1Jlc3VsdHMudGVzdF90ZXN0X2ZhaWxfdG9fcnVucyk6CisKIDIwMjAtMDktMDQgIENhcmxvcyBH YXJjaWEgQ2FtcG9zICA8Y2dhcmNpYUBpZ2FsaWEuY29tPgogCiAgICAgICAgIFtHTElCXSBTdG9w IHVzaW5nIGZpcmVmb3ggdXNlciBhZ2VudCBxdWlyayBmb3IgZ29vZ2xlIGRvY3MKSW5kZXg6IFRv b2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weQo9PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBU b29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHMucHkJKHJldmlzaW9uIDI2NjU5 NikKKysrIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weQkod29ya2lu ZyBjb3B5KQpAQCAtMjEzNSw3ICsyMTM1LDggQEAgY2xhc3MgQW5hbHl6ZUxheW91dFRlc3RzUmVz dWx0cyhidWlsZHN0ZQogCiAgICAgZGVmIHJldHJ5X2J1aWxkKHNlbGYsIG1lc3NhZ2U9JycpOgog ICAgICAgICBzZWxmLmZpbmlzaGVkKFJFVFJZKQotICAgICAgICBtZXNzYWdlID0gJ1VuYWJsZSB0 byBjb25maXJtIGlmIHRlc3QgZmFpbHVyZXMgYXJlIGludHJvZHVjZWQgYnkgcGF0Y2gsIHJldHJ5 aW5nIGJ1aWxkJworICAgICAgICBpZiBub3QgbWVzc2FnZToKKyAgICAgICAgICAgIG1lc3NhZ2Ug PSAnVW5hYmxlIHRvIGNvbmZpcm0gaWYgdGVzdCBmYWlsdXJlcyBhcmUgaW50cm9kdWNlZCBieSBw YXRjaCwgcmV0cnlpbmcgYnVpbGQnCiAgICAgICAgIHNlbGYuZGVzY3JpcHRpb25Eb25lID0gbWVz c2FnZQogICAgICAgICBzZWxmLmJ1aWxkLmJ1aWxkRmluaXNoZWQoW21lc3NhZ2VdLCBSRVRSWSkK ICAgICAgICAgcmV0dXJuIGRlZmVyLnN1Y2NlZWQoTm9uZSkKQEAgLTIyMTEsNiArMjIxMiwxMiBA QCBjbGFzcyBBbmFseXplTGF5b3V0VGVzdHNSZXN1bHRzKGJ1aWxkc3RlCiAgICAgICAgIGNsZWFu X3RyZWVfcmVzdWx0c19mYWlsaW5nX3Rlc3RzID0gc2V0KHNlbGYuZ2V0UHJvcGVydHkoJ2NsZWFu X3RyZWVfcnVuX2ZhaWx1cmVzJywgW10pKQogICAgICAgICBmbGFreV9mYWlsdXJlcyA9IGZpcnN0 X3Jlc3VsdHNfZmFpbGluZ190ZXN0cy51bmlvbihzZWNvbmRfcmVzdWx0c19mYWlsaW5nX3Rlc3Rz KSAtIGZpcnN0X3Jlc3VsdHNfZmFpbGluZ190ZXN0cy5pbnRlcnNlY3Rpb24oc2Vjb25kX3Jlc3Vs dHNfZmFpbGluZ190ZXN0cykKIAorICAgICAgICBpZiAobm90IGZpcnN0X3Jlc3VsdHNfZmFpbGlu Z190ZXN0cykgYW5kIChub3Qgc2Vjb25kX3Jlc3VsdHNfZmFpbGluZ190ZXN0cyk6CisgICAgICAg ICAgICAjIElmIHdlJ3ZlIG1hZGUgaXQgaGVyZSwgdGhlbiBsYXlvdXQtdGVzdHMgYW5kIHJlLXJ1 bi1sYXlvdXQtdGVzdHMgZmFpbGVkLCB3aGljaCBtZWFucworICAgICAgICAgICAgIyB0aGVyZSBz aG91bGQgaGF2ZSBiZWVuIHNvbWUgdGVzdCBmYWlsdXJlcy4gT3RoZXJ3aXNlIHRoZXJlIGlzIHNv bWUgdW5leHBlY3RlZCBpc3N1ZS4KKyAgICAgICAgICAgICMgVE9ETzogZW1haWwgRVdTIGFkbWlu cworICAgICAgICAgICAgcmV0dXJuIHNlbGYucmV0cnlfYnVpbGQoJ1VuZXhwZWN0ZWQgaW5mcmFz dHJ1Y3R1cmUgaXNzdWUsIHJldHJ5aW5nIGJ1aWxkJykKKwogICAgICAgICBpZiBmaXJzdF9yZXN1 bHRzX2RpZF9leGNlZWRfdGVzdF9mYWlsdXJlX2xpbWl0IGFuZCBzZWNvbmRfcmVzdWx0c19kaWRf ZXhjZWVkX3Rlc3RfZmFpbHVyZV9saW1pdDoKICAgICAgICAgICAgIGlmIChsZW4oZmlyc3RfcmVz dWx0c19mYWlsaW5nX3Rlc3RzKSAtIGxlbihjbGVhbl90cmVlX3Jlc3VsdHNfZmFpbGluZ190ZXN0 cykpIDw9IDU6CiAgICAgICAgICAgICAgICAgIyBJZiB3ZSd2ZSBtYWRlIGl0IGhlcmUsIHRoZW4g bWFueSB0ZXN0cyBhcmUgZmFpbGluZyB3aXRoIHRoZSBwYXRjaCBhcHBsaWVkLCBidXQKSW5kZXg6 IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwc191bml0dGVzdC5weQo9PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09Ci0tLSBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHNfdW5pdHRl c3QucHkJKHJldmlzaW9uIDI2NjU5NikKKysrIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1i dWlsZC9zdGVwc191bml0dGVzdC5weQkod29ya2luZyBjb3B5KQpAQCAtMjA2OCw2ICsyMDY4LDE0 IEBAIGNsYXNzIFRlc3RBbmFseXplTGF5b3V0VGVzdHNSZXN1bHRzKEJ1aWwKICAgICAgICAgc2Vs Zi5hc3NlcnRFcXVhbChzZWxmLmdldFByb3BlcnR5KCdidWlsZF9maW5pc2hfc3VtbWFyeScpLCBm YWlsdXJlX21lc3NhZ2UpCiAgICAgICAgIHJldHVybiByYwogCisgICAgZGVmIHRlc3RfdW5leHBl Y3RlZF9pbmZyYV9pc3N1ZShzZWxmKToKKyAgICAgICAgc2VsZi5jb25maWd1cmVTdGVwKCkKKyAg ICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnZmlyc3RfcnVuX2ZhaWx1cmVzJywgW10pCisgICAgICAg IHNlbGYuc2V0UHJvcGVydHkoJ3NlY29uZF9ydW5fZmFpbHVyZXMnLCBbXSkKKyAgICAgICAgc2Vs Zi5leHBlY3RPdXRjb21lKHJlc3VsdD1SRVRSWSwgc3RhdGVfc3RyaW5nPSdVbmV4cGVjdGVkIGlu ZnJhc3RydWN0dXJlIGlzc3VlLCByZXRyeWluZyBidWlsZCAocmV0cnkpJykKKyAgICAgICAgcmV0 dXJuIHNlbGYucnVuU3RlcCgpCisKKwogY2xhc3MgVGVzdENoZWNrT3V0U3BlY2lmaWNSZXZpc2lv bihCdWlsZFN0ZXBNaXhpbkFkZGl0aW9ucywgdW5pdHRlc3QuVGVzdENhc2UpOgogICAgIGRlZiBz ZXRVcChzZWxmKToKICAgICAgICAgc2VsZi5sb25nTWVzc2FnZSA9IFRydWUK