208003 2020-02-20 07:02:50 -0800 EWS fails to retry build in case of kill-old-processes exception 2020-02-28 13:08:14 -0800 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=203858 InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain ap commit-queue jbedard webkit-bug-importer oldest_to_newest 1621192 0 aakash_jain 2020-02-20 07:02:50 -0800 EWS retries the build in case of kill-old-processes failure (since https://trac.webkit.org/changeset/252324). However, EWS fails to retry build in case of kill-old-processes Exception. Exception in case of kill-old-processes is noticeable only on WinCairo queue. e.g.: https://ews-build.webkit.org/#/builders/12/builds/17905 https://ews-build.webkit.org/#/builders/12/builds/17860 https://ews-build.webkit.org/#/builders/12/builds/17858 It would be nice if EWS can retry the build in case of kill-old-processes exception as well. 1621193 1 391283 aakash_jain 2020-02-20 07:04:27 -0800 Created attachment 391283 Patch 1621194 2 aakash_jain 2020-02-20 07:05:02 -0800 (In reply to Aakash Jain from comment #1) > Created attachment 391283 [details] > Patch Speculative fix. 1621217 3 jbedard 2020-02-20 07:46:53 -0800 (In reply to Aakash Jain from comment #0) > EWS retries the build in case of kill-old-processes failure (since > https://trac.webkit.org/changeset/252324). However, EWS fails to retry build > in case of kill-old-processes Exception. Exception in case of > kill-old-processes is noticeable only on WinCairo queue. > > e.g.: > https://ews-build.webkit.org/#/builders/12/builds/17905 > https://ews-build.webkit.org/#/builders/12/builds/17860 > https://ews-build.webkit.org/#/builders/12/builds/17858 > > It would be nice if EWS can retry the build in case of kill-old-processes > exception as well. What even causes a kill-old-processes exception? If that script is having issues, I would bet that the entire machine is in trouble. 1621223 4 aakash_jain 2020-02-20 08:06:43 -0800 (In reply to Jonathan Bedard from comment #3) > What even causes a kill-old-processes exception? Probably the machine is in a very bad state. > If that script is having issues, I would bet that the entire machine is in trouble. Yes. The idea is to retry the build so that the patch gets re-tested on some other machine. If we don't retry the status-bubble for that patch will remain purple, until someone manually retries the build. Even though that particular machine will keep failing at kill-old-processes, the patch being tested will eventually be tested on some other machine because of this RETRY. Hopefully someone would notice that the machine is in bad state and fix it eventually. For example: https://ews-build.webkit.org/#/builders/22/builds/10002 failed with kill-old-processed, however the build was retried and picked up by another bot (in https://ews-build.webkit.org/#/builders/22/builds/10034) and passed. The status-bubble for corresponding patch 390315 (in Bug 207516) became green without any manual intervention (of bot-watcher or patch author). Similar thing in https://ews-build.webkit.org/#/builders/3/builds/16167 and https://ews-build.webkit.org/#/builders/3/builds/16168 Retry made the system work fine even though one bot was in bad state. 1621306 5 391283 jbedard 2020-02-20 11:03:46 -0800 Comment on attachment 391283 Patch Do we actually have a practical example of an exception? 1621310 6 aakash_jain 2020-02-20 11:13:38 -0800 > Do we actually have a practical example of an exception? https://ews-build.webkit.org/#/builders/12/builds/17858 Although, this is speculative fix. I am not sure if it would really work in case of exception. It's hard to reproduce and I don't have machine to test it out. 1621378 7 jbedard 2020-02-20 13:05:46 -0800 (In reply to Aakash Jain from comment #6) > > Do we actually have a practical example of an exception? > https://ews-build.webkit.org/#/builders/12/builds/17858 > > Although, this is speculative fix. I am not sure if it would really work in > case of exception. It's hard to reproduce and I don't have machine to test > it out. Fair enough. As I said in the other bug, if a machine fails kill-old-processes, it probably needs to be rebooted. 1624085 8 391283 commit-queue 2020-02-28 13:07:32 -0800 Comment on attachment 391283 Patch Clearing flags on attachment: 391283 Committed r257654: <https://trac.webkit.org/changeset/257654> 1624086 9 commit-queue 2020-02-28 13:07:33 -0800 All reviewed patches have been landed. Closing bug. 1624087 10 webkit-bug-importer 2020-02-28 13:08:14 -0800 <rdar://problem/59899567> 391283 2020-02-20 07:04:27 -0800 2020-02-28 13:07:32 -0800 Patch bug-208003-20200220100426.patch text/plain 1543 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI1NzA1NSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE0IEBACisyMDIwLTAyLTIwICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBFV1MgZmFpbHMgdG8gcmV0cnkgYnVpbGQgaW4gY2FzZSBvZiBraWxsLW9s ZC1wcm9jZXNzZXMgZXhjZXB0aW9uCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3No b3dfYnVnLmNnaT9pZD0yMDgwMDMKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMh KS4KKworICAgICAgICAqIEJ1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weToKKyAg ICAgICAgKEtpbGxPbGRQcm9jZXNzZXMuZXZhbHVhdGVDb21tYW5kKToKKyAgICAgICAgKEtpbGxP bGRQcm9jZXNzZXMuZ2V0UmVzdWx0U3VtbWFyeSk6CisKIDIwMjAtMDItMjAgIERvbiBPbG1zdGVh ZCAgPGRvbi5vbG1zdGVhZEBzb255LmNvbT4KIAogICAgICAgICBbQ01ha2VdIEFkZCBXUEU6Omxp YndwZSB0YXJnZXQKSW5kZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVw cy5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09Ci0tLSBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3Rl cHMucHkJKHJldmlzaW9uIDI1NzA1NSkKKysrIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1i dWlsZC9zdGVwcy5weQkod29ya2luZyBjb3B5KQpAQCAtMTM3MSwxMiArMTM3MSwxMiBAQCBjbGFz cyBLaWxsT2xkUHJvY2Vzc2VzKHNoZWxsLkNvbXBpbGUpOgogICAgICAgICBzdXBlcihLaWxsT2xk UHJvY2Vzc2VzLCBzZWxmKS5fX2luaXRfXyh0aW1lb3V0PTYwLCBsb2dFbnZpcm9uPUZhbHNlLCAq Kmt3YXJncykKIAogICAgIGRlZiBldmFsdWF0ZUNvbW1hbmQoc2VsZiwgY21kKToKLSAgICAgICAg aWYgY21kLmRpZEZhaWwoKToKKyAgICAgICAgaWYgc2VsZi5yZXN1bHRzIGluIFtGQUlMVVJFLCBF WENFUFRJT05dOgogICAgICAgICAgICAgc2VsZi5idWlsZC5idWlsZEZpbmlzaGVkKFsnRmFpbGVk IHRvIGtpbGwgb2xkIHByb2Nlc3NlcywgcmV0cnlpbmcgYnVpbGQnXSwgUkVUUlkpCiAgICAgICAg IHJldHVybiBzaGVsbC5Db21waWxlLmV2YWx1YXRlQ29tbWFuZChzZWxmLCBjbWQpCiAKICAgICBk ZWYgZ2V0UmVzdWx0U3VtbWFyeShzZWxmKToKLSAgICAgICAgaWYgc2VsZi5yZXN1bHRzID09IEZB SUxVUkU6CisgICAgICAgIGlmIHNlbGYucmVzdWx0cyBpbiBbRkFJTFVSRSwgRVhDRVBUSU9OXToK ICAgICAgICAgICAgIHJldHVybiB7dSdzdGVwJzogdSdGYWlsZWQgdG8ga2lsbCBvbGQgcHJvY2Vz c2VzJ30KICAgICAgICAgcmV0dXJuIHNoZWxsLkNvbXBpbGUuZ2V0UmVzdWx0U3VtbWFyeShzZWxm KQogCg==