177723 2017-10-01 12:44:42 -0700 EWS bot should fail if a new test is missing its result 2020-10-15 09:43:28 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 sam aakash_jain aakash_jain ap jbedard lforschler megan_gardner ryanhaddad simon.fraser webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1355037 0 sam 2017-10-01 12:44:42 -0700 I found it quite surprising that the EWS bot didn't fail when I updated a change with test missing. I would expect that to also be a failure condition. 1355251 1 webkit-bug-importer 2017-10-02 09:31:40 -0700 <rdar://problem/34768404> 1391774 2 ap 2018-01-22 13:25:09 -0800 *** Bug 181940 has been marked as a duplicate of this bug. *** 1698086 3 aakash_jain 2020-10-15 06:37:31 -0700 Noticed this in: https://ews-build.webkit.org/#/builders/24/builds/28039. Patch 411263 added two tests with missing results. layout-tests step indicated '2 missing results' but did not mark the step/build as failed. The fix is to simply mark the build step as failed, that would cause layout-tests to be retried, and then retry on clean build, and since clean build would pass, it would cause the build to be marked as failed. 1698088 4 411436 aakash_jain 2020-10-15 06:49:10 -0700 Created attachment 411436 Patch 1698089 5 411436 aakash_jain 2020-10-15 06:50:04 -0700 Comment on attachment 411436 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=411436&action=review > Tools/BuildSlaveSupport/ews-build/steps.py:2000 > + elif line.find('missing results') >= 0: Added this elif explicitly for readability. Can remove this as well if reviewers feel strongly about it. 1698122 6 411436 jbedard 2020-10-15 08:52:59 -0700 Comment on attachment 411436 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=411436&action=review >> Tools/BuildSlaveSupport/ews-build/steps.py:2000 >> + elif line.find('missing results') >= 0: > > Added this elif explicitly for readability. Can remove this as well if reviewers feel strongly about it. I'm fine with the elif, but it seems like the failure condition should be first. (since in the event a line contained both 'flakes' and 'missing results', we would want to mark the build as failed). I know this concern is more about purity than practicality at the moment, though. 1698126 7 aakash_jain 2020-10-15 09:15:57 -0700 (In reply to Jonathan Bedard from comment #6) > since in the event a line contained both 'flakes' and 'missing results' 'flakes' and 'missing results' words aren't being checked literally, they store output of regular expressions matching from https://trac.webkit.org/browser/webkit/trunk/Tools/BuildSlaveSupport/ews-build/steps.py#L1949 1698131 8 ews-feeder 2020-10-15 09:27:53 -0700 Committed r268526: <https://trac.webkit.org/changeset/268526> All reviewed patches have been landed. Closing bug and clearing flags on attachment 411436. 1698136 9 aakash_jain 2020-10-15 09:43:28 -0700 Restarted EWS server to pick up this change. 411436 2020-10-15 06:49:10 -0700 2020-10-15 09:27:53 -0700 Patch bug-177723-20201015094909.patch text/plain 4081 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI2ODUyMSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE0IEBACisyMDIwLTEwLTE1ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBFV1MgYm90IHNob3VsZCBmYWlsIGlmIGEgbmV3IHRlc3QgaXMgbWlzc2lu ZyBpdHMgcmVzdWx0CisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNn aT9pZD0xNzc3MjMKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAg ICAgICAqIEJ1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weToKKyAgICAgICAgKFJ1 bldlYktpdFRlc3RzLmV2YWx1YXRlUmVzdWx0KToKKyAgICAgICAgKiBCdWlsZFNsYXZlU3VwcG9y dC9ld3MtYnVpbGQvc3RlcHNfdW5pdHRlc3QucHk6IEFkZGVkIHVuaXQtdGVzdC4KKwogMjAyMC0x MC0xNSAgTGF1cm8gTW91cmEgIDxsbW91cmFAaWdhbGlhLmNvbT4KIAogICAgICAgICB3ZWJraXRw eTogRml4IHJlbWFpbmluZyBweXRlc3QgYXV0b2luc3RhbGwgaW1wb3J0cwpJbmRleDogVG9vbHMv QnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5Cj09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIFRvb2xz L0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weQkocmV2aXNpb24gMjY4NTIxKQor KysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5CSh3b3JraW5nIGNv cHkpCkBAIC0xOTk1LDggKzE5OTUsMTAgQEAgY2xhc3MgUnVuV2ViS2l0VGVzdHMoc2hlbGwuVGVz dCk6CiAgICAgICAgICAgICAgICAgICAgIHJldHVybiBXQVJOSU5HUwogCiAgICAgICAgICAgICBm b3IgbGluZSBpbiBzZWxmLmluY29ycmVjdExheW91dExpbmVzOgotICAgICAgICAgICAgICAgIGlm IGxpbmUuZmluZCgnZmxha2VzJykgPj0gMCBvciBsaW5lLmZpbmQoJ25ldyBwYXNzZXMnKSA+PSAw IG9yIGxpbmUuZmluZCgnbWlzc2luZyByZXN1bHRzJykgPj0gMDoKKyAgICAgICAgICAgICAgICBp ZiBsaW5lLmZpbmQoJ2ZsYWtlcycpID49IDAgb3IgbGluZS5maW5kKCduZXcgcGFzc2VzJykgPj0g MDoKICAgICAgICAgICAgICAgICAgICAgcmVzdWx0ID0gV0FSTklOR1MKKyAgICAgICAgICAgICAg ICBlbGlmIGxpbmUuZmluZCgnbWlzc2luZyByZXN1bHRzJykgPj0gMDoKKyAgICAgICAgICAgICAg ICAgICAgcmV0dXJuIEZBSUxVUkUKICAgICAgICAgICAgICAgICBlbHNlOgogICAgICAgICAgICAg ICAgICAgICByZXR1cm4gRkFJTFVSRQogCkluZGV4OiBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9l d3MtYnVpbGQvc3RlcHNfdW5pdHRlc3QucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQnVpbGRTbGF2 ZVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzX3VuaXR0ZXN0LnB5CShyZXZpc2lvbiAyNjg1MjEpCisr KyBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHNfdW5pdHRlc3QucHkJKHdv cmtpbmcgY29weSkKQEAgLTE1OTEsNiArMTU5MSw5IEBAIGl0aW9uLXRvLWZyb20tYXV0by5odG1s Ijp7InJlcG9ydCI6IkZMQUsKIHRzIiwidmVyc2lvbiI6NCwibnVtX3Bhc3NlcyI6NDIxNTgsInBp eGVsX3Rlc3RzX2VuYWJsZWQiOmZhbHNlLCJkYXRlIjoiMTE6MjhBTSBvbiBKdWx5IDE2LCAyMDE5 IiwiaGFzX3ByZXR0eV9wYXRjaCI6dHJ1ZSwiZml4YWJsZSI6NTUzMjksIm51bV9mbGFreSI6NSwi dXNlc19leHBlY3RhdGlvbnNfZmlsZSI6dHJ1ZX0pOwogICAgICAgICAnJycKIAorICAgICAgICBz ZWxmLnJlc3VsdHNfd2l0aF9taXNzaW5nX3Jlc3VsdHMgPSAnJydBRERfUkVTVUxUUyh7InRlc3Rz Ijp7Imh0dHAiOnsid3B0Ijp7ImNzcyI6eyJjc3MtaGlnaGxpZ2h0LWFwaSI6eyJoaWdobGlnaHQt aW1hZ2UtZXhwZWN0ZWQtbWlzbWF0Y2hlZC5odG1sIjp7InJlcG9ydCI6Ik1JU1NJTkciLCJleHBl Y3RlZCI6IlBBU1MiLCJpc19taXNzaW5nX3RleHQiOnRydWUsImFjdHVhbCI6Ik1JU1NJTkcifSwi aGlnaGxpZ2h0LWltYWdlLmh0bWwiOnsicmVwb3J0IjoiTUlTU0lORyIsImV4cGVjdGVkIjoiUEFT UyIsImlzX21pc3NpbmdfdGV4dCI6dHJ1ZSwiYWN0dWFsIjoiTUlTU0lORyJ9fX19fX0sICJpbnRl cnJ1cHRlZCI6ZmFsc2V9KTsKKyAgICAgICAgJycnCisKICAgICAgICAgcmV0dXJuIHNlbGYuc2V0 VXBCdWlsZFN0ZXAoKQogCiAgICAgZGVmIHRlYXJEb3duKHNlbGYpOgpAQCAtMTczOSw2ICsxNzQy LDI3IEBAIHRzIiwidmVyc2lvbiI6NCwibnVtX3Bhc3NlcyI6NDIxNTgsInBpeGUKICAgICAgICAg c2VsZi5hc3NlcnRFcXVhbChzZWxmLmdldFByb3BlcnR5KHNlbGYucHJvcGVydHlfZmFpbHVyZXMp LCBbJ2Zhc3Qvc2Nyb2xsaW5nL2lvcy9yZWNvbmNpbGUtbGF5ZXItcG9zaXRpb24tcmVjdXJzaXZl Lmh0bWwnXSkKICAgICAgICAgcmV0dXJuIHJjCiAKKyAgICBkZWYgdGVzdF9wYXJzZV9yZXN1bHRz X2pzb25fd2l0aF9taXNzaW5nX3Jlc3VsdHMoc2VsZik6CisgICAgICAgIHNlbGYuY29uZmlndXJl U3RlcCgpCisgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ2Z1bGxQbGF0Zm9ybScsICdpb3Mtc2lt dWxhdG9yJykKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnY29uZmlndXJhdGlvbicsICdyZWxl YXNlJykKKyAgICAgICAgc2VsZi5leHBlY3RSZW1vdGVDb21tYW5kcygKKyAgICAgICAgICAgIEV4 cGVjdFNoZWxsKHdvcmtkaXI9J3drZGlyJywKKyAgICAgICAgICAgICAgICAgICAgICAgIGxvZ2Zp bGVzPXsnanNvbic6IHNlbGYuanNvbkZpbGVOYW1lfSwKKyAgICAgICAgICAgICAgICAgICAgICAg IGxvZ0Vudmlyb249RmFsc2UsCisgICAgICAgICAgICAgICAgICAgICAgICBjb21tYW5kPVsncHl0 aG9uJywgJ1Rvb2xzL1NjcmlwdHMvcnVuLXdlYmtpdC10ZXN0cycsICctLW5vLWJ1aWxkJywgJy0t bm8tc2hvdy1yZXN1bHRzJywgJy0tbm8tbmV3LXRlc3QtcmVzdWx0cycsICctLWNsb2JiZXItb2xk LXJlc3VsdHMnLCAnLS1yZWxlYXNlJywgJy0tcmVzdWx0cy1kaXJlY3RvcnknLCAnbGF5b3V0LXRl c3QtcmVzdWx0cycsICctLWRlYnVnLXJ3dC1sb2dnaW5nJywgJy0tZXhpdC1hZnRlci1uLWZhaWx1 cmVzJywgJzMwJywgJy0tc2tpcC1mYWlsaW5nLXRlc3RzJ10sCisgICAgICAgICAgICAgICAgICAg ICAgICApCisgICAgICAgICAgICArIDIKKyAgICAgICAgICAgICsgRXhwZWN0U2hlbGwubG9nKCdq c29uJywgc3Rkb3V0PXNlbGYucmVzdWx0c193aXRoX21pc3NpbmdfcmVzdWx0cyksCisgICAgICAg ICkKKyAgICAgICAgc2VsZi5leHBlY3RPdXRjb21lKHJlc3VsdD1GQUlMVVJFLCBzdGF0ZV9zdHJp bmc9J2xheW91dC10ZXN0cyAoZmFpbHVyZSknKQorICAgICAgICByYyA9IHNlbGYucnVuU3RlcCgp CisgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwoc2VsZi5nZXRQcm9wZXJ0eShzZWxmLnByb3BlcnR5 X2V4Y2VlZF9mYWlsdXJlX2xpbWl0KSwgRmFsc2UpCisgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwo c2VsZi5nZXRQcm9wZXJ0eShzZWxmLnByb3BlcnR5X2ZhaWx1cmVzKSwKKyAgICAgICAgICAgICAg ICAgICAgICAgICBbJ2h0dHAvd3B0L2Nzcy9jc3MtaGlnaGxpZ2h0LWFwaS9oaWdobGlnaHQtaW1h Z2UtZXhwZWN0ZWQtbWlzbWF0Y2hlZC5odG1sJywKKyAgICAgICAgICAgICAgICAgICAgICAgICAg J2h0dHAvd3B0L2Nzcy9jc3MtaGlnaGxpZ2h0LWFwaS9oaWdobGlnaHQtaW1hZ2UuaHRtbCddKQor ICAgICAgICByZXR1cm4gcmMKKwogICAgIGRlZiB0ZXN0X3VuZXhwZWN0ZWRfZXJyb3Ioc2VsZik6 CiAgICAgICAgIHNlbGYuY29uZmlndXJlU3RlcCgpCiAgICAgICAgIHNlbGYuc2V0UHJvcGVydHko J2Z1bGxQbGF0Zm9ybScsICdtYWMtaGlnaHNpZXJyYScpCg==