210855 2020-04-22 05:27:54 -0700 [ews] Display flaky test names in build summary when ReRunJavaScriptCoreTests passes 2020-04-29 09:30:42 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=207504 InRadar P2 Normal --- 1 aakash_jain aakash_jain ap jbedard webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1644271 0 aakash_jain 2020-04-22 05:27:54 -0700 When JSC tests (RunJavaScriptCoreTests) fail, we re-run them. If the re-run pass, we mark the build as successful. The build summary is set to: 'Passed JSC tests'. It would be better to indicate the flaky test name in the build summary in this case, so that bot-watchers can easily notice flaky tests (and the pattern). 1646939 1 aakash_jain 2020-04-29 03:12:14 -0700 stress/array-buffer-view-watchpoint-can-be-fired-in-really-add-in-dfg.js.default was flaky in https://ews-build.webkit.org/#/builders/26/builds/15609 and https://ews-build.webkit.org/#/builders/26/builds/15541 but it's hard to find it since this flakiness information is not shown in build summary and so it doesn't show up on corresponding builder page (https://ews-build.webkit.org/#/builders/26). Showing this info (as build summary for the build) on builder page would make it easy for bot-watchers or someone to easily notice flakiness info and the pattern. 1646941 2 397951 aakash_jain 2020-04-29 03:18:12 -0700 Created attachment 397951 Patch 1646971 3 397951 jbedard 2020-04-29 07:00:26 -0700 Comment on attachment 397951 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=397951&action=review > Tools/BuildSlaveSupport/ews-build/steps.py:1514 > + pluralSuffix = 's' if len(flaky_failures) > 1 else '' It seems possible that there are no flakey failures, right? Wouldn't we want to keep the 'Passed JSC tests' in that case? 1646972 4 aakash_jain 2020-04-29 07:11:57 -0700 (In reply to Jonathan Bedard from comment #3) > It seems possible that there are no flakey failures, right? This step (ReRunJavaScriptCoreTests) would be run only if there were some failures in first run. Since first run had some failures, and re-run passed, that means the failures were flaky. 1646998 5 ews-feeder 2020-04-29 08:25:12 -0700 Committed r260897: <https://trac.webkit.org/changeset/260897> All reviewed patches have been landed. Closing bug and clearing flags on attachment 397951. 1647000 6 webkit-bug-importer 2020-04-29 08:26:13 -0700 <rdar://problem/62601528> 1647024 7 aakash_jain 2020-04-29 09:30:42 -0700 Restarted EWS server to pick up this change. 397951 2020-04-29 03:18:12 -0700 2020-04-29 08:25:12 -0700 Patch bug-210855-20200429061811.patch text/plain 4702 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI2MDg4OSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE0IEBACisyMDIwLTA0LTI5ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbZXdzXSBEaXNwbGF5IGZsYWt5IHRlc3QgbmFtZXMgaW4gYnVpbGQgc3Vt bWFyeSB3aGVuIFJlUnVuSmF2YVNjcmlwdENvcmVUZXN0cyBwYXNzZXMKKyAgICAgICAgaHR0cHM6 Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTIxMDg1NQorCisgICAgICAgIFJldmll d2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgICogQnVpbGRTbGF2ZVN1cHBvcnQvZXdz LWJ1aWxkL3N0ZXBzLnB5OgorICAgICAgICAoUmVSdW5KYXZhU2NyaXB0Q29yZVRlc3RzLmV2YWx1 YXRlQ29tbWFuZCk6IERpc3BsYXkgZmxha3kgdGVzdCBuYW1lcyBpbiBidWlsZCBzdW1tYXJ5Lgor ICAgICAgICAqIEJ1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwc191bml0dGVzdC5weTog QWRkZWQgdW5pdC10ZXN0cyBhbmQgcmVtb3ZlZCBleHRyYSBUZXN0UmVSdW5KYXZhU2NyaXB0Q29y ZVRlc3RzIGNsYXNzLgorCiAyMDIwLTA0LTI5ICBBZHJpYW4gUGVyZXogZGUgQ2FzdHJvICA8YXBl cmV6QGlnYWxpYS5jb20+CiAKICAgICAgICAgW0dUS10gTWlzcGxhY2VkIHJpZ2h0IGNsaWNrIG1l bnUgb24gd2ViIHBhZ2UgZHVlIHRvIGRlcHJlY2F0ZWQgZ3RrX21lbnVfcG9wdXAoKQpJbmRleDog VG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5Cj09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0t IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weQkocmV2aXNpb24gMjYw ODg5KQorKysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5CSh3b3Jr aW5nIGNvcHkpCkBAIC0xNTA1LDggKzE1MDUsMTQgQEAgY2xhc3MgUmVSdW5KYXZhU2NyaXB0Q29y ZVRlc3RzKFJ1bkphdmFTYwogCiAgICAgZGVmIGV2YWx1YXRlQ29tbWFuZChzZWxmLCBjbWQpOgog ICAgICAgICByYyA9IHNoZWxsLlRlc3QuZXZhbHVhdGVDb21tYW5kKHNlbGYsIGNtZCkKKyAgICAg ICAgZmlyc3RfcnVuX2ZhaWx1cmVzID0gc2V0KHNlbGYuZ2V0UHJvcGVydHkoJ2pzY19zdHJlc3Nf dGVzdF9mYWlsdXJlcycsIFtdKSArIHNlbGYuZ2V0UHJvcGVydHkoJ2pzY19iaW5hcnlfZmFpbHVy ZXMnLCBbXSkpCisgICAgICAgIHNlY29uZF9ydW5fZmFpbHVyZXMgPSBzZXQoc2VsZi5nZXRQcm9w ZXJ0eSgnanNjX3JlcnVuX3N0cmVzc190ZXN0X2ZhaWx1cmVzJywgW10pICsgc2VsZi5nZXRQcm9w ZXJ0eSgnanNjX3JlcnVuX2JpbmFyeV9mYWlsdXJlcycsIFtdKSkKKyAgICAgICAgZmxha3lfZmFp bHVyZXMgPSBmaXJzdF9ydW5fZmFpbHVyZXMudW5pb24oc2Vjb25kX3J1bl9mYWlsdXJlcykgLSBm aXJzdF9ydW5fZmFpbHVyZXMuaW50ZXJzZWN0aW9uKHNlY29uZF9ydW5fZmFpbHVyZXMpCisgICAg ICAgIGZsYWt5X2ZhaWx1cmVzX3N0cmluZyA9ICcsICcuam9pbihmbGFreV9mYWlsdXJlcykKKwog ICAgICAgICBpZiByYyA9PSBTVUNDRVNTIG9yIHJjID09IFdBUk5JTkdTOgotICAgICAgICAgICAg bWVzc2FnZSA9ICdQYXNzZWQgSlNDIHRlc3RzJworICAgICAgICAgICAgcGx1cmFsU3VmZml4ID0g J3MnIGlmIGxlbihmbGFreV9mYWlsdXJlcykgPiAxIGVsc2UgJycKKyAgICAgICAgICAgIG1lc3Nh Z2UgPSAnRm91bmQgZmxha3kgdGVzdHt9OiB7fScuZm9ybWF0KHBsdXJhbFN1ZmZpeCwgZmxha3lf ZmFpbHVyZXNfc3RyaW5nKQogICAgICAgICAgICAgc2VsZi5kZXNjcmlwdGlvbkRvbmUgPSBtZXNz YWdlCiAgICAgICAgICAgICBzZWxmLmJ1aWxkLnJlc3VsdHMgPSBTVUNDRVNTCiAgICAgICAgICAg ICBzZWxmLmJ1aWxkLmJ1aWxkRmluaXNoZWQoW21lc3NhZ2VdLCBTVUNDRVNTKQpJbmRleDogVG9v bHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzX3VuaXR0ZXN0LnB5Cj09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT0KLS0tIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwc191bml0dGVzdC5w eQkocmV2aXNpb24gMjYwODg5KQorKysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxk L3N0ZXBzX3VuaXR0ZXN0LnB5CSh3b3JraW5nIGNvcHkpCkBAIC00MzMsMTEgKzQzMyw2IEBAIEZh aWxlZCAxLzQwIHRlc3QgcHJvZ3JhbXMuIDEwLzYzMCBzdWJ0ZXMKICAgICAgICAgcmV0dXJuIHNl bGYucnVuU3RlcCgpCiAKIAotY2xhc3MgVGVzdFJlUnVuSmF2YVNjcmlwdENvcmVUZXN0cyhUZXN0 UnVuV2ViS2l0UGVybFRlc3RzKToKLSAgICBkZWYgY29uZmlndXJlU3RlcChzZWxmKToKLSAgICAg ICAgc2VsZi5zZXR1cFN0ZXAoUmVSdW5XZWJLaXRQZXJsVGVzdHMoKSkKLQotCiBjbGFzcyBUZXN0 V2ViS2l0UHlQeXRob24yVGVzdHMoQnVpbGRTdGVwTWl4aW5BZGRpdGlvbnMsIHVuaXR0ZXN0LlRl c3RDYXNlKToKICAgICBkZWYgc2V0VXAoc2VsZik6CiAgICAgICAgIHNlbGYubG9uZ01lc3NhZ2Ug PSBUcnVlCkBAIC0xMzQ4LDYgKzEzNDMsMzUgQEAgY2xhc3MgVGVzdFJlUnVuSmF2YVNjcmlwdENv cmVUZXN0cyhUZXN0UgogICAgICAgICBpZiBjb25maWd1cmF0aW9uOgogICAgICAgICAgICAgc2Vs Zi5zZXRQcm9wZXJ0eSgnY29uZmlndXJhdGlvbicsIGNvbmZpZ3VyYXRpb24pCiAKKyAgICBkZWYg dGVzdF9zdWNjZXNzKHNlbGYpOgorICAgICAgICBzZWxmLmNvbmZpZ3VyZVN0ZXAocGxhdGZvcm09 J21hYycsIGZ1bGxQbGF0Zm9ybT0nbWFjLWhpZ2hzaWVycmEnLCBjb25maWd1cmF0aW9uPSdyZWxl YXNlJykKKyAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnanNjX3N0cmVzc190ZXN0X2ZhaWx1cmVz JywgWyd0ZXN0MScsICd0ZXN0MiddKQorICAgICAgICBzZWxmLmV4cGVjdFJlbW90ZUNvbW1hbmRz KAorICAgICAgICAgICAgRXhwZWN0U2hlbGwod29ya2Rpcj0nd2tkaXInLAorICAgICAgICAgICAg ICAgICAgICAgICAgbG9nRW52aXJvbj1GYWxzZSwKKyAgICAgICAgICAgICAgICAgICAgICAgIGNv bW1hbmQ9WydwZXJsJywgJ1Rvb2xzL1NjcmlwdHMvcnVuLWphdmFzY3JpcHRjb3JlLXRlc3RzJywg Jy0tbm8tYnVpbGQnLCAnLS1uby1mYWlsLWZhc3QnLCAnLS1qc29uLW91dHB1dD17MH0nLmZvcm1h dChzZWxmLmpzb25GaWxlTmFtZSksICctLXJlbGVhc2UnXSwKKyAgICAgICAgICAgICAgICAgICAg ICAgIGxvZ2ZpbGVzPXsnanNvbic6IHNlbGYuanNvbkZpbGVOYW1lfSwKKyAgICAgICAgICAgICAg ICAgICAgICAgICkgKworICAgICAgICAgICAgMCwKKyAgICAgICAgKQorICAgICAgICBzZWxmLmV4 cGVjdE91dGNvbWUocmVzdWx0PVNVQ0NFU1MsIHN0YXRlX3N0cmluZz0nRm91bmQgZmxha3kgdGVz dHM6IHRlc3QxLCB0ZXN0MicpCisgICAgICAgIHJldHVybiBzZWxmLnJ1blN0ZXAoKQorCisgICAg ZGVmIHRlc3RfcmVtb3RlX3N1Y2Nlc3Moc2VsZik6CisgICAgICAgIHNlbGYuY29uZmlndXJlU3Rl cChwbGF0Zm9ybT0nanNjLW9ubHknLCBmdWxsUGxhdGZvcm09J2pzYy1vbmx5JywgY29uZmlndXJh dGlvbj0ncmVsZWFzZScpCisgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ3JlbW90ZXMnLCAncmVt b3RlLW1hY2hpbmVzLmpzb24nKQorICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdqc2NfYmluYXJ5 X2ZhaWx1cmVzJywgWyd0ZXN0bWFzbSddKQorICAgICAgICBzZWxmLmV4cGVjdFJlbW90ZUNvbW1h bmRzKAorICAgICAgICAgICAgRXhwZWN0U2hlbGwod29ya2Rpcj0nd2tkaXInLAorICAgICAgICAg ICAgICAgICAgICAgICAgbG9nRW52aXJvbj1GYWxzZSwKKyAgICAgICAgICAgICAgICAgICAgICAg IGNvbW1hbmQ9WydwZXJsJywgJ1Rvb2xzL1NjcmlwdHMvcnVuLWphdmFzY3JpcHRjb3JlLXRlc3Rz JywgJy0tbm8tYnVpbGQnLCAnLS1uby1mYWlsLWZhc3QnLCAnLS1qc29uLW91dHB1dD17MH0nLmZv cm1hdChzZWxmLmpzb25GaWxlTmFtZSksICctLXJlbGVhc2UnLCAnLS1yZW1vdGUtY29uZmlnLWZp bGU9cmVtb3RlLW1hY2hpbmVzLmpzb24nLCAnLS1uby10ZXN0bWFzbScsICctLW5vLXRlc3RhaXIn LCAnLS1uby10ZXN0YjMnLCAnLS1uby10ZXN0ZGZnJywgJy0tbm8tdGVzdGFwaScsICctLW1lbW9y eS1saW1pdGVkJywgJy0tanNjLW9ubHknXSwKKyAgICAgICAgICAgICAgICAgICAgICAgIGxvZ2Zp bGVzPXsnanNvbic6IHNlbGYuanNvbkZpbGVOYW1lfSwKKyAgICAgICAgICAgICAgICAgICAgICAg ICkgKworICAgICAgICAgICAgMCwKKyAgICAgICAgKQorICAgICAgICBzZWxmLmV4cGVjdE91dGNv bWUocmVzdWx0PVNVQ0NFU1MsIHN0YXRlX3N0cmluZz0nRm91bmQgZmxha3kgdGVzdDogdGVzdG1h c20nKQorICAgICAgICByZXR1cm4gc2VsZi5ydW5TdGVwKCkKKwogCiBjbGFzcyBUZXN0UnVuSlND VGVzdHNXaXRob3V0UGF0Y2goQnVpbGRTdGVwTWl4aW5BZGRpdGlvbnMsIHVuaXR0ZXN0LlRlc3RD YXNlKToKICAgICBkZWYgc2V0VXAoc2VsZik6Cg==