199525 2019-07-05 12:15:06 -0700 [ews-build] Display pre-existing API test names in the build summary 2019-10-31 12:57:35 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=199941 InRadar P2 Normal --- 1 aakash_jain aakash_jain aakash_jain ap commit-queue jbedard ryanhaddad webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1550466 0 aakash_jain 2019-07-05 12:15:06 -0700 Currently when there is a failure on clean tree (ToT), API test build summary shows: "Found x pre-existing API test failure". This is fine, but not very informative. It would be better to include the names of the pre-existing failures in the build summary string. Those failing test names would show up on the builder page, and it would be very convenient to notice the test failure pattern over multiple builds. 1550467 1 373522 aakash_jain 2019-07-05 12:17:04 -0700 Created attachment 373522 Patch 1550469 2 aakash_jain 2019-07-05 12:21:16 -0700 Sample runs: https://ews-build.webkit.org/#/builders/9/builds/4064 https://ews-build.webkit.org/#/builders/3/builds/4179 Also notice the build statuses on builder page: https://ews-build.webkit.org/#/builders/3 1550821 3 373522 jbedard 2019-07-08 09:02:00 -0700 Comment on attachment 373522 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=373522&action=review > Tools/BuildSlaveSupport/ews-build/steps.py:1088 > + clean_tree_failures_string = ', '.join([failure_name.replace('TestWebKitAPI.', '') for failure_name in clean_tree_failures]) What we call 'API' tests is actually 2 different binaries on Mac, 3 for Windows (at the moment, at least). That first key in the test name is the binary. It could be TestWebKitAPI, TestWTF, TestWTF.exe, TestWebCore.exe or TestWebKitLegacy.exe. I'm not sure it makes sense to strip the binary name at all, run-webkit-tests is smart enough to understand it. If we do want to string the binary name, though, we need to do it with split('.'). 1585995 4 382483 aakash_jain 2019-10-31 11:39:33 -0700 Created attachment 382483 Patch 1585996 5 aakash_jain 2019-10-31 11:42:31 -0700 Not removing 'TestWebKitAPI.' string anymore. Also updated the patch to also limit the number of failures to display to 10 (although it should be rare to have more than 10 failures on trunk). This change would not affect the status-bubble tooltip. Sample runs: https://ews-build.webkit-uat.org/#/builders/4/builds/67 https://ews-build.webkit-uat.org/#/builders/4/builds/66 1586025 6 382483 commit-queue 2019-10-31 12:56:40 -0700 Comment on attachment 382483 Patch Clearing flags on attachment: 382483 Committed r251869: <https://trac.webkit.org/changeset/251869> 1586026 7 commit-queue 2019-10-31 12:56:42 -0700 All reviewed patches have been landed. Closing bug. 1586027 8 webkit-bug-importer 2019-10-31 12:57:35 -0700 <rdar://problem/56791844> 373522 2019-07-05 12:17:04 -0700 2019-10-31 11:39:31 -0700 Patch bug-199525-20190705151703.patch text/plain 2187 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI0NzE2NykKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDEzIEBACisyMDE5LTA3LTA1ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbZXdzLWJ1aWxkXSBEaXNwbGF5IHByZS1leGlzdGluZyBBUEkgdGVzdCBu YW1lcyBpbiB0aGUgYnVpbGQgc3VtbWFyeQorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9y Zy9zaG93X2J1Zy5jZ2k/aWQ9MTk5NTI1CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChP T1BTISkuCisKKyAgICAgICAgKiBCdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHMucHk6 CisgICAgICAgIChBbmFseXplQVBJVGVzdHNSZXN1bHRzLmFuYWx5emVSZXN1bHRzKTogSW5jbHVk ZSBjbGVhbl90cmVlX2ZhaWx1cmVzX3N0cmluZyBpbiB0aGUgYnVpbGQgcmVzdWx0IHN1bW1hcnku CisKIDIwMTktMDctMDUgIFJ5YW4gSGFkZGFkICA8cnlhbmhhZGRhZEBhcHBsZS5jb20+CiAKICAg ICAgICAgVW5yZXZpZXdlZCwgcm9sbGluZyBvdXQgcjI0NzExNS4KSW5kZXg6IFRvb2xzL0J1aWxk U2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weQo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9CdWls ZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHMucHkJKHJldmlzaW9uIDI0NzE2NykKKysrIFRv b2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weQkod29ya2luZyBjb3B5KQpA QCAtMTA4NSw2ICsxMDg1LDcgQEAgY2xhc3MgQW5hbHl6ZUFQSVRlc3RzUmVzdWx0cyhidWlsZHN0 ZXAuQgogICAgICAgICBmaXJzdF9ydW5fZmFpbHVyZXMgPSBnZXRBUElUZXN0RmFpbHVyZXMoZmly c3RfcnVuX3Jlc3VsdHMpCiAgICAgICAgIHNlY29uZF9ydW5fZmFpbHVyZXMgPSBnZXRBUElUZXN0 RmFpbHVyZXMoc2Vjb25kX3J1bl9yZXN1bHRzKQogICAgICAgICBjbGVhbl90cmVlX2ZhaWx1cmVz ID0gZ2V0QVBJVGVzdEZhaWx1cmVzKGNsZWFuX3RyZWVfcmVzdWx0cykKKyAgICAgICAgY2xlYW5f dHJlZV9mYWlsdXJlc19zdHJpbmcgPSAnLCAnLmpvaW4oW2ZhaWx1cmVfbmFtZS5yZXBsYWNlKCdU ZXN0V2ViS2l0QVBJLicsICcnKSBmb3IgZmFpbHVyZV9uYW1lIGluIGNsZWFuX3RyZWVfZmFpbHVy ZXNdKQogCiAgICAgICAgIGZhaWx1cmVzX3dpdGhfcGF0Y2ggPSBmaXJzdF9ydW5fZmFpbHVyZXMu aW50ZXJzZWN0aW9uKHNlY29uZF9ydW5fZmFpbHVyZXMpCiAgICAgICAgIGZsYWt5X2ZhaWx1cmVz ID0gZmlyc3RfcnVuX2ZhaWx1cmVzLnVuaW9uKHNlY29uZF9ydW5fZmFpbHVyZXMpIC0gZmlyc3Rf cnVuX2ZhaWx1cmVzLmludGVyc2VjdGlvbihzZWNvbmRfcnVuX2ZhaWx1cmVzKQpAQCAtMTExMSw3 ICsxMTEyLDcgQEAgY2xhc3MgQW5hbHl6ZUFQSVRlc3RzUmVzdWx0cyhidWlsZHN0ZXAuQgogICAg ICAgICAgICAgc2VsZi5idWlsZC5yZXN1bHRzID0gU1VDQ0VTUwogICAgICAgICAgICAgc2VsZi5k ZXNjcmlwdGlvbkRvbmUgPSAnUGFzc2VkIEFQSSB0ZXN0cycKICAgICAgICAgICAgIHBsdXJhbFN1 ZmZpeCA9ICdzJyBpZiBsZW4oY2xlYW5fdHJlZV9mYWlsdXJlcykgPiAxIGVsc2UgJycKLSAgICAg ICAgICAgIG1lc3NhZ2UgPSAnRm91bmQge30gcHJlLWV4aXN0aW5nIEFQSSB0ZXN0IGZhaWx1cmV7 fScuZm9ybWF0KGxlbihjbGVhbl90cmVlX2ZhaWx1cmVzKSwgcGx1cmFsU3VmZml4KQorICAgICAg ICAgICAgbWVzc2FnZSA9ICdGb3VuZCB7fSBwcmUtZXhpc3RpbmcgQVBJIHRlc3QgZmFpbHVyZXt9 OiB7fScuZm9ybWF0KGxlbihjbGVhbl90cmVlX2ZhaWx1cmVzKSwgcGx1cmFsU3VmZml4LCBjbGVh bl90cmVlX2ZhaWx1cmVzX3N0cmluZykKICAgICAgICAgICAgIGlmIGZsYWt5X2ZhaWx1cmVzOgog ICAgICAgICAgICAgICAgIG1lc3NhZ2UgKz0gJy4gRmxha3kgdGVzdHM6IHt9Jy5mb3JtYXQoZmxh a3lfZmFpbHVyZXNfc3RyaW5nKQogICAgICAgICAgICAgc2VsZi5idWlsZC5idWlsZEZpbmlzaGVk KFttZXNzYWdlXSwgU1VDQ0VTUykK 382483 2019-10-31 11:39:33 -0700 2019-10-31 12:56:40 -0700 Patch bug-199525-20191031143932.patch text/plain 2441 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI1MTg2MykKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE0IEBACisyMDE5LTEwLTMxICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbZXdzLWJ1aWxkXSBEaXNwbGF5IHByZS1leGlzdGluZyBBUEkgdGVzdCBu YW1lcyBpbiB0aGUgYnVpbGQgc3VtbWFyeQorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9y Zy9zaG93X2J1Zy5jZ2k/aWQ9MTk5NTI1CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChP T1BTISkuCisKKyAgICAgICAgKiBCdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHMucHk6 CisgICAgICAgIChBbmFseXplQVBJVGVzdHNSZXN1bHRzLmFuYWx5emVSZXN1bHRzKTogSW5jbHVk ZSB0aGUgbmFtZXMgb2YgcHJlLWV4aXN0aW5nIHRlc3QgZmFpbHVyZXMgaW4gc3VtbWFyeSBzdHJp bmcgYW5kIAorICAgICAgICBsaW1pdCB0aGUgbnVtYmVyIG9mIGZhaWx1cmVzIHRvIGRpc3BsYXkg dG8gTlVNX0FQSV9GQUlMVVJFU19UT19ESVNQTEFZLgorCiAyMDE5LTEwLTMxICBBbGV4IENocmlz dGVuc2VuICA8YWNocmlzdGVuc2VuQHdlYmtpdC5vcmc+CiAKICAgICAgICAgRXhwb3NlIG1vcmUg V0tQcmVmZXJlbmNlcyBTUEkKSW5kZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWls ZC9zdGVwcy5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVp bGQvc3RlcHMucHkJKHJldmlzaW9uIDI1MTg2MykKKysrIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0 L2V3cy1idWlsZC9zdGVwcy5weQkod29ya2luZyBjb3B5KQpAQCAtMTQ4OCw2ICsxNDg4LDggQEAg Y2xhc3MgQW5hbHl6ZUFQSVRlc3RzUmVzdWx0cyhidWlsZHN0ZXAuQgogICAgICAgICBmaXJzdF9y dW5fZmFpbHVyZXMgPSBnZXRBUElUZXN0RmFpbHVyZXMoZmlyc3RfcnVuX3Jlc3VsdHMpCiAgICAg ICAgIHNlY29uZF9ydW5fZmFpbHVyZXMgPSBnZXRBUElUZXN0RmFpbHVyZXMoc2Vjb25kX3J1bl9y ZXN1bHRzKQogICAgICAgICBjbGVhbl90cmVlX2ZhaWx1cmVzID0gZ2V0QVBJVGVzdEZhaWx1cmVz KGNsZWFuX3RyZWVfcmVzdWx0cykKKyAgICAgICAgY2xlYW5fdHJlZV9mYWlsdXJlc190b19kaXNw bGF5ID0gbGlzdChjbGVhbl90cmVlX2ZhaWx1cmVzKVs6c2VsZi5OVU1fQVBJX0ZBSUxVUkVTX1RP X0RJU1BMQVldCisgICAgICAgIGNsZWFuX3RyZWVfZmFpbHVyZXNfc3RyaW5nID0gJywgJy5qb2lu KGNsZWFuX3RyZWVfZmFpbHVyZXNfdG9fZGlzcGxheSkKIAogICAgICAgICBmYWlsdXJlc193aXRo X3BhdGNoID0gZmlyc3RfcnVuX2ZhaWx1cmVzLmludGVyc2VjdGlvbihzZWNvbmRfcnVuX2ZhaWx1 cmVzKQogICAgICAgICBmbGFreV9mYWlsdXJlcyA9IGZpcnN0X3J1bl9mYWlsdXJlcy51bmlvbihz ZWNvbmRfcnVuX2ZhaWx1cmVzKSAtIGZpcnN0X3J1bl9mYWlsdXJlcy5pbnRlcnNlY3Rpb24oc2Vj b25kX3J1bl9mYWlsdXJlcykKQEAgLTE1MTcsNyArMTUxOSw5IEBAIGNsYXNzIEFuYWx5emVBUElU ZXN0c1Jlc3VsdHMoYnVpbGRzdGVwLkIKICAgICAgICAgICAgIHNlbGYuYnVpbGQucmVzdWx0cyA9 IFNVQ0NFU1MKICAgICAgICAgICAgIHNlbGYuZGVzY3JpcHRpb25Eb25lID0gJ1Bhc3NlZCBBUEkg dGVzdHMnCiAgICAgICAgICAgICBwbHVyYWxTdWZmaXggPSAncycgaWYgbGVuKGNsZWFuX3RyZWVf ZmFpbHVyZXMpID4gMSBlbHNlICcnCi0gICAgICAgICAgICBtZXNzYWdlID0gJ0ZvdW5kIHt9IHBy ZS1leGlzdGluZyBBUEkgdGVzdCBmYWlsdXJle30nLmZvcm1hdChsZW4oY2xlYW5fdHJlZV9mYWls dXJlcyksIHBsdXJhbFN1ZmZpeCkKKyAgICAgICAgICAgIG1lc3NhZ2UgPSAnRm91bmQge30gcHJl LWV4aXN0aW5nIEFQSSB0ZXN0IGZhaWx1cmV7fToge30nLmZvcm1hdChsZW4oY2xlYW5fdHJlZV9m YWlsdXJlcyksIHBsdXJhbFN1ZmZpeCwgY2xlYW5fdHJlZV9mYWlsdXJlc19zdHJpbmcpCisgICAg ICAgICAgICBpZiBsZW4oY2xlYW5fdHJlZV9mYWlsdXJlcykgPiBzZWxmLk5VTV9BUElfRkFJTFVS RVNfVE9fRElTUExBWToKKyAgICAgICAgICAgICAgICBtZXNzYWdlICs9ICcgLi4uJwogICAgICAg ICAgICAgaWYgZmxha3lfZmFpbHVyZXM6CiAgICAgICAgICAgICAgICAgbWVzc2FnZSArPSAnLiBG bGFreSB0ZXN0czoge30nLmZvcm1hdChmbGFreV9mYWlsdXJlc19zdHJpbmcpCiAgICAgICAgICAg ICBzZWxmLmJ1aWxkLmJ1aWxkRmluaXNoZWQoW21lc3NhZ2VdLCBTVUNDRVNTKQo=