229533 2021-08-25 20:27:32 -0700 [EWS] compile-webkit-without-patch failed, but bot ran layout tests anyway and blamed the patch for a pre-existing failure 2021-09-03 06:23:22 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 ryanhaddad aakash_jain aakash_jain ap jbedard webkit-bug-importer oldest_to_newest 1787444 0 ryanhaddad 2021-08-25 20:27:32 -0700 As you can see in https://ews-build.webkit.org/#/builders/60/builds/10553, the layout-tests and re-run-layout-tests both failed. The bot then proceeded to the compile-webkit-without-patch step and failed to compile. Surprisingly, the bot still proceeded to the run-layout-tests-without-patch step, which failed with the following: 16:30:10.005 10889 WebKitTestRunner was not found at /Volumes/Data/worker/macOS-AppleSilicon-Big-Sur-Debug-WK2-Tests-EWS/build/WebKitBuild/Debug/WebKitTestRunner I would expect that the EWS run would be considered inconclusive or invalid, but instead it blamed the patch under test for introducing a new test failure that really should have been considered pre-existing. Other examples from around the same time: https://ews-build.webkit.org/#/builders/60/builds/10560 https://ews-build.webkit.org/#/builders/60/builds/10561 https://ews-build.webkit.org/#/builders/60/builds/10562 https://ews-build.webkit.org/#/builders/60/builds/10563 1787445 1 webkit-bug-importer 2021-08-25 20:28:02 -0700 <rdar://problem/82371726> 1787780 2 436560 aakash_jain 2021-08-26 13:38:25 -0700 Created attachment 436560 Patch 1787783 3 aakash_jain 2021-08-26 13:39:53 -0700 Tested on uat instance, e.g.: https://ews-build.webkit-uat.org/#/builders/34/builds/40260 1787823 4 ews-feeder 2021-08-26 15:07:20 -0700 Committed r281660 (241013@main): <https://commits.webkit.org/241013@main> All reviewed patches have been landed. Closing bug and clearing flags on attachment 436560. 1787984 5 aakash_jain 2021-08-27 03:34:32 -0700 Restarted buildbot to pick up this change. 1787990 6 aakash_jain 2021-08-27 04:20:14 -0700 buildbot on EWS and build.webkit.org has been restarted to pick up this change. 1790139 7 aakash_jain 2021-09-03 06:23:22 -0700 This change seems to be working fine. e.g.: https://ews-build.webkit.org/#/builders/51/builds/20327 https://ews-build.webkit.org/#/builders/51/builds/20385 https://ews-build.webkit.org/#/builders/51/builds/20418 https://ews-build.webkit.org/#/builders/51/builds/20448 https://ews-build.webkit.org/#/builders/10/builds/103629 436560 2021-08-26 13:38:25 -0700 2021-08-26 15:07:21 -0700 Patch bug-229533-20210826163823.patch text/plain 4976 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI4MTY0OCkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDIwIEBACisyMDIxLTA4LTI2ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBbRVdTXSBjb21waWxlLXdlYmtpdC13aXRob3V0LXBhdGNoIGZhaWxlZCwg YnV0IGJvdCByYW4gbGF5b3V0IHRlc3RzIGFueXdheSBhbmQgYmxhbWVkIHRoZSBwYXRjaCBmb3Ig YSBwcmUtZXhpc3RpbmcgZmFpbHVyZQorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9z aG93X2J1Zy5jZ2k/aWQ9MjI5NTMzCisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BT ISkuCisKKyAgICAgICAgV2hlbiB0aGUgY29tcGlsZS13ZWJraXQtd2l0aG91dC1wYXRjaCBzdGVw IGZhaWxzLCBpbnN0ZWFkIG9mIHJ1bm5pbmcgZnVydGhlciBsYXlvdXQtdGVzdHMsIHJldHJ5IHRo ZSBidWlsZCBhbmQgZW1haWwKKyAgICAgICAgYm90IHdhdGNoZXIncy4gVGhlIHJldHJpZWQgYnVp bGQgd2lsbCBob3BlZnVsbHkgYmUgcGlja2VkIHVwIGJ5IG90aGVyIGJvdCBldmVudHVhbGx5LiBU aGlzIGlzIHNpbWlsYXIgdG8gd2hhdCB3ZSBkbworICAgICAgICBpbiBjYXNlIG9mIG90aGVyIGlu ZnJhc3RydWN0dXJlIGlzc3VlcyBsaWtlIGtpbGwtb2xkLXByb2Nlc3NlcyBzdGVwIGZhaWx1cmUu CisgICAgICAgICogQ0lTdXBwb3J0L2V3cy1idWlsZC9zdGVwcy5weToKKyAgICAgICAgKENvbXBp bGVXZWJLaXRXaXRob3V0UGF0Y2guX19pbml0X18pOiBBZGRlZCByZXRyeV9idWlsZF9vbl9mYWls dXJlIHBhcmFtZXRlci4KKyAgICAgICAgKENvbXBpbGVXZWJLaXRXaXRob3V0UGF0Y2guZXZhbHVh dGVDb21tYW5kKTogSWYgYnVpbGQgZmFpbGVkIHVuZXhwZWN0ZWRseSwgcmV0cnkgaXQgYW5kIGVt YWlsIGJvdCB3YXRjaGVycy4KKyAgICAgICAgKENvbXBpbGVXZWJLaXRXaXRob3V0UGF0Y2guc2Vu ZF9lbWFpbF9mb3JfdW5leHBlY3RlZF9idWlsZF9mYWlsdXJlKToKKyAgICAgICAgKFJlUnVuV2Vi S2l0VGVzdHMuZXZhbHVhdGVDb21tYW5kKToKKyAgICAgICAgKFJlUnVuQVBJVGVzdHMuZXZhbHVh dGVDb21tYW5kKToKKwogMjAyMS0wOC0yNiAgV2Vuc29uIEhzaWVoICA8d2Vuc29uX2hzaWVoQGFw cGxlLmNvbT4KIAogICAgICAgICBUZXN0V2ViS2l0QVBJLl9XS0FjdGl2YXRlZEVsZW1lbnRJbmZv LkluZm9Gb3JSb3RhdGVkSW1hZ2UgaXMgZmFpbGluZyBvbiBpT1MgMTUKSW5kZXg6IFRvb2xzL0NJ U3VwcG9ydC9ld3MtYnVpbGQvc3RlcHMucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQ0lTdXBwb3J0 L2V3cy1idWlsZC9zdGVwcy5weQkocmV2aXNpb24gMjgxNjQ4KQorKysgVG9vbHMvQ0lTdXBwb3J0 L2V3cy1idWlsZC9zdGVwcy5weQkod29ya2luZyBjb3B5KQpAQCAtMTYzMiw2ICsxNjMyLDEwIEBA IGNsYXNzIENvbXBpbGVXZWJLaXRXaXRob3V0UGF0Y2goQ29tcGlsZVcKICAgICBuYW1lID0gJ2Nv bXBpbGUtd2Via2l0LXdpdGhvdXQtcGF0Y2gnCiAgICAgaGFsdE9uRmFpbHVyZSA9IEZhbHNlCiAK KyAgICBkZWYgX19pbml0X18oc2VsZiwgcmV0cnlfYnVpbGRfb25fZmFpbHVyZT1GYWxzZSwgKipr d2FyZ3MpOgorICAgICAgICBzZWxmLnJldHJ5X2J1aWxkX29uX2ZhaWx1cmUgPSByZXRyeV9idWls ZF9vbl9mYWlsdXJlCisgICAgICAgIHN1cGVyKENvbXBpbGVXZWJLaXRXaXRob3V0UGF0Y2gsIHNl bGYpLl9faW5pdF9fKCoqa3dhcmdzKQorCiAgICAgZGVmIGRvU3RlcElmKHNlbGYsIHN0ZXApOgog ICAgICAgICByZXR1cm4gc2VsZi5nZXRQcm9wZXJ0eSgncGF0Y2hGYWlsZWRUb0J1aWxkJykgb3Ig c2VsZi5nZXRQcm9wZXJ0eSgncGF0Y2hGYWlsZWRUZXN0cycpCiAKQEAgLTE2MzksNyArMTY0Mywy NiBAQCBjbGFzcyBDb21waWxlV2ViS2l0V2l0aG91dFBhdGNoKENvbXBpbGVXCiAgICAgICAgIHJl dHVybiBub3Qgc2VsZi5kb1N0ZXBJZihzdGVwKQogCiAgICAgZGVmIGV2YWx1YXRlQ29tbWFuZChz ZWxmLCBjbWQpOgotICAgICAgICByZXR1cm4gc2hlbGwuQ29tcGlsZS5ldmFsdWF0ZUNvbW1hbmQo c2VsZiwgY21kKQorICAgICAgICByYyA9IHNoZWxsLkNvbXBpbGUuZXZhbHVhdGVDb21tYW5kKHNl bGYsIGNtZCkKKyAgICAgICAgaWYgcmMgPT0gRkFJTFVSRSBhbmQgc2VsZi5yZXRyeV9idWlsZF9v bl9mYWlsdXJlOgorICAgICAgICAgICAgbWVzc2FnZSA9ICdVbmFibGUgdG8gYnVpbGQgV2ViS2l0 IHdpdGhvdXQgcGF0Y2gsIHJldHJ5aW5nIGJ1aWxkJworICAgICAgICAgICAgc2VsZi5kZXNjcmlw dGlvbkRvbmUgPSBtZXNzYWdlCisgICAgICAgICAgICBzZWxmLnNlbmRfZW1haWxfZm9yX3VuZXhw ZWN0ZWRfYnVpbGRfZmFpbHVyZSgpCisgICAgICAgICAgICBzZWxmLmJ1aWxkLmJ1aWxkRmluaXNo ZWQoW21lc3NhZ2VdLCBSRVRSWSkKKyAgICAgICAgcmV0dXJuIHJjCisKKyAgICBkZWYgc2VuZF9l bWFpbF9mb3JfdW5leHBlY3RlZF9idWlsZF9mYWlsdXJlKHNlbGYpOgorICAgICAgICB0cnk6Cisg ICAgICAgICAgICBidWlsZGVyX25hbWUgPSBzZWxmLmdldFByb3BlcnR5KCdidWlsZGVybmFtZScs ICcnKQorICAgICAgICAgICAgd29ya2VyX25hbWUgPSBzZWxmLmdldFByb3BlcnR5KCd3b3JrZXJu YW1lJywgJycpCisgICAgICAgICAgICBidWlsZF91cmwgPSAne30jL2J1aWxkZXJzL3t9L2J1aWxk cy97fScuZm9ybWF0KHNlbGYubWFzdGVyLmNvbmZpZy5idWlsZGJvdFVSTCwgc2VsZi5idWlsZC5f YnVpbGRlcmlkLCBzZWxmLmJ1aWxkLm51bWJlcikKKyAgICAgICAgICAgIGVtYWlsX3N1YmplY3Qg PSAne30gbWlnaHQgYmUgaW4gYmFkIHN0YXRlLCB1bmFibGUgdG8gYnVpbGQgV2ViS2l0Jy5mb3Jt YXQod29ya2VyX25hbWUpCisgICAgICAgICAgICBlbWFpbF90ZXh0ID0gJ3t9IG1pZ2h0IGJlIGlu IGJhZCBzdGF0ZS4gSXQgaXMgdW5hYmxlIHRvIGJ1aWxkIFdlYktpdC4nLmZvcm1hdCh3b3JrZXJf bmFtZSkKKyAgICAgICAgICAgIGVtYWlsX3RleHQgKz0gJyBTYW1lIHBhdGNoIHdhcyBidWlsdCBz dWNjZXNzZnVseSBvbiBidWlsZGVyIHF1ZXVlIHByZXZpb3VzbHkuXG5cbkJ1aWxkOiB7fVxuXG5C dWlsZGVyOiB7fScuZm9ybWF0KGJ1aWxkX3VybCwgYnVpbGRlcl9uYW1lKQorICAgICAgICAgICAg cmVmZXJlbmNlID0gJ2J1aWxkLWZhaWx1cmUte30nLmZvcm1hdCh3b3JrZXJfbmFtZSkKKyAgICAg ICAgICAgIHNlbmRfZW1haWxfdG9fYm90X3dhdGNoZXJzKGVtYWlsX3N1YmplY3QsIGVtYWlsX3Rl eHQsIGJ1aWxkZXJfbmFtZSwgcmVmZXJlbmNlKQorICAgICAgICBleGNlcHQgRXhjZXB0aW9uIGFz IGU6CisgICAgICAgICAgICBwcmludCgnRXJyb3IgaW4gc2VuZGluZyBlbWFpbCBmb3IgdW5leHBl Y3RlZCBidWlsZCBmYWlsdXJlOiB7fScuZm9ybWF0KGUpKQogCiAKIGNsYXNzIEFuYWx5emVDb21w aWxlV2ViS2l0UmVzdWx0cyhidWlsZHN0ZXAuQnVpbGRTdGVwLCBCdWd6aWxsYU1peGluKToKQEAg LTIzODgsNyArMjQxMSw3IEBAIGNsYXNzIFJlUnVuV2ViS2l0VGVzdHMoUnVuV2ViS2l0VGVzdHMp OgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgRXh0cmFj dFRlc3RSZXN1bHRzKGlkZW50aWZpZXI9J3JlcnVuJyksCiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBVbkFwcGx5UGF0Y2hJZlJlcXVpcmVkKCksCiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBWYWxpZGF0ZVBhdGNo KHZlcmlmeUJ1Z0Nsb3NlZD1GYWxzZSwgYWRkVVJMcz1GYWxzZSksCi0gICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBDb21waWxlV2ViS2l0V2l0aG91dFBhdGNo KCksCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBDb21w aWxlV2ViS2l0V2l0aG91dFBhdGNoKHJldHJ5X2J1aWxkX29uX2ZhaWx1cmU9VHJ1ZSksCiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBWYWxpZGF0ZVBhdGNo KHZlcmlmeUJ1Z0Nsb3NlZD1GYWxzZSwgYWRkVVJMcz1GYWxzZSksCiAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBLaWxsT2xkUHJvY2Vzc2VzKCksCiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBSdW5XZWJLaXRUZXN0 c1dpdGhvdXRQYXRjaCgpXSkKQEAgLTI4NTYsNyArMjg3OSw3IEBAIGNsYXNzIFJlUnVuQVBJVGVz dHMoUnVuQVBJVGVzdHMpOgogICAgICAgICAgICAgICAgIHN0ZXBzX3RvX2FkZC5hcHBlbmQoSW5z dGFsbFdwZURlcGVuZGVuY2llcygpKQogICAgICAgICAgICAgZWxpZiBwbGF0Zm9ybSA9PSAnZ3Rr JzoKICAgICAgICAgICAgICAgICBzdGVwc190b19hZGQuYXBwZW5kKEluc3RhbGxHdGtEZXBlbmRl bmNpZXMoKSkKLSAgICAgICAgICAgIHN0ZXBzX3RvX2FkZC5hcHBlbmQoQ29tcGlsZVdlYktpdFdp dGhvdXRQYXRjaCgpKQorICAgICAgICAgICAgc3RlcHNfdG9fYWRkLmFwcGVuZChDb21waWxlV2Vi S2l0V2l0aG91dFBhdGNoKHJldHJ5X2J1aWxkX29uX2ZhaWx1cmU9VHJ1ZSkpCiAgICAgICAgICAg ICBzdGVwc190b19hZGQuYXBwZW5kKFZhbGlkYXRlUGF0Y2godmVyaWZ5QnVnQ2xvc2VkPUZhbHNl LCBhZGRVUkxzPUZhbHNlKSkKICAgICAgICAgICAgIHN0ZXBzX3RvX2FkZC5hcHBlbmQoS2lsbE9s ZFByb2Nlc3NlcygpKQogICAgICAgICAgICAgc3RlcHNfdG9fYWRkLmFwcGVuZChSdW5BUElUZXN0 c1dpdGhvdXRQYXRjaCgpKQo=