210115 2020-04-07 08:53:24 -0700 EWS should skip mac-wk1 and mac-debug-wk1 tests for patches that only change WebKit2 sources 2020-04-29 15:37:23 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=210094 https://bugs.webkit.org/show_bug.cgi?id=211210 InRadar P2 Normal --- 1 ddkilzer aakash_jain aakash_jain ap jbedard ryanhaddad webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1638561 0 ddkilzer 2020-04-07 08:53:24 -0700 EWS should skip mac-wk1 and mac-debug-wk1 tests for patches that only change WebKit2 sources. The patches attached to Bug 210094 only changed files in the Source/WebKit folder. Running mac-wk1 and mac-debug-wk1 tests doesn't seem useful in that case. 1638562 1 ddkilzer 2020-04-07 08:57:04 -0700 (In reply to David Kilzer (:ddkilzer) from comment #0) > Running mac-wk1 and mac-debug-wk1 tests doesn't seem useful in that case. The only thing it would provide is more evidence of flakey WK1 tests on those bots, I guess. I'm not sure if folks use EWS test results such a specific way, though. (I suspect there are plenty of patches that would run the WK1 tests to discover flakey tests, though.) 1638581 2 webkit-bug-importer 2020-04-07 09:29:28 -0700 <rdar://problem/61395869> 1638583 3 aakash_jain 2020-04-07 09:32:02 -0700 > I'm not sure if folks use EWS test results such a specific way, though. We don't currently use the flakiness information from EWS in any meaningful manner, apart from checking manually when a test becomes too much flaky. 1638584 4 aakash_jain 2020-04-07 09:35:10 -0700 (In reply to David Kilzer (:ddkilzer) from comment #0) > EWS should skip mac-wk1 and mac-debug-wk1 tests for patches that only change WebKit2 sources. I like the idea. Can we come up with a list of folders for which wk1 tests should be run? e.g.: 'Source/WebKitLegacy', 'LayoutTests' The tests will be skipped if the patch doesn't touch any of those. We have similar logic in ews for various queues like jsc/webkitpy/service/bindings in https://trac.webkit.org/browser/webkit/trunk/Tools/BuildSlaveSupport/ews-build/steps.py#L250 1638619 5 ddkilzer 2020-04-07 10:20:06 -0700 (In reply to Aakash Jain from comment #4) > (In reply to David Kilzer (:ddkilzer) from comment #0) > > EWS should skip mac-wk1 and mac-debug-wk1 tests for patches that only change WebKit2 sources. > I like the idea. > Can we come up with a list of folders for which wk1 tests should be run? > e.g.: 'Source/WebKitLegacy', 'LayoutTests' Basically anything in WebKitLegacy or "above", including tests: Source/bmalloc Source/WTF Source/JavaScriptCore Source/ThirdParty/{ANGLE,libwebrtc,gtest} Source/WebCore Source/WebDriver (?) Source/WebInspectorUI (?) Source/WebKitLegacy LayoutTests Tools I think it would be easier to say "if the patch only changes files under Source/WebKit, then skip wk1 EWS tests"; rather than "if the patch changes these directories, run wk1 EWS tests". Maybe it's easier to do the latter, though? 1639048 6 395807 aakash_jain 2020-04-08 07:44:12 -0700 Created attachment 395807 Patch 1639052 7 aakash_jain 2020-04-08 08:00:31 -0700 (In reply to David Kilzer (:ddkilzer) from comment #5) > I think it would be easier to say "if the patch only changes files under Source/WebKit, then skip wk1 EWS tests"; rather than "if the patch changes these directories, run wk1 EWS tests". > > Maybe it's easier to do the latter, though? Yes, we already have the code for that. I can look into adding support for the former if we realize that current approach isn't working well. 1639053 8 aakash_jain 2020-04-08 08:01:11 -0700 Sample runs: Patch without relevant change: https://ews-build.webkit-uat.org/#/builders/32/builds/4 Patch with relevant change: https://ews-build.webkit-uat.org/#/builders/32/builds/3 1639073 9 ews-feeder 2020-04-08 08:42:04 -0700 Committed r259718: <https://trac.webkit.org/changeset/259718> All reviewed patches have been landed. Closing bug and clearing flags on attachment 395807. 1639564 10 aakash_jain 2020-04-09 04:09:03 -0700 Deployed on the server. Seems to be working fine. e.g.: https://bugs.webkit.org/show_bug.cgi?id=210229 Please let me know if you notice wk1 tests being skipped on any patch on which it shouldn't be skipped, or if you have further ideas to skip build/test on other queues. 1639811 11 ddkilzer 2020-04-09 13:10:08 -0700 (In reply to Aakash Jain from comment #10) > Deployed on the server. Seems to be working fine. e.g.: > https://bugs.webkit.org/show_bug.cgi?id=210229 > > Please let me know if you notice wk1 tests being skipped on any patch on > which it shouldn't be skipped, or if you have further ideas to skip > build/test on other queues. Awesome! Will do. 395807 2020-04-08 07:44:12 -0700 2020-04-08 08:42:05 -0700 Patch bug-210115-20200408104411.patch text/plain 8382 aakash_jain SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI1OTcxNSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDIzIEBACisyMDIwLTA0LTA4ICBBYWthc2ggSmFpbiAgPGFha2FzaF9qYWluQGFwcGxlLmNv bT4KKworICAgICAgICBFV1Mgc2hvdWxkIHNraXAgbWFjLXdrMSBhbmQgbWFjLWRlYnVnLXdrMSB0 ZXN0cyBmb3IgcGF0Y2hlcyB0aGF0IG9ubHkgY2hhbmdlIFdlYktpdDIgc291cmNlcworICAgICAg ICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MjEwMTE1CisKKyAgICAg ICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgKiBCdWlsZFNsYXZlU3Vw cG9ydC9ld3MtYnVpbGQvZmFjdG9yaWVzLnB5OgorICAgICAgICAobWFjT1NXSzFGYWN0b3J5Ll9f aW5pdF9fKTogU2V0IGNoZWNrUmVsZXZhbmNlIHRvIFRydWUgZm9yIG1hYyB3azEgcXVldWUuCisg ICAgICAgICogQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5OgorICAgICAgICAo Q2hlY2tQYXRjaFJlbGV2YW5jZSk6IEFkZGVkIHJlbGV2YW5jZSBpbmZvIGZvciB3azEgcXVldWVz LgorICAgICAgICAqIEJ1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9mYWN0b3JpZXNfdW5pdHRl c3QucHk6IFVwZGF0ZWQgdW5pdC10ZXN0cy4KKyAgICAgICAgKFRlc3RMYXlvdXRUZXN0c0ZhY3Rv cnkudGVzdF9tYWNvc193azFfcmVsZWFzZV9mYWN0b3J5KTogCisgICAgICAgIChUZXN0TGF5b3V0 VGVzdHNGYWN0b3J5LnRlc3RfbWFjb3Nfd2sxX2RlYnVnX2ZhY3RvcnkpOgorICAgICAgICAqIEJ1 aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwc191bml0dGVzdC5weToKKyAgICAgICAgKFRl c3RDaGVja1BhdGNoUmVsZXZhbmNlLnRlc3RfcmVsZXZhbnRfanNjX3BhdGNoKToKKyAgICAgICAg KFRlc3RDaGVja1BhdGNoUmVsZXZhbmNlLnRlc3RfcmVsZXZhbnRfd2sxX3BhdGNoKToKKyAgICAg ICAgKFRlc3RDaGVja1BhdGNoUmVsZXZhbmNlLnRlc3RfcXVldWVzX3dpdGhvdXRfcmVsZXZhbmNl X2luZm8pOgorICAgICAgICAoVGVzdENoZWNrUGF0Y2hSZWxldmFuY2UudGVzdF9ub25fcmVsZXZh bnRfcGF0Y2gpOgorCiAyMDIwLTA0LTA4ICBDbGF1ZGlvIFNhYXZlZHJhICA8Y3NhYXZlZHJhQGln YWxpYS5jb20+CiAKICAgICAgICAgW0dUS10gRHJvcCB1c2FnZSBvZiBkZXByZWNhdGVkIEdka0Rl dmljZU1hbmFnZXIKSW5kZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9mYWN0 b3JpZXMucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxk L2ZhY3Rvcmllcy5weQkocmV2aXNpb24gMjU5NzE1KQorKysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBv cnQvZXdzLWJ1aWxkL2ZhY3Rvcmllcy5weQkod29ya2luZyBjb3B5KQpAQCAtMTExLDggKzExMSw4 IEBAIGNsYXNzIFRlc3RGYWN0b3J5KEZhY3RvcnkpOgogICAgICAgICBzZWxmLmFkZFN0ZXAoRG93 bmxvYWRCdWlsdFByb2R1Y3QoKSkKICAgICAgICAgc2VsZi5hZGRTdGVwKEV4dHJhY3RCdWlsdFBy b2R1Y3QoKSkKIAotICAgIGRlZiBfX2luaXRfXyhzZWxmLCBwbGF0Zm9ybSwgY29uZmlndXJhdGlv bj1Ob25lLCBhcmNoaXRlY3R1cmVzPU5vbmUsIGFkZGl0aW9uYWxBcmd1bWVudHM9Tm9uZSwgKipr d2FyZ3MpOgotICAgICAgICBGYWN0b3J5Ll9faW5pdF9fKHNlbGYsIHBsYXRmb3JtPXBsYXRmb3Jt LCBjb25maWd1cmF0aW9uPWNvbmZpZ3VyYXRpb24sIGFyY2hpdGVjdHVyZXM9YXJjaGl0ZWN0dXJl cywgYnVpbGRPbmx5PUZhbHNlLCBhZGRpdGlvbmFsQXJndW1lbnRzPWFkZGl0aW9uYWxBcmd1bWVu dHMpCisgICAgZGVmIF9faW5pdF9fKHNlbGYsIHBsYXRmb3JtLCBjb25maWd1cmF0aW9uPU5vbmUs IGFyY2hpdGVjdHVyZXM9Tm9uZSwgYWRkaXRpb25hbEFyZ3VtZW50cz1Ob25lLCBjaGVja1JlbGV2 YW5jZT1GYWxzZSwgKiprd2FyZ3MpOgorICAgICAgICBGYWN0b3J5Ll9faW5pdF9fKHNlbGYsIHBs YXRmb3JtPXBsYXRmb3JtLCBjb25maWd1cmF0aW9uPWNvbmZpZ3VyYXRpb24sIGFyY2hpdGVjdHVy ZXM9YXJjaGl0ZWN0dXJlcywgYnVpbGRPbmx5PUZhbHNlLCBhZGRpdGlvbmFsQXJndW1lbnRzPWFk ZGl0aW9uYWxBcmd1bWVudHMsIGNoZWNrUmVsZXZhbmNlPWNoZWNrUmVsZXZhbmNlKQogICAgICAg ICBpZiBwbGF0Zm9ybSA9PSAnZ3RrJzoKICAgICAgICAgICAgIHNlbGYuYWRkU3RlcChJbnN0YWxs R3RrRGVwZW5kZW5jaWVzKCkpCiAgICAgICAgIHNlbGYuZ2V0UHJvZHVjdCgpCkBAIC0xNTIsNiAr MTUyLDkgQEAgY2xhc3MgbWFjT1NCdWlsZEZhY3RvcnkoQnVpbGRGYWN0b3J5KToKIGNsYXNzIG1h Y09TV0sxRmFjdG9yeShUZXN0RmFjdG9yeSk6CiAgICAgTGF5b3V0VGVzdENsYXNzID0gUnVuV2Vi S2l0MVRlc3RzCiAKKyAgICBkZWYgX19pbml0X18oc2VsZiwgcGxhdGZvcm0sIGNvbmZpZ3VyYXRp b249Tm9uZSwgYXJjaGl0ZWN0dXJlcz1Ob25lLCBhZGRpdGlvbmFsQXJndW1lbnRzPU5vbmUsIGNo ZWNrUmVsZXZhbmNlPUZhbHNlLCAqKmt3YXJncyk6CisgICAgICAgIHN1cGVyKG1hY09TV0sxRmFj dG9yeSwgc2VsZikuX19pbml0X18ocGxhdGZvcm09cGxhdGZvcm0sIGNvbmZpZ3VyYXRpb249Y29u ZmlndXJhdGlvbiwgYXJjaGl0ZWN0dXJlcz1hcmNoaXRlY3R1cmVzLCBhZGRpdGlvbmFsQXJndW1l bnRzPWFkZGl0aW9uYWxBcmd1bWVudHMsIGNoZWNrUmVsZXZhbmNlPVRydWUsICoqa3dhcmdzKQor CiAKIGNsYXNzIG1hY09TV0syRmFjdG9yeShUZXN0RmFjdG9yeSk6CiAgICAgTGF5b3V0VGVzdENs YXNzID0gUnVuV2ViS2l0VGVzdHMKSW5kZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1i dWlsZC9mYWN0b3JpZXNfdW5pdHRlc3QucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQnVpbGRTbGF2 ZVN1cHBvcnQvZXdzLWJ1aWxkL2ZhY3Rvcmllc191bml0dGVzdC5weQkocmV2aXNpb24gMjU5NzE1 KQorKysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxkL2ZhY3Rvcmllc191bml0dGVz dC5weQkod29ya2luZyBjb3B5KQpAQCAtMjA4LDYgKzIwOCw3IEBAIGNsYXNzIFRlc3RMYXlvdXRU ZXN0c0ZhY3RvcnkoVGVzdENhc2UpOgogICAgICAgICBmYWN0b3J5ID0gZmFjdG9yaWVzLm1hY09T V0sxRmFjdG9yeShwbGF0Zm9ybT0nbWFjLW1vamF2ZScsIGNvbmZpZ3VyYXRpb249J3JlbGVhc2Un LCBhcmNoaXRlY3R1cmVzPVsieDg2XzY0Il0pCiAgICAgICAgIHNlbGYuYXNzZXJ0QnVpbGRTdGVw cyhmYWN0b3J5LnN0ZXBzLCBbCiAgICAgICAgICAgICBfQnVpbGRTdGVwRmFjdG9yeShzdGVwcy5D b25maWd1cmVCdWlsZCwgcGxhdGZvcm09J21hYy1tb2phdmUnLCBjb25maWd1cmF0aW9uPSdyZWxl YXNlJywgYXJjaGl0ZWN0dXJlcz1bIng4Nl82NCJdLCBidWlsZE9ubHk9RmFsc2UsIHRyaWdnZXJz PU5vbmUsIHJlbW90ZXM9Tm9uZSwgYWRkaXRpb25hbEFyZ3VtZW50cz1Ob25lKSwKKyAgICAgICAg ICAgIF9CdWlsZFN0ZXBGYWN0b3J5KHN0ZXBzLkNoZWNrUGF0Y2hSZWxldmFuY2UpLAogICAgICAg ICAgICAgX0J1aWxkU3RlcEZhY3Rvcnkoc3RlcHMuVmFsaWRhdGVQYXRjaCksCiAgICAgICAgICAg ICBfQnVpbGRTdGVwRmFjdG9yeShzdGVwcy5QcmludENvbmZpZ3VyYXRpb24pLAogICAgICAgICAg ICAgX0J1aWxkU3RlcEZhY3Rvcnkoc3RlcHMuQ2hlY2tPdXRTb3VyY2UpLApAQCAtMjI0LDYgKzIy NSw3IEBAIGNsYXNzIFRlc3RMYXlvdXRUZXN0c0ZhY3RvcnkoVGVzdENhc2UpOgogICAgICAgICBm YWN0b3J5ID0gZmFjdG9yaWVzLm1hY09TV0sxRmFjdG9yeShwbGF0Zm9ybT0nbWFjLW1vamF2ZScs IGNvbmZpZ3VyYXRpb249J2RlYnVnJywgYXJjaGl0ZWN0dXJlcz1bIng4Nl82NCJdKQogICAgICAg ICBzZWxmLmFzc2VydEJ1aWxkU3RlcHMoZmFjdG9yeS5zdGVwcywgWwogICAgICAgICAgICAgX0J1 aWxkU3RlcEZhY3Rvcnkoc3RlcHMuQ29uZmlndXJlQnVpbGQsIHBsYXRmb3JtPSdtYWMtbW9qYXZl JywgY29uZmlndXJhdGlvbj0nZGVidWcnLCBhcmNoaXRlY3R1cmVzPVsieDg2XzY0Il0sIGJ1aWxk T25seT1GYWxzZSwgdHJpZ2dlcnM9Tm9uZSwgcmVtb3Rlcz1Ob25lLCBhZGRpdGlvbmFsQXJndW1l bnRzPU5vbmUpLAorICAgICAgICAgICAgX0J1aWxkU3RlcEZhY3Rvcnkoc3RlcHMuQ2hlY2tQYXRj aFJlbGV2YW5jZSksCiAgICAgICAgICAgICBfQnVpbGRTdGVwRmFjdG9yeShzdGVwcy5WYWxpZGF0 ZVBhdGNoKSwKICAgICAgICAgICAgIF9CdWlsZFN0ZXBGYWN0b3J5KHN0ZXBzLlByaW50Q29uZmln dXJhdGlvbiksCiAgICAgICAgICAgICBfQnVpbGRTdGVwRmFjdG9yeShzdGVwcy5DaGVja091dFNv dXJjZSksCkluZGV4OiBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3RlcHMucHkK PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PQotLS0gVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvZXdzLWJ1aWxkL3N0ZXBzLnB5 CShyZXZpc2lvbiAyNTk3MTUpCisrKyBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQv c3RlcHMucHkJKHdvcmtpbmcgY29weSkKQEAgLTI4MCw2ICsyODAsMTkgQEAgY2xhc3MgQ2hlY2tQ YXRjaFJlbGV2YW5jZShidWlsZHN0ZXAuQnVpbAogICAgICAgICAnVG9vbHMvU2NyaXB0cy93ZWJr aXRkaXJzLnBtJywKICAgICBdCiAKKyAgICB3azFfcGF0aHMgPSBbCisgICAgICAgICdTb3VyY2Uv V2ViS2l0TGVnYWN5JywKKyAgICAgICAgJ1NvdXJjZS9XZWJDb3JlJywKKyAgICAgICAgJ1NvdXJj ZS9XZWJJbnNwZWN0b3JVSScsCisgICAgICAgICdTb3VyY2UvV2ViRHJpdmVyJywKKyAgICAgICAg J1NvdXJjZS9XVEYnLAorICAgICAgICAnU291cmNlL2JtYWxsb2MnLAorICAgICAgICAnU291cmNl L0phdmFTY3JpcHRDb3JlJywKKyAgICAgICAgJ1NvdXJjZS9UaGlyZFBhcnR5JywKKyAgICAgICAg J0xheW91dFRlc3RzJywKKyAgICAgICAgJ1Rvb2xzJywKKyAgICBdCisKICAgICB3ZWJraXRweV9w YXRocyA9IFsKICAgICAgICAgJ1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvJywKICAgICAgICAgJ1Rv b2xzL1F1ZXVlU3RhdHVzU2VydmVyLycsCkBAIC0yOTAsNiArMzAzLDcgQEAgY2xhc3MgQ2hlY2tQ YXRjaFJlbGV2YW5jZShidWlsZHN0ZXAuQnVpbAogICAgICAgICAnc2VydmljZXMtZXdzJzogc2Vy dmljZXNfcGF0aHMsCiAgICAgICAgICdqc2MnOiBqc2NfcGF0aHMsCiAgICAgICAgICd3ZWJraXRw eSc6IHdlYmtpdHB5X3BhdGhzLAorICAgICAgICAnd2sxLXRlc3RzJzogd2sxX3BhdGhzLAogICAg IH0KIAogICAgIGRlZiBfcGF0Y2hfaXNfcmVsZXZhbnQoc2VsZiwgcGF0Y2gsIGJ1aWxkZXJOYW1l KToKSW5kZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2V3cy1idWlsZC9zdGVwc191bml0dGVz dC5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09Ci0tLSBUb29scy9CdWlsZFNsYXZlU3VwcG9ydC9ld3MtYnVpbGQvc3Rl cHNfdW5pdHRlc3QucHkJKHJldmlzaW9uIDI1OTcxNSkKKysrIFRvb2xzL0J1aWxkU2xhdmVTdXBw b3J0L2V3cy1idWlsZC9zdGVwc191bml0dGVzdC5weQkod29ya2luZyBjb3B5KQpAQCAtMjE1OSw3 ICsyMTU5LDcgQEAgY2xhc3MgVGVzdENoZWNrUGF0Y2hSZWxldmFuY2UoQnVpbGRTdGVwTQogICAg IGRlZiB0ZWFyRG93bihzZWxmKToKICAgICAgICAgcmV0dXJuIHNlbGYudGVhckRvd25CdWlsZFN0 ZXAoKQogCi0gICAgZGVmIHRlc3RfcmVsZXZhbnRfcGF0Y2goc2VsZik6CisgICAgZGVmIHRlc3Rf cmVsZXZhbnRfanNjX3BhdGNoKHNlbGYpOgogICAgICAgICBDaGVja1BhdGNoUmVsZXZhbmNlLl9n ZXRfcGF0Y2ggPSBsYW1iZGEgeDogJ1NhbXBsZSBwYXRjaDsgZmlsZTogSlNUZXN0cy8nCiAgICAg ICAgIHNlbGYuc2V0dXBTdGVwKENoZWNrUGF0Y2hSZWxldmFuY2UoKSkKICAgICAgICAgc2VsZi5z ZXRQcm9wZXJ0eSgnYnVpbGRlcm5hbWUnLCAnSlNDLVRlc3RzLUVXUycpCkBAIC0yMTY4LDIwICsy MTY4LDMwIEBAIGNsYXNzIFRlc3RDaGVja1BhdGNoUmVsZXZhbmNlKEJ1aWxkU3RlcE0KICAgICAg ICAgc2VsZi5leHBlY3RPdXRjb21lKHJlc3VsdD1TVUNDRVNTLCBzdGF0ZV9zdHJpbmc9J1BhdGNo IGNvbnRhaW5zIHJlbGV2YW50IGNoYW5nZXMnKQogICAgICAgICByZXR1cm4gc2VsZi5ydW5TdGVw KCkKIAotICAgIGRlZiB0ZXN0X3F1ZXVlX3dpdGhvdXRfcmVsZXZhbmNlX2luZm8oc2VsZik6Ci0g ICAgICAgIENoZWNrUGF0Y2hSZWxldmFuY2UuX2dldF9wYXRjaCA9IGxhbWJkYSB4OiAnU2FtcGxl IHBhdGNoJworICAgIGRlZiB0ZXN0X3JlbGV2YW50X3drMV9wYXRjaChzZWxmKToKKyAgICAgICAg Q2hlY2tQYXRjaFJlbGV2YW5jZS5fZ2V0X3BhdGNoID0gbGFtYmRhIHg6ICdTYW1wbGUgcGF0Y2g7 IGZpbGU6IFNvdXJjZS9XZWJLaXRMZWdhY3knCiAgICAgICAgIHNlbGYuc2V0dXBTdGVwKENoZWNr UGF0Y2hSZWxldmFuY2UoKSkKLSAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnYnVpbGRlcm5hbWUn LCAnQ29tbWl0LVF1ZXVlJykKLSAgICAgICAgQ2hlY2tQYXRjaFN0YXR1c09uRVdTUXVldWVzLmdl dF9wYXRjaF9zdGF0dXMgPSBsYW1iZGEgY2xzLCBwYXRjaF9pZCwgcXVldWU6IEZBSUxVUkUKKyAg ICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnYnVpbGRlcm5hbWUnLCAnbWFjT1MtTW9qYXZlLVJlbGVh c2UtV0sxLVRlc3RzLUVXUycpCiAgICAgICAgIHNlbGYuZXhwZWN0T3V0Y29tZShyZXN1bHQ9U1VD Q0VTUywgc3RhdGVfc3RyaW5nPSdQYXRjaCBjb250YWlucyByZWxldmFudCBjaGFuZ2VzJykKICAg ICAgICAgcmV0dXJuIHNlbGYucnVuU3RlcCgpCiAKKyAgICBkZWYgdGVzdF9xdWV1ZXNfd2l0aG91 dF9yZWxldmFuY2VfaW5mbyhzZWxmKToKKyAgICAgICAgQ2hlY2tQYXRjaFJlbGV2YW5jZS5fZ2V0 X3BhdGNoID0gbGFtYmRhIHg6ICdTYW1wbGUgcGF0Y2gnCisgICAgICAgIHF1ZXVlcyA9IFsnQ29t bWl0LVF1ZXVlJywgJ1N0eWxlLUVXUycsICdBcHBseS1XYXRjaExpc3QtRVdTJywgJ0dUSy1CdWls ZC1FV1MnLCAnR1RLLVdLMi1UZXN0cy1FV1MnLAorICAgICAgICAgICAgICAgICAgJ2lPUy0xMy1C dWlsZC1FV1MnLCAnaU9TLTEzLVNpbXVsYXRvci1CdWlsZC1FV1MnLCAnaU9TLTEzLVNpbXVsYXRv ci1XSzItVGVzdHMtRVdTJywKKyAgICAgICAgICAgICAgICAgICdtYWNPUy1Nb2phdmUtUmVsZWFz ZS1CdWlsZC1FV1MnLCAnbWFjT1MtTW9qYXZlLVJlbGVhc2UtV0syLVRlc3RzLUVXUycsICdtYWNP Uy1Nb2phdmUtRGVidWctQnVpbGQtRVdTJywKKyAgICAgICAgICAgICAgICAgICdXaW5kb3dzLUVX UycsICdXaW5DYWlyby1FV1MnLCAnV1BFLUVXUycsICdXZWJLaXRQZXJsLVRlc3RzLUVXUyddCisg ICAgICAgIGZvciBxdWV1ZSBpbiBxdWV1ZXM6CisgICAgICAgICAgICBzZWxmLnNldHVwU3RlcChD aGVja1BhdGNoUmVsZXZhbmNlKCkpCisgICAgICAgICAgICBzZWxmLnNldFByb3BlcnR5KCdidWls ZGVybmFtZScsIHF1ZXVlKQorICAgICAgICAgICAgc2VsZi5leHBlY3RPdXRjb21lKHJlc3VsdD1T VUNDRVNTLCBzdGF0ZV9zdHJpbmc9J1BhdGNoIGNvbnRhaW5zIHJlbGV2YW50IGNoYW5nZXMnKQor ICAgICAgICByZXR1cm4gc2VsZi5ydW5TdGVwKCkKKwogICAgIGRlZiB0ZXN0X25vbl9yZWxldmFu dF9wYXRjaChzZWxmKToKICAgICAgICAgQ2hlY2tQYXRjaFJlbGV2YW5jZS5fZ2V0X3BhdGNoID0g bGFtYmRhIHg6ICdTYW1wbGUgcGF0Y2gnCiAgICAgICAgIHNlbGYuc2V0dXBTdGVwKENoZWNrUGF0 Y2hSZWxldmFuY2UoKSkKICAgICAgICAgc2VsZi5zZXRQcm9wZXJ0eSgnYnVpbGRlcm5hbWUnLCAn SlNDLVRlc3RzLUVXUycpCiAgICAgICAgIHNlbGYuc2V0UHJvcGVydHkoJ3BhdGNoX2lkJywgJzEy MzQnKQotICAgICAgICBDaGVja1BhdGNoU3RhdHVzT25FV1NRdWV1ZXMuZ2V0X3BhdGNoX3N0YXR1 cyA9IGxhbWJkYSBjbHMsIHBhdGNoX2lkLCBxdWV1ZTogRkFJTFVSRQogICAgICAgICBzZWxmLmV4 cGVjdE91dGNvbWUocmVzdWx0PUZBSUxVUkUsIHN0YXRlX3N0cmluZz0nUGF0Y2ggZG9lc25cJ3Qg Y29udGFpbiByZWxldmFudCBjaGFuZ2VzJykKICAgICAgICAgcmV0dXJuIHNlbGYucnVuU3RlcCgp CiAK