185054 2018-04-26 15:02:34 -0700 REGRESSION (r229955): run-webkit-tests runs tests in skipped directories and subdirectories of directory given on command line 2018-05-07 11:33:00 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=66308 InRadar, Regression P1 Normal --- 183681 1 dbates dbates ews-watchlist glenn jbedard lforschler rniwa webkit-bug-importer oldest_to_newest 1418272 0 dbates 2018-04-26 15:02:34 -0700 According the description of --skipped in run-webkit-tests --help, when --skipped is omitted run-webkit-tests should "skip tests [marked SKIP explicitly or implicitly because they are in a directory marked SKIP (*)] unless explicitly listed on the command line". Steps to reproduce: The following steps assume you are running the tests on a Mac and that http/tests/quicklook is skipped on Mac, which it is at the time of writing it is by <https://trac.webkit.org/browser/trunk/LayoutTests/TestExpectations?rev=231065#L110>. (http/tests/quicklook is skipped on all non-iOS ports). 1. Run `Tools/Scripts/run-webkit-tests http/tests/quicklook`. Then run-webkit-tests will run the tests in http/tests/quicklook. But it should not have because --skipped was not given and http/test/quicklook is skipped on Mac. Similarly, running `Tools/Scripts/run-webkit-tests http/tests` will run tests in http/tests/quicklook. But it should not have by the same aforementioned reason. (*) This criterion is implied by the second paragraph in the ChangeLog that added --skipped: <https://trac.webkit.org/changeset/120348/>. 1418273 1 dbates 2018-04-26 15:04:52 -0700 On another note, run-webkit-tests does not seem to honor --skipped. Running `Tools/Scripts/run-webkit-tests --skipped=always http/tests/quicklook` runs the tests. But it should not because the directory http/tests/quicklook is skipped on Mac. 1418279 2 dbates 2018-04-26 15:13:17 -0700 This bugs significantly impacts my development because I often test correctness of a change in select directories to save time and build confidence before running all the tests. Having run-webkit-tests run tests that should be skipped when I invoke it with a selection of directories makes it difficult to differentiate between regressions I caused (if any) from test failures associated with skipped tests that are being run. 1418281 3 webkit-bug-importer 2018-04-26 15:13:48 -0700 <rdar://problem/39773209> 1418314 4 dbates 2018-04-26 16:21:13 -0700 This regressed with the change in <https://trac.webkit.org/changeset/229955/> (bug #183681). 1418322 5 dbates 2018-04-26 16:48:30 -0700 If I remove <https://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/port/mac.py?rev=230998#L63> and <https://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/port/mac.py?rev=230998#L64> then the bug does not occur when I run `Tools/Scripts/run-webkit-tests http/tests/quicklook`. 1420761 6 339568 dbates 2018-05-04 11:10:21 -0700 Created attachment 339568 Patch I did not write a test because the port's OS version is a private field as it represents an implementation detail and I did not feel the risk of regression was great enough to justify exposing this detail. Moreover, I was unclear how I could indirectly test this change by way of MacPort.default_baseline_search_path(). 1420766 7 dbates 2018-05-04 11:16:01 -0700 EWS reveals that there is a way to test this behavior change. 1420779 8 339574 dbates 2018-05-04 11:32:23 -0700 Created attachment 339574 Patch and unit tests 1420870 9 339574 jbedard 2018-05-04 14:01:01 -0700 Comment on attachment 339574 Patch and unit tests View in context: https://bugs.webkit.org/attachment.cgi?id=339574&action=review Looks sane to me. Unofficial r+. > Tools/Scripts/webkitpy/port/mac_unittest.py:163 > + self.assertEqual(port.version_name(), VersionNameMap.map(port.host.platform).to_name(MacPort.CURRENT_VERSION, platform=MacPort.port_name)) We shouldn't need the platform passed to VersionNameMap.map() and passed to VersionNameMap.to_name(...). One of the two should suffice. 1421354 10 dbates 2018-05-07 11:30:55 -0700 (In reply to Jonathan Bedard from comment #9) > > Tools/Scripts/webkitpy/port/mac_unittest.py:163 > > + self.assertEqual(port.version_name(), VersionNameMap.map(port.host.platform).to_name(MacPort.CURRENT_VERSION, platform=MacPort.port_name)) > > We shouldn't need the platform passed to VersionNameMap.map() and passed to > VersionNameMap.to_name(...). One of the two should suffice. Will change VersionNameMap expression in this line and throughout the patch to read: VersionNameMap().to_name(MacPort.CURRENT_VERSION, platform=MacPort.port_name) Additional remarks: VersionNameMap is so confusing to use (see bug #185386). I counted at least 8 ways to map a version number to name! Filed bug #185387. 1421355 11 dbates 2018-05-07 11:33:00 -0700 Committed r231449: <https://trac.webkit.org/changeset/231449> 339568 2018-05-04 11:10:21 -0700 2018-05-04 11:32:21 -0700 Patch bug-185054-20180504111020.patch text/plain 2089 dbates U3VidmVyc2lvbiBSZXZpc2lvbjogMjMxMzU1CmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggNzE3OTczMDM5ODI0MDQyMzc5NjQxNDY2ZTdlNDk5MGEx YzcyMDkzNi4uYzc2NDdlNjE3ODg3NDhiN2MxNzI4NzgzNGVmZjM1ZWUyYTU0ZTg5NiAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDIx IEBACisyMDE4LTA1LTA0ICBEYW5pZWwgQmF0ZXMgIDxkYWJhdGVzQGFwcGxlLmNvbT4KKworICAg ICAgICBSRUdSRVNTSU9OIChyMjI5OTU1KTogcnVuLXdlYmtpdC10ZXN0cyBydW5zIHRlc3RzIGlu IHNraXBwZWQgZGlyZWN0b3JpZXMgYW5kIHN1YmRpcmVjdG9yaWVzIG9mIGRpcmVjdG9yeSBnaXZl biBvbiBjb21tYW5kIGxpbmUKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19i dWcuY2dpP2lkPTE4NTA1NAorICAgICAgICA8cmRhcjovL3Byb2JsZW0vMzk3NzMyMDk+CisKKyAg ICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgRmFsbGJhY2sgdG8g TWFjUG9ydC5DVVJSRU5UX1ZFUlNJT04gaWYgd2UgYXJlIG5vdCB1c2luZyBhIGtub3duIG5hbWVk IG1hY09TIGFuZCBkbyBub3QgaGF2ZSBBcHBsZSBBZGRpdGlvbnMuCisKKyAgICAgICAgRGFyd2lu LWJhc2VkIHBvcnRzIGhhdmUgT1MgdmVyc2lvbi1zcGVjaWZpYyB0ZXN0IGV4cGVjdGF0aW9ucyBh bmQgYXJlIGNlcnRpZmllZCB0byBzdXBwb3J0IHJ1bm5pbmcgdGVzdHMgdXNpbmcKKyAgICAgICAg dGhlIGN1cnJlbnRseSBzaGlwcGluZyBPUyBieSB3YXkgb2YgYSBoYXJkY29kZWQgY2xhc3MgdmFy aWFibGUgQ1VSUkVOVF9WRVJTSU9OLiBUaGUgcnVuLXdlYmtpdC10ZXN0cyBzY3JpcHQKKyAgICAg ICAgZGVwZW5kcyBvbiB0aGUgT1MgdmVyc2lvbiB0byBmaW5kIHRoZSBhcHBsaWNhYmxlIFRlc3RF eHBlY3RhdGlvbnMgZmlsZXMgdG8gcGFyc2UgdG8gY29tcHV0ZSB0aGUgbGlzdCBvZiBza2lwcGVk CisgICAgICAgIHRlc3RzIGZvciB0aGUgcG9ydC4KKworICAgICAgICAqIFNjcmlwdHMvd2Via2l0 cHkvcG9ydC9tYWMucHk6CisgICAgICAgIChNYWNQb3J0Ll9faW5pdF9fKToKKwogMjAxOC0wNS0w NCAgQ2FybG9zIEdhcmNpYSBDYW1wb3MgIDxjZ2FyY2lhQGlnYWxpYS5jb20+CiAKICAgICAgICAg W0dUS10gRXBpcGhhbnkgKEdOT01FIFdlYikgc2F5cyAiRXJyb3IgZG93bmxvYWRpbmc6IFNlcnZp Y2UgVW5hdmFpbGFibGUuIiB3aGVuIHRyeWluZyB0byBkb3dubG9hZCBhbiBpbWFnZSBmcm9tIGRp c2NvZ3MuY29tCmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3BvcnQvbWFjLnB5 IGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wb3J0L21hYy5weQppbmRleCA3MTBhODEwNDEyZmEx ZTUyOWMyNDE2ZTFjYzRmMTY3YmU5NTI0ODJjLi5mMmQ3NjFlODJlYTJkOGFiMmQwZmIxYmUyMmU1 OTU1NGI1N2Y1MjBjIDEwMDY0NAotLS0gYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3BvcnQvbWFj LnB5CisrKyBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvcG9ydC9tYWMucHkKQEAgLTYwLDcgKzYw LDcgQEAgY2xhc3MgTWFjUG9ydChEYXJ3aW5Qb3J0KToKICAgICAgICAgc3BsaXRfcG9ydF9uYW1l ID0gcG9ydF9uYW1lLnNwbGl0KCctJykKICAgICAgICAgaWYgbGVuKHNwbGl0X3BvcnRfbmFtZSkg PiAxIGFuZCBzcGxpdF9wb3J0X25hbWVbMV0gIT0gJ3drMic6CiAgICAgICAgICAgICBzZWxmLl9v c192ZXJzaW9uID0gdmVyc2lvbl9uYW1lX21hcC5mcm9tX25hbWUoc3BsaXRfcG9ydF9uYW1lWzFd KVsxXQotICAgICAgICBlbGlmIHNlbGYuaG9zdC5wbGF0Zm9ybS5pc19tYWMoKToKKyAgICAgICAg ZWxpZiBzZWxmLmhvc3QucGxhdGZvcm0uaXNfbWFjKCkgYW5kIGFwcGxlX2FkZGl0aW9ucygpOgog ICAgICAgICAgICAgc2VsZi5fb3NfdmVyc2lvbiA9IHNlbGYuaG9zdC5wbGF0Zm9ybS5vc192ZXJz aW9uCiAgICAgICAgIGlmIG5vdCBzZWxmLl9vc192ZXJzaW9uOgogICAgICAgICAgICAgc2VsZi5f b3NfdmVyc2lvbiA9IE1hY1BvcnQuQ1VSUkVOVF9WRVJTSU9OCg== 339574 2018-05-04 11:32:23 -0700 2018-05-04 13:58:12 -0700 Patch and unit tests bug-185054-20180504113222.patch text/plain 6575 dbates U3VidmVyc2lvbiBSZXZpc2lvbjogMjMxMzU1CmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggNzE3OTczMDM5ODI0MDQyMzc5NjQxNDY2ZTdlNDk5MGEx YzcyMDkzNi4uYjY3OTQ1YjVjMmJiNjE4MGRjZDNlMThiNTk5NzZlNDNjOTcwYzBhZiAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDI4 IEBACisyMDE4LTA1LTA0ICBEYW5pZWwgQmF0ZXMgIDxkYWJhdGVzQGFwcGxlLmNvbT4KKworICAg ICAgICBSRUdSRVNTSU9OIChyMjI5OTU1KTogcnVuLXdlYmtpdC10ZXN0cyBydW5zIHRlc3RzIGlu IHNraXBwZWQgZGlyZWN0b3JpZXMgYW5kIHN1YmRpcmVjdG9yaWVzIG9mIGRpcmVjdG9yeSBnaXZl biBvbiBjb21tYW5kIGxpbmUKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19i dWcuY2dpP2lkPTE4NTA1NAorICAgICAgICA8cmRhcjovL3Byb2JsZW0vMzk3NzMyMDk+CisKKyAg ICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgRmFsbGJhY2sgdG8g TWFjUG9ydC5DVVJSRU5UX1ZFUlNJT04gaWYgd2UgYXJlIG5vdCB1c2luZyBhIGtub3duIG5hbWVk IG1hY09TIGFuZCBkbyBub3QgaGF2ZSBBcHBsZSBBZGRpdGlvbnMuCisKKyAgICAgICAgRGFyd2lu LWJhc2VkIHBvcnRzIGhhdmUgT1MgdmVyc2lvbi1zcGVjaWZpYyB0ZXN0IGV4cGVjdGF0aW9ucyBh bmQgYXJlIGNlcnRpZmllZCB0byBzdXBwb3J0IHJ1bm5pbmcgdGVzdHMgdXNpbmcKKyAgICAgICAg dGhlIGN1cnJlbnRseSBzaGlwcGluZyBPUyBieSB3YXkgb2YgYSBoYXJkY29kZWQgY2xhc3MgdmFy aWFibGUgQ1VSUkVOVF9WRVJTSU9OLiBUaGUgcnVuLXdlYmtpdC10ZXN0cyBzY3JpcHQKKyAgICAg ICAgZGVwZW5kcyBvbiB0aGUgT1MgdmVyc2lvbiB0byBmaW5kIHRoZSBhcHBsaWNhYmxlIFRlc3RF eHBlY3RhdGlvbnMgZmlsZXMgdG8gcGFyc2UgdG8gY29tcHV0ZSB0aGUgbGlzdCBvZiBza2lwcGVk CisgICAgICAgIHRlc3RzIGZvciB0aGUgcG9ydC4KKworICAgICAgICAqIFNjcmlwdHMvd2Via2l0 cHkvcG9ydC9tYWMucHk6CisgICAgICAgIChNYWNQb3J0Ll9faW5pdF9fKTogRmFsbCBiYWNrIHRv IE1hY1BvcnQuQ1VSUkVOVF9WRVJTSU9OIGlmIHdlIGFyZSBub3QgdXNpbmcgYSBrbm93biBuYW1l ZCBtYWNPUyBhbmQgZG8gbm90CisgICAgICAgIGhhdmUgQXBwbGUgQWRkaXRpb25zLgorICAgICAg ICAqIFNjcmlwdHMvd2Via2l0cHkvcG9ydC9tYWNfdW5pdHRlc3QucHk6CisgICAgICAgIChNYWNU ZXN0LnRlc3RfZmFjdG9yeV93aXRoX2Z1dHVyZV92ZXJzaW9uKTogQWRkZWQuCisgICAgICAgIChN YWNUZXN0LnRlc3RfZmFjdG9yeV93aXRoX2Z1dHVyZV92ZXJzaW9uX2FuZF9hcHBsZV9hZGRpdGlv bnMpOiBSZW5hbWVkIGZyb20gTWFjVGVzdC50ZXN0X2ZhY3Rvcnlfd2l0aF9mdXR1cmVfdmVyc2lv bigpCisgICAgICAgIGFuZCBhZGRlZCBjb2RlIHRvIHVzZSB0aGUgbW9jayBBcHBsZSBBZGRpdGlv bnMgc28gdGhhdCBydW5uaW5nIHRoaXMgY29kZSB3aXRoLSBhbmQgd2l0aG91dC0gdGhlIHJlYWwg QXBwbGUgQWRkaXRpb25zCisgICAgICAgIHByb2R1Y2VzIGNvbnNpc3RlbnQgcmVzdWx0cyBub3cg dGhhdCB0aGUgTWFjIHBvcnQncyBPUyB2ZXJzaW9uIGZhbGxzIGJhY2sgdG8gTWFjUG9ydC5DVVJS RU5UX1ZFUlNJT04gd2hlbiBBcHBsZSBBZGRpdGlvbnMKKyAgICAgICAgaXMgYWJzZW50LgorCiAy MDE4LTA1LTA0ICBDYXJsb3MgR2FyY2lhIENhbXBvcyAgPGNnYXJjaWFAaWdhbGlhLmNvbT4KIAog ICAgICAgICBbR1RLXSBFcGlwaGFueSAoR05PTUUgV2ViKSBzYXlzICJFcnJvciBkb3dubG9hZGlu ZzogU2VydmljZSBVbmF2YWlsYWJsZS4iIHdoZW4gdHJ5aW5nIHRvIGRvd25sb2FkIGFuIGltYWdl IGZyb20gZGlzY29ncy5jb20KZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvcG9y dC9tYWMucHkgYi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3BvcnQvbWFjLnB5CmluZGV4IDcxMGE4 MTA0MTJmYTFlNTI5YzI0MTZlMWNjNGYxNjdiZTk1MjQ4MmMuLmYyZDc2MWU4MmVhMmQ4YWIyZDBm YjFiZTIyZTU5NTU0YjU3ZjUyMGMgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkv cG9ydC9tYWMucHkKKysrIGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wb3J0L21hYy5weQpAQCAt NjAsNyArNjAsNyBAQCBjbGFzcyBNYWNQb3J0KERhcndpblBvcnQpOgogICAgICAgICBzcGxpdF9w b3J0X25hbWUgPSBwb3J0X25hbWUuc3BsaXQoJy0nKQogICAgICAgICBpZiBsZW4oc3BsaXRfcG9y dF9uYW1lKSA+IDEgYW5kIHNwbGl0X3BvcnRfbmFtZVsxXSAhPSAnd2syJzoKICAgICAgICAgICAg IHNlbGYuX29zX3ZlcnNpb24gPSB2ZXJzaW9uX25hbWVfbWFwLmZyb21fbmFtZShzcGxpdF9wb3J0 X25hbWVbMV0pWzFdCi0gICAgICAgIGVsaWYgc2VsZi5ob3N0LnBsYXRmb3JtLmlzX21hYygpOgor ICAgICAgICBlbGlmIHNlbGYuaG9zdC5wbGF0Zm9ybS5pc19tYWMoKSBhbmQgYXBwbGVfYWRkaXRp b25zKCk6CiAgICAgICAgICAgICBzZWxmLl9vc192ZXJzaW9uID0gc2VsZi5ob3N0LnBsYXRmb3Jt Lm9zX3ZlcnNpb24KICAgICAgICAgaWYgbm90IHNlbGYuX29zX3ZlcnNpb246CiAgICAgICAgICAg ICBzZWxmLl9vc192ZXJzaW9uID0gTWFjUG9ydC5DVVJSRU5UX1ZFUlNJT04KZGlmZiAtLWdpdCBh L1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvcG9ydC9tYWNfdW5pdHRlc3QucHkgYi9Ub29scy9TY3Jp cHRzL3dlYmtpdHB5L3BvcnQvbWFjX3VuaXR0ZXN0LnB5CmluZGV4IDk3YmE1YzE3ODVlMDNjNGU1 OTAzZTRiZTBmNjg3NjMzZmU4NmMzMzkuLjhhMTQ2YjA1YmNmNzU0ZGFhNDVmODVkOWRlMjEzNzkx Mjc3OWZhMTYgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvcG9ydC9tYWNfdW5p dHRlc3QucHkKKysrIGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wb3J0L21hY191bml0dGVzdC5w eQpAQCAtMzQsNiArMzQsNyBAQCBmcm9tIHdlYmtpdHB5LmNvbW1vbi5zeXN0ZW0ub3V0cHV0Y2Fw dHVyZSBpbXBvcnQgT3V0cHV0Q2FwdHVyZQogZnJvbSB3ZWJraXRweS50b29sLm1vY2t0b29sIGlt cG9ydCBNb2NrT3B0aW9ucwogZnJvbSB3ZWJraXRweS5jb21tb24uc3lzdGVtLmV4ZWN1dGl2ZV9t b2NrIGltcG9ydCBNb2NrRXhlY3V0aXZlLCBNb2NrRXhlY3V0aXZlMiwgU2NyaXB0RXJyb3IKIGZy b20gd2Via2l0cHkuY29tbW9uLnZlcnNpb24gaW1wb3J0IFZlcnNpb24KK2Zyb20gd2Via2l0cHku Y29tbW9uLnZlcnNpb25fbmFtZV9tYXAgaW1wb3J0IFZlcnNpb25OYW1lTWFwCiAKIAogY2xhc3Mg TWFjVGVzdChkYXJ3aW5fdGVzdGNhc2UuRGFyd2luVGVzdCk6CkBAIC0xNTksMTkgKzE2MCwzNyBA QCBjbGFzcyBNYWNUZXN0KGRhcndpbl90ZXN0Y2FzZS5EYXJ3aW5UZXN0KToKICAgICBkZWYgdGVz dF9mYWN0b3J5X3dpdGhfZnV0dXJlX3ZlcnNpb24oc2VsZik6CiAgICAgICAgIHBvcnQgPSBzZWxm Lm1ha2VfcG9ydChvcHRpb25zPU1vY2tPcHRpb25zKHdlYmtpdF90ZXN0X3J1bm5lcj1UcnVlKSwg b3NfdmVyc2lvbj1NYWNUZXN0LkZVVFVSRV9WRVJTSU9OLCBvc19uYW1lPSdtYWMnLCBwb3J0X25h bWU9J21hYycpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwocG9ydC5kcml2ZXJfbmFtZSgpLCAn V2ViS2l0VGVzdFJ1bm5lcicpCi0gICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwocG9ydC52ZXJzaW9u X25hbWUoKSwgTm9uZSkKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChwb3J0LnZlcnNpb25fbmFt ZSgpLCBWZXJzaW9uTmFtZU1hcC5tYXAocG9ydC5ob3N0LnBsYXRmb3JtKS50b19uYW1lKE1hY1Bv cnQuQ1VSUkVOVF9WRVJTSU9OLCBwbGF0Zm9ybT1NYWNQb3J0LnBvcnRfbmFtZSkpCiAKICAgICAg ICAgcG9ydCA9IHNlbGYubWFrZV9wb3J0KG9wdGlvbnM9TW9ja09wdGlvbnMod2Via2l0X3Rlc3Rf cnVubmVyPUZhbHNlKSwgb3NfdmVyc2lvbj1NYWNUZXN0LkZVVFVSRV9WRVJTSU9OLCBvc19uYW1l PSdtYWMnLCBwb3J0X25hbWU9J21hYycpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwocG9ydC5k cml2ZXJfbmFtZSgpLCAnRHVtcFJlbmRlclRyZWUnKQotICAgICAgICBzZWxmLmFzc2VydEVxdWFs KHBvcnQudmVyc2lvbl9uYW1lKCksIE5vbmUpCisgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwocG9y dC52ZXJzaW9uX25hbWUoKSwgVmVyc2lvbk5hbWVNYXAubWFwKHBvcnQuaG9zdC5wbGF0Zm9ybSku dG9fbmFtZShNYWNQb3J0LkNVUlJFTlRfVkVSU0lPTiwgcGxhdGZvcm09TWFjUG9ydC5wb3J0X25h bWUpKQogCiAgICAgICAgIHBvcnQgPSBzZWxmLm1ha2VfcG9ydChvcHRpb25zPU1vY2tPcHRpb25z KHdlYmtpdF90ZXN0X3J1bm5lcj1GYWxzZSksIG9zX3ZlcnNpb249TWFjVGVzdC5GVVRVUkVfVkVS U0lPTiwgb3NfbmFtZT0nbWFjJywgcG9ydF9uYW1lPSdtYWMtd2syJykKICAgICAgICAgc2VsZi5h c3NlcnRFcXVhbChwb3J0LmRyaXZlcl9uYW1lKCksICdXZWJLaXRUZXN0UnVubmVyJykKLSAgICAg ICAgc2VsZi5hc3NlcnRFcXVhbChwb3J0LnZlcnNpb25fbmFtZSgpLCBOb25lKQorICAgICAgICBz ZWxmLmFzc2VydEVxdWFsKHBvcnQudmVyc2lvbl9uYW1lKCksIFZlcnNpb25OYW1lTWFwLm1hcChw b3J0Lmhvc3QucGxhdGZvcm0pLnRvX25hbWUoTWFjUG9ydC5DVVJSRU5UX1ZFUlNJT04sIHBsYXRm b3JtPU1hY1BvcnQucG9ydF9uYW1lKSkKIAogICAgICAgICBwb3J0ID0gc2VsZi5tYWtlX3BvcnQo b3B0aW9ucz1Nb2NrT3B0aW9ucyh3ZWJraXRfdGVzdF9ydW5uZXI9VHJ1ZSksIG9zX3ZlcnNpb249 TWFjVGVzdC5GVVRVUkVfVkVSU0lPTiwgb3NfbmFtZT0nbWFjJywgcG9ydF9uYW1lPSdtYWMtd2sy JykKICAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChwb3J0LmRyaXZlcl9uYW1lKCksICdXZWJLaXRU ZXN0UnVubmVyJykKLSAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChwb3J0LnZlcnNpb25fbmFtZSgp LCBOb25lKQorICAgICAgICBzZWxmLmFzc2VydEVxdWFsKHBvcnQudmVyc2lvbl9uYW1lKCksIFZl cnNpb25OYW1lTWFwLm1hcChwb3J0Lmhvc3QucGxhdGZvcm0pLnRvX25hbWUoTWFjUG9ydC5DVVJS RU5UX1ZFUlNJT04sIHBsYXRmb3JtPU1hY1BvcnQucG9ydF9uYW1lKSkKKworICAgIGRlZiB0ZXN0 X2ZhY3Rvcnlfd2l0aF9mdXR1cmVfdmVyc2lvbl9hbmRfYXBwbGVfYWRkaXRpb25zKHNlbGYpOgor ICAgICAgICB3aXRoIHBvcnRfdGVzdGNhc2UuYmluZF9tb2NrX2FwcGxlX2FkZGl0aW9ucygpOgor ICAgICAgICAgICAgcG9ydCA9IHNlbGYubWFrZV9wb3J0KG9wdGlvbnM9TW9ja09wdGlvbnMod2Vi a2l0X3Rlc3RfcnVubmVyPVRydWUpLCBvc192ZXJzaW9uPU1hY1Rlc3QuRlVUVVJFX1ZFUlNJT04s IG9zX25hbWU9J21hYycsIHBvcnRfbmFtZT0nbWFjJykKKyAgICAgICAgICAgIHNlbGYuYXNzZXJ0 RXF1YWwocG9ydC5kcml2ZXJfbmFtZSgpLCAnV2ViS2l0VGVzdFJ1bm5lcicpCisgICAgICAgICAg ICBzZWxmLmFzc2VydEVxdWFsKHBvcnQudmVyc2lvbl9uYW1lKCksIE5vbmUpCisKKyAgICAgICAg ICAgIHBvcnQgPSBzZWxmLm1ha2VfcG9ydChvcHRpb25zPU1vY2tPcHRpb25zKHdlYmtpdF90ZXN0 X3J1bm5lcj1GYWxzZSksIG9zX3ZlcnNpb249TWFjVGVzdC5GVVRVUkVfVkVSU0lPTiwgb3NfbmFt ZT0nbWFjJywgcG9ydF9uYW1lPSdtYWMnKQorICAgICAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChw b3J0LmRyaXZlcl9uYW1lKCksICdEdW1wUmVuZGVyVHJlZScpCisgICAgICAgICAgICBzZWxmLmFz c2VydEVxdWFsKHBvcnQudmVyc2lvbl9uYW1lKCksIE5vbmUpCisKKyAgICAgICAgICAgIHBvcnQg PSBzZWxmLm1ha2VfcG9ydChvcHRpb25zPU1vY2tPcHRpb25zKHdlYmtpdF90ZXN0X3J1bm5lcj1G YWxzZSksIG9zX3ZlcnNpb249TWFjVGVzdC5GVVRVUkVfVkVSU0lPTiwgb3NfbmFtZT0nbWFjJywg cG9ydF9uYW1lPSdtYWMtd2syJykKKyAgICAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwocG9ydC5k cml2ZXJfbmFtZSgpLCAnV2ViS2l0VGVzdFJ1bm5lcicpCisgICAgICAgICAgICBzZWxmLmFzc2Vy dEVxdWFsKHBvcnQudmVyc2lvbl9uYW1lKCksIE5vbmUpCisKKyAgICAgICAgICAgIHBvcnQgPSBz ZWxmLm1ha2VfcG9ydChvcHRpb25zPU1vY2tPcHRpb25zKHdlYmtpdF90ZXN0X3J1bm5lcj1UcnVl KSwgb3NfdmVyc2lvbj1NYWNUZXN0LkZVVFVSRV9WRVJTSU9OLCBvc19uYW1lPSdtYWMnLCBwb3J0 X25hbWU9J21hYy13azInKQorICAgICAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChwb3J0LmRyaXZl cl9uYW1lKCksICdXZWJLaXRUZXN0UnVubmVyJykKKyAgICAgICAgICAgIHNlbGYuYXNzZXJ0RXF1 YWwocG9ydC52ZXJzaW9uX25hbWUoKSwgTm9uZSkKIAogICAgIGRlZiB0ZXN0X2ZhY3Rvcnlfd2l0 aF9wb3J0bmFtZV92ZXJzaW9uKHNlbGYpOgogICAgICAgICBwb3J0ID0gc2VsZi5tYWtlX3BvcnQo b3B0aW9ucz1Nb2NrT3B0aW9ucyh3ZWJraXRfdGVzdF9ydW5uZXI9RmFsc2UpLCBwb3J0X25hbWU9 J21hYy1tb3VudGFpbmxpb24nKQo=