143090 2015-03-26 03:52:02 -0700 run-webkit-tests should have an option to run new tests 2015-03-27 10:43:20 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED WONTFIX P2 Normal --- 118294 1 youennf youennf ap commit-queue glenn ossy oldest_to_newest 1080252 0 youennf 2015-03-26 03:52:02 -0700 It would be nice to run all tests modified/created through run-webkit-tests. That may help developers as well as provide limited EWS bots the ability to check a small but critical amount of tests. 1080256 1 249482 youennf 2015-03-26 04:28:18 -0700 Created attachment 249482 Patch 1080263 2 youennf 2015-03-26 04:41:17 -0700 Uploaded a patch at bug 143093 to show a potential use of --new-tests and --pass-if-no-test options. 1080307 3 249482 ap 2015-03-26 10:00:46 -0700 Comment on attachment 249482 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=249482&action=review > Tools/ChangeLog:8 > + Add a --new-tests option to run-webkit-tests which adds tests that have been changed/created compared to the repository. Can't we figure that out outside run-webkit-tests? I'm not sure if this is a mode that engineers need to have access to. EWS can just pass the list of new tests if any. 1080357 4 ossy 2015-03-26 11:45:06 -0700 (In reply to comment #3) > Comment on attachment 249482 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=249482&action=review > > > Tools/ChangeLog:8 > > + Add a --new-tests option to run-webkit-tests which adds tests that have been changed/created compared to the repository. > > Can't we figure that out outside run-webkit-tests? I'm not sure if this is a > mode that engineers need to have access to. > > EWS can just pass the list of new tests if any. EWS can pass list of the tests to run-webkit-tests as command line arguments directly and in a file with --test-list option. But I'm not sure how are test expectations handled in this cases. AFAIR skipped tests are run too, but I'm not sure. 1080366 5 youennf 2015-03-26 12:01:01 -0700 (In reply to comment #4) > (In reply to comment #3) > > Comment on attachment 249482 [details] > > Patch > > > > View in context: > > https://bugs.webkit.org/attachment.cgi?id=249482&action=review > > > > > Tools/ChangeLog:8 > > > + Add a --new-tests option to run-webkit-tests which adds tests that have been changed/created compared to the repository. > > > > Can't we figure that out outside run-webkit-tests? I'm not sure if this is a > > mode that engineers need to have access to. > > > > EWS can just pass the list of new tests if any. Combined with git, this option runs all tests added/modified by the set of commits compared to the remote origin master. Handy for branch development, rebasing -i... It might also be useful when reimporting W3C tests. > EWS can pass list of the tests to run-webkit-tests as command > line arguments directly and in a file with --test-list option. > > But I'm not sure how are test expectations handled in this > cases. AFAIR skipped tests are run too, but I'm not sure. Good point. It should be checked as part of integration/unit tests that should be added. 1080384 6 youennf 2015-03-26 13:08:25 -0700 > > EWS can pass list of the tests to run-webkit-tests as command > > line arguments directly and in a file with --test-list option. > > > > But I'm not sure how are test expectations handled in this > > cases. AFAIR skipped tests are run too, but I'm not sure. > > Good point. > It should be checked as part of integration/unit tests that should be added. There is the --skipped=always/default option which enables to run/skip the tests here. It all ends up whether we want to expose this option in rwt or not... I see some value to it as exposed previously. The main downsides I see with that option is that there is the need to handle within rwt the case of runs with no tests. This patch introduces the --pass-if-no-option for that purpose. But this case would probably be best handled within EWS. Thoughts? 1080413 7 ap 2015-03-26 14:47:58 -0700 Another complication is layering. Currently, run-webkit-tests knows nothing about version control, and it would be confusing if it started to use that just for one command. If enough people want a special mode to run just the new/modified tests, webkit-patch is unclean enough to have that added too, I think, but probably not run-webkit-tests. 1080640 8 youennf 2015-03-27 10:27:45 -0700 (In reply to comment #7) > Another complication is layering. Currently, run-webkit-tests knows nothing > about version control, and it would be confusing if it started to use that > just for one command. > > If enough people want a special mode to run just the new/modified tests, > webkit-patch is unclean enough to have that added too, I think, but probably > not run-webkit-tests. Then we should directly go to bug 143093... 249482 2015-03-26 04:28:18 -0700 2015-03-27 10:27:57 -0700 Patch bug-143090-20150326122739.patch text/plain 7917 youennf U3VidmVyc2lvbiBSZXZpc2lvbjogMTgyMDAxCmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggOWIyODRlYjFmOGY4M2FkMzI0MTllMjIwNTJhMjZlMzMw OTU2M2IxMS4uN2Q5OGE3MGNjNDQ2Yjc3ZDEzZGJiZjA1NTc2NzdjMmQ0MDg3NWZmNCAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDI3 IEBACisyMDE1LTAzLTI2ICBZb3Vlbm4gRmFibGV0ICA8eW91ZW5uLmZhYmxldEBjcmYuY2Fub24u ZnI+CisKKyAgICAgICAgcnVuLXdlYmtpdC10ZXN0cyBzaG91bGQgaGF2ZSBhbiBvcHRpb24gdG8g cnVuIG5ldyB0ZXN0cworICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5j Z2k/aWQ9MTQzMDkwCisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAg ICAgICAgQWRkIGEgLS1uZXctdGVzdHMgb3B0aW9uIHRvIHJ1bi13ZWJraXQtdGVzdHMgd2hpY2gg YWRkcyB0ZXN0cyB0aGF0IGhhdmUgYmVlbiBjaGFuZ2VkL2NyZWF0ZWQgY29tcGFyZWQgdG8gdGhl IHJlcG9zaXRvcnkuCisgICAgICAgIEFkZCBhIC0tcGFzcy1pZi1uby10ZXN0IG9wdGlvbiB0byBs ZXQgcnVuLXdlYmtpdC10ZXN0cyBleGl0IG5pY2VseSBpbiBjYXNlIG5vIHRlc3QgaXMgcnVuLAor ICAgICAgICB3aGljaCBtYXkgaGFwcGVuIHdoZW4gb25seSBuZXcgdGVzdHMgYXJlIGV4cGVjdGVk IHRvIGJlIHJ1bi4KKworICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2Nv bnRyb2xsZXJzL2xheW91dF90ZXN0X2ZpbmRlci5weToKKyAgICAgICAgKExheW91dFRlc3RGaW5k ZXIuZmluZF90ZXN0cyk6IFNlYXJjaGVzIGZvciBjaGFuZ2VkIHRlc3RzIGlmIG5ld190ZXN0cyBv cHRpb24gaXMgc2V0LgorICAgICAgICBBZGRpbmcgYSBkdW1teSBmb2xkZXIgaW4gY2FzZSBubyBw YXRoIGlzIGZvdW5kIHRvIGVuc3VyZSBub3QgYWxsIHRlc3RzIGFyZSBmb3VuZCBieSAgcG9ydC50 ZXN0cygpLgorICAgICAgICAoTGF5b3V0VGVzdEZpbmRlci5fcmVhZF9wb3RlbnRpYWxfdGVzdF9u YW1lc19mcm9tX2NoYW5nZWRfZmlsZXMpOiBDYWxscyBzY20gdG8gaWRlbnRpZnkgbW9kaWZpZWQg ZmlsZXMuCisgICAgICAgIFNwZWNpYWwgaGFuZGxpbmcgb2YgLWV4cGVjdGVkIGZpbGVzIHRvIGlu c2VydCB0aGVpciByZWxhdGVkIHRlc3RzLgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5 b3V0X3Rlc3RzL2NvbnRyb2xsZXJzL21hbmFnZXIucHk6CisgICAgICAgIChNYW5hZ2VyLnJ1bik6 IElmIG9wdGlvbiBwYXNzX2lmX25vX3Rlc3QgaXMgc2V0LCBkbyBub3QgcmFpc2UgYW4gZXJyb3Ig aWYgbm8gdGVzdCBpcyBmb3VuZC4KKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90 ZXN0cy9ydW5fd2Via2l0X3Rlc3RzLnB5OgorICAgICAgICAobWFpbik6IERvIG5vdCBwcmludCBy ZXN1bHRzIGlmIG5vIHRlc3QgZm91bmQuCisgICAgICAgIChwYXJzZV9hcmdzKTogQWRkIC0tbmV3 LXRlc3RzIGFuZCAtLXBhc3MtaWYtbm8tdGVzdCBvcHRpb25zLgorICAgICAgICAqIFNjcmlwdHMv d2Via2l0cHkvbGF5b3V0X3Rlc3RzL3J1bl93ZWJraXRfdGVzdHNfaW50ZWdyYXRpb250ZXN0LnB5 OgorICAgICAgICAoUnVuVGVzdC50ZXN0X25vX3Rlc3RzX2ZvdW5kX29rKTogQ2hlY2sgLS1wYXNz LWlmLW5vLXRlc3Qgb3B0aW9uLgorCiAyMDE1LTAzLTI1ICBKb3NlcGggUGVjb3Jhcm8gIDxwZWNv cmFyb0BhcHBsZS5jb20+CiAKICAgICAgICAgVXBkYXRlIHByZXBhcmUtQ2hhbmdlTG9nIHRvIHdv cmsgd2l0aCBFUzYgQ2xhc3Mgc3ludGF4CmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRzL3dlYmtp dHB5L2xheW91dF90ZXN0cy9jb250cm9sbGVycy9sYXlvdXRfdGVzdF9maW5kZXIucHkgYi9Ub29s cy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9jb250cm9sbGVycy9sYXlvdXRfdGVzdF9m aW5kZXIucHkKaW5kZXggN2ZmZmQ3ZTViMDYwM2NjOTM1NDVhYzAzYzlkZDNiNDRjMzA0Y2NkZC4u NzIwZDBiNzgyMTA0M2Y5ZTAyM2RkYWM4MGMwZmE2M2ZjMjg1ZWJhMCAxMDA2NDQKLS0tIGEvVG9v bHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvY29udHJvbGxlcnMvbGF5b3V0X3Rlc3Rf ZmluZGVyLnB5CisrKyBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2NvbnRy b2xsZXJzL2xheW91dF90ZXN0X2ZpbmRlci5weQpAQCAtMzAsNiArMzAsNyBAQCBpbXBvcnQgZXJy bm8KIGltcG9ydCBsb2dnaW5nCiBpbXBvcnQgcmUKIAorZnJvbSB3ZWJraXRweS5jb21tb24uY2hl Y2tvdXQuc2NtLmRldGVjdGlvbiBpbXBvcnQgU0NNRGV0ZWN0b3IKIGZyb20gd2Via2l0cHkubGF5 b3V0X3Rlc3RzLm1vZGVscyBpbXBvcnQgdGVzdF9leHBlY3RhdGlvbnMKIAogCkBAIC00Nyw2ICs0 OCwxMSBAQCBjbGFzcyBMYXlvdXRUZXN0RmluZGVyKG9iamVjdCk6CiAgICAgICAgIHBhdGhzID0g c2VsZi5fc3RyaXBfdGVzdF9kaXJfcHJlZml4ZXMoYXJncykKICAgICAgICAgaWYgb3B0aW9ucy50 ZXN0X2xpc3Q6CiAgICAgICAgICAgICBwYXRocyArPSBzZWxmLl9zdHJpcF90ZXN0X2Rpcl9wcmVm aXhlcyhzZWxmLl9yZWFkX3Rlc3RfbmFtZXNfZnJvbV9maWxlKG9wdGlvbnMudGVzdF9saXN0LCBz ZWxmLl9wb3J0LlRFU1RfUEFUSF9TRVBBUkFUT1IpKQorICAgICAgICBpZiBvcHRpb25zLm5ld190 ZXN0czoKKyAgICAgICAgICAgIHBhdGhzICs9IHNlbGYuX3N0cmlwX3Rlc3RfZGlyX3ByZWZpeGVz KHNlbGYuX3JlYWRfcG90ZW50aWFsX3Rlc3RfbmFtZXNfZnJvbV9jaGFuZ2VkX2ZpbGVzKCkpCisg ICAgICAgICAgICBpZiBub3QgcGF0aHM6CisgICAgICAgICAgICAgICAgIyBBZGQgdGhpcyB0byBl bnN1cmUgcGF0aCBpcyBub3QgZW1wdHkgYW5kIHRoYXQgd2Ugd2lsbCBub3QgZ2F0aGVyIGFsbCB0 ZXN0cy4KKyAgICAgICAgICAgICAgICBwYXRocyArPSAnLmR1bW15X2RpcmVjdG9yeScKICAgICAg ICAgdGVzdF9maWxlcyA9IHNlbGYuX3BvcnQudGVzdHMocGF0aHMpCiAgICAgICAgIHJldHVybiAo cGF0aHMsIHRlc3RfZmlsZXMpCiAKQEAgLTgxLDYgKzg3LDI3IEBAIGNsYXNzIExheW91dFRlc3RG aW5kZXIob2JqZWN0KToKICAgICAgICAgICAgICAgICByYWlzZQogICAgICAgICByZXR1cm4gdGVz dHMKIAorICAgIGRlZiBfcmVhZF9wb3RlbnRpYWxfdGVzdF9uYW1lc19mcm9tX2NoYW5nZWRfZmls ZXMoc2VsZik6CisgICAgICAgIHNjbSA9IFNDTURldGVjdG9yKHNlbGYuX3BvcnQuaG9zdC5maWxl c3lzdGVtLCBzZWxmLl9wb3J0Lmhvc3QuZXhlY3V0aXZlKS5kZXRlY3Rfc2NtX3N5c3RlbShzZWxm Ll9wb3J0LndlYmtpdF9iYXNlKCkpIG9yIHNlbGYuX3BvcnQuaG9zdC5zY20oKQorICAgICAgICBw b3RlbnRpYWxfdGVzdF9uYW1lcyA9IFtdCisgICAgICAgIGZvciB0ZXN0X2ZpbGUgaW4gc2NtLmNo YW5nZWRfZmlsZXMoKToKKyAgICAgICAgICAgIGlmIHRlc3RfZmlsZSBpbiBwb3RlbnRpYWxfdGVz dF9uYW1lczoKKyAgICAgICAgICAgICAgICBjb250aW51ZQorICAgICAgICAgICAgICMgRklYTUU6 IFdlIHNob3VsZCB0cnkgdG8gZGV0ZWN0ICdyZXNvdXJjZXMnIGZpbGVzIGFuZCBhcHBlbmQgcmVs YXRlZCB0ZXN0IGZpbGVzLgorICAgICAgICAgICAgaWYgJy1leHBlY3RlZC4nIGluIHRlc3RfZmls ZToKKyAgICAgICAgICAgICAgICBmdWxscGF0aCA9IHNlbGYuX2ZpbGVzeXN0ZW0uam9pbihzZWxm Ll9wb3J0LndlYmtpdF9iYXNlKCksIHRlc3RfZmlsZSkKKyAgICAgICAgICAgICAgICBpZiAncGxh dGZvcm0nIGluIHRlc3RfZmlsZToKKyAgICAgICAgICAgICAgICAgICAgc3RlcHMgPSB0ZXN0X2Zp bGUuc3BsaXQoc2VsZi5fZmlsZXN5c3RlbS5zZXApCisgICAgICAgICAgICAgICAgICAgIGZ1bGxw YXRoID0gc2VsZi5fZmlsZXN5c3RlbS5qb2luKHNlbGYuX3BvcnQud2Via2l0X2Jhc2UoKSwgJ0xh eW91dFRlc3RzJywgc2VsZi5fZmlsZXN5c3RlbS5zZXAuam9pbihzdGVwc1szOl0pKQorCisgICAg ICAgICAgICAgICAgZm9yIHBvdGVudGlhbF9maWxlIGluIHNlbGYuX2ZpbGVzeXN0ZW0uZ2xvYihy ZS5zdWIoJy1leHBlY3RlZC4qJywgJy4qJywgZnVsbHBhdGgpKToKKyAgICAgICAgICAgICAgICAg ICAgcG90ZW50aWFsX3Rlc3QgPSBwb3RlbnRpYWxfZmlsZVtsZW4oc2VsZi5fcG9ydC53ZWJraXRf YmFzZSgpKSArIDE6XQorICAgICAgICAgICAgICAgICAgICBpZiBub3QgcG90ZW50aWFsX3Rlc3Qg aW4gcG90ZW50aWFsX3Rlc3RfbmFtZXM6CisgICAgICAgICAgICAgICAgICAgICAgICBwb3RlbnRp YWxfdGVzdF9uYW1lcy5hcHBlbmQocG90ZW50aWFsX3Rlc3QpCisgICAgICAgICAgICAgICAgY29u dGludWUKKyAgICAgICAgICAgIHBvdGVudGlhbF90ZXN0X25hbWVzLmFwcGVuZCh0ZXN0X2ZpbGUp CisgICAgICAgIHJldHVybiBwb3RlbnRpYWxfdGVzdF9uYW1lcworCiAgICAgQHN0YXRpY21ldGhv ZAogICAgIGRlZiBfc3RyaXBfY29tbWVudHMobGluZSk6CiAgICAgICAgIGNvbW1lbnRJbmRleCA9 IGxpbmUuZmluZCgnLy8nKQpkaWZmIC0tZ2l0IGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlv dXRfdGVzdHMvY29udHJvbGxlcnMvbWFuYWdlci5weSBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkv bGF5b3V0X3Rlc3RzL2NvbnRyb2xsZXJzL21hbmFnZXIucHkKaW5kZXggZmRiYTJiZWNmMDNhZTY3 YWQzNjBiMWZiMTI3MmQ3M2ZlZGI3NDA5ZC4uZWEwMTc1NmUyMGViZWIyMmRmY2Y0NjRiOWNiMDQ0 ODIzMGM1MTU0ZCAxMDA2NDQKLS0tIGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVz dHMvY29udHJvbGxlcnMvbWFuYWdlci5weQorKysgYi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2xh eW91dF90ZXN0cy9jb250cm9sbGVycy9tYW5hZ2VyLnB5CkBAIC0xODAsNiArMTgwLDggQEAgY2xh c3MgTWFuYWdlcihvYmplY3QpOgogCiAgICAgICAgICMgQ2hlY2sgdG8gbWFrZSBzdXJlIHdlJ3Jl IG5vdCBza2lwcGluZyBldmVyeSB0ZXN0LgogICAgICAgICBpZiBub3QgdGVzdHNfdG9fcnVuOgor ICAgICAgICAgICAgaWYgc2VsZi5fb3B0aW9ucy5wYXNzX2lmX25vX3Rlc3Q6CisgICAgICAgICAg ICAgICAgcmV0dXJuIHRlc3RfcnVuX3Jlc3VsdHMuUnVuRGV0YWlscyhleGl0X2NvZGU9MCkKICAg ICAgICAgICAgIF9sb2cuY3JpdGljYWwoJ05vIHRlc3RzIHRvIHJ1bi4nKQogICAgICAgICAgICAg cmV0dXJuIHRlc3RfcnVuX3Jlc3VsdHMuUnVuRGV0YWlscyhleGl0X2NvZGU9LTEpCiAKZGlmZiAt LWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3J1bl93ZWJraXRfdGVz dHMucHkgYi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9ydW5fd2Via2l0X3Rl c3RzLnB5CmluZGV4IGY2OWEyNmViNjQxMjYyMjRlOGEzMTBkNmZhOWE5ZWY0NzYwNjEyODMuLjFk NGYzODhhMjY1MTdlMDUyOWUzOTU0OThlYjJlNzIxZmNhZTY3NzAgMTAwNzU1Ci0tLSBhL1Rvb2xz L1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3J1bl93ZWJraXRfdGVzdHMucHkKKysrIGIv VG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcnVuX3dlYmtpdF90ZXN0cy5weQpA QCAtNzgsNyArNzgsNyBAQCBkZWYgbWFpbihhcmd2LCBzdGRvdXQsIHN0ZGVycik6CiAKICAgICB0 cnk6CiAgICAgICAgIHJ1bl9kZXRhaWxzID0gcnVuKHBvcnQsIG9wdGlvbnMsIGFyZ3MsIHN0ZGVy cikKLSAgICAgICAgaWYgcnVuX2RldGFpbHMuZXhpdF9jb2RlICE9IC0xOgorICAgICAgICBpZiBy dW5fZGV0YWlscy5leGl0X2NvZGUgIT0gLTEgYW5kIHJ1bl9kZXRhaWxzLnN1bW1hcml6ZWRfcmVz dWx0czoKICAgICAgICAgICAgIGJvdF9wcmludGVyID0gYnVpbGRib3RfcmVzdWx0cy5CdWlsZEJv dFByaW50ZXIoc3Rkb3V0LCBvcHRpb25zLmRlYnVnX3J3dF9sb2dnaW5nKQogICAgICAgICAgICAg Ym90X3ByaW50ZXIucHJpbnRfcmVzdWx0cyhydW5fZGV0YWlscykKIApAQCAtMjI5LDYgKzIyOSwx MCBAQCBkZWYgcGFyc2VfYXJncyhhcmdzKToKICAgICAgICAgICAgICAgICAgInJ1bm5pbmcuIChF eGFtcGxlOiAtLXdyYXBwZXI9J3ZhbGdyaW5kIC0tc21jLWNoZWNrPWFsbCcpIiksCiAgICAgICAg IG9wdHBhcnNlLm1ha2Vfb3B0aW9uKCItaSIsICItLWlnbm9yZS10ZXN0cyIsIGFjdGlvbj0iYXBw ZW5kIiwgZGVmYXVsdD1bXSwKICAgICAgICAgICAgIGhlbHA9ImRpcmVjdG9yaWVzIG9yIHRlc3Qg dG8gaWdub3JlIChtYXkgc3BlY2lmeSBtdWx0aXBsZSB0aW1lcykiKSwKKyAgICAgICAgb3B0cGFy c2UubWFrZV9vcHRpb24oIi0tbmV3LXRlc3RzIiwgYWN0aW9uPSJzdG9yZV90cnVlIiwgZGVmYXVs dD1GYWxzZSwKKyAgICAgICAgICAgIGhlbHA9InJ1biBvbmx5IHRoZSB0ZXN0cyBtb2RpZmllZCBv ciBjcmVhdGVkIGNvbXBhcmVkIHRvIHRoZSBiYXNlIHJlcG9zaXRvcnkuIiksCisgICAgICAgIG9w dHBhcnNlLm1ha2Vfb3B0aW9uKCItLXBhc3MtaWYtbm8tdGVzdCIsIGFjdGlvbj0ic3RvcmVfdHJ1 ZSIsIGRlZmF1bHQ9RmFsc2UsCisgICAgICAgICAgICBoZWxwPSJkbyBub3QgcmFpc2UgYW4gZXJy b3IgaWYgbm8gdGVzdCBpcyBydW4uIFRoaXMgbWF5IGJlIHVzZWQgd2l0aCB0aGUgLS1uZXctdGVz dHMgb3B0aW9uLiIpLAogICAgICAgICBvcHRwYXJzZS5tYWtlX29wdGlvbigiLS10ZXN0LWxpc3Qi LCBhY3Rpb249ImFwcGVuZCIsCiAgICAgICAgICAgICBoZWxwPSJyZWFkIGxpc3Qgb2YgdGVzdHMg dG8gcnVuIGZyb20gZmlsZSIsIG1ldGF2YXI9IkZJTEUiKSwKICAgICAgICAgb3B0cGFyc2UubWFr ZV9vcHRpb24oIi0tc2tpcHBlZCIsIGFjdGlvbj0ic3RvcmUiLCBkZWZhdWx0PSJkZWZhdWx0IiwK ZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3J1bl93ZWJr aXRfdGVzdHNfaW50ZWdyYXRpb250ZXN0LnB5IGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlv dXRfdGVzdHMvcnVuX3dlYmtpdF90ZXN0c19pbnRlZ3JhdGlvbnRlc3QucHkKaW5kZXggZTlmZDAz ZmJjNjZjNWVjNjhiOTI4YjgxM2FmMjlhNTc2NzNlYmI2NC4uYzY3NDY3YjRhMDIwOGZkNjA4ODRk ODg4NzY4MzdkOTJhYjIwNTQ2MyAxMDA2NDQKLS0tIGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9s YXlvdXRfdGVzdHMvcnVuX3dlYmtpdF90ZXN0c19pbnRlZ3JhdGlvbnRlc3QucHkKKysrIGIvVG9v bHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcnVuX3dlYmtpdF90ZXN0c19pbnRlZ3Jh dGlvbnRlc3QucHkKQEAgLTI5Miw2ICsyOTIsMTAgQEAgY2xhc3MgUnVuVGVzdCh1bml0dGVzdC5U ZXN0Q2FzZSwgU3RyZWFtVGVzdGluZ01peGluKToKICAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChk ZXRhaWxzLmV4aXRfY29kZSwgLTEpCiAgICAgICAgIHNlbGYuYXNzZXJ0Q29udGFpbnMoZXJyLCAn Tm8gdGVzdHMgdG8gcnVuLlxuJykKIAorICAgIGRlZiB0ZXN0X25vX3Rlc3RzX2ZvdW5kX29rKHNl bGYpOgorICAgICAgICBkZXRhaWxzLCBlcnIsIF8gPSBsb2dnaW5nX3J1bihbJ3Jlc291cmNlcycs ICctLXBhc3MtaWYtbm8tdGVzdCddLCB0ZXN0c19pbmNsdWRlZD1UcnVlKQorICAgICAgICBzZWxm LmFzc2VydEVxdWFsKGRldGFpbHMuZXhpdF9jb2RlLCAwKQorCiAgICAgZGVmIHRlc3RfbmF0dXJh bF9vcmRlcihzZWxmKToKICAgICAgICAgdGVzdHNfdG9fcnVuID0gWydwYXNzZXMvYXVkaW8uaHRt bCcsICdmYWlsdXJlcy9leHBlY3RlZC90ZXh0Lmh0bWwnLCAnZmFpbHVyZXMvZXhwZWN0ZWQvbWlz c2luZ190ZXh0Lmh0bWwnLCAncGFzc2VzL2FyZ3MuaHRtbCddCiAgICAgICAgIHRlc3RzX3J1biA9 IGdldF90ZXN0c19ydW4oWyctLW9yZGVyPW5hdHVyYWwnXSArIHRlc3RzX3RvX3J1bikK