105219 2012-12-17 14:58:23 -0800 Running a skipped test with run-perf-tests could alert the user 2012-12-18 13:16:18 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 77037 1 pdr rniwa abarth dpranke eric rniwa webkit.review.bot oldest_to_newest 793060 0 pdr 2012-12-17 14:58:23 -0800 Today I ran a skipped perf test like so: run-perf-tests [skipped test] Running 0 tests It would be great if this ran the skipped test anyway, or alerted the user that the test would not be run. 793062 1 rniwa 2012-12-17 15:06:56 -0800 (In reply to comment #0) > Today I ran a skipped perf test like so: > run-perf-tests [skipped test] > Running 0 tests > > It would be great if this ran the skipped test anyway, or alerted the user that the test would not be run. Use --force? 793075 2 pdr 2012-12-17 15:20:21 -0800 (In reply to comment #1) > (In reply to comment #0) > > Today I ran a skipped perf test like so: > > run-perf-tests [skipped test] > > Running 0 tests > > > > It would be great if this ran the skipped test anyway, or alerted the user that the test would not be run. > > Use --force? The issue is simply that the user may not realize the test was skipped. 793078 3 eric 2012-12-17 15:22:47 -0800 Yeah, I'm sure there are ways around it. It was just very confusing. :) It gave no info. He was using: %run-perf-tests SVG/GearFlowers.html Running 0 tests And that's literally all the output. :) I guess in general we should validate the arguments and explain better to the user what's happening! If you pass non/existent/path you get the same confusion. 793079 4 eric 2012-12-17 15:23:37 -0800 This isn't a critical bug, just something we should be aware of. :) 793421 5 179892 rniwa 2012-12-18 00:06:50 -0800 Created attachment 179892 Fixes the bug 793423 6 179894 rniwa 2012-12-18 00:08:44 -0800 Created attachment 179894 Fix a minor typo 793826 7 179894 eric 2012-12-18 10:52:01 -0800 Comment on attachment 179894 Fix a minor typo Lgtm 793854 8 179894 webkit.review.bot 2012-12-18 11:16:29 -0800 Comment on attachment 179894 Fix a minor typo Clearing flags on attachment: 179894 Committed r138045: <http://trac.webkit.org/changeset/138045> 793855 9 webkit.review.bot 2012-12-18 11:16:33 -0800 All reviewed patches have been landed. Closing bug. 793877 10 179894 dpranke 2012-12-18 11:33:07 -0800 Comment on attachment 179894 Fix a minor typo View in context: https://bugs.webkit.org/attachment.cgi?id=179894&action=review > Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py:158 > + if self._options.use_skipped_list and self._port.skips_perf_test(relative_path) and filesystem.normpath(path) in paths: I'm feeling stupid today, so I'm probably missing something, but why do you want the 'filesystem.normpath(path) in paths' part? won't that make you skip things that you listed on the command line? 793881 11 179894 rniwa 2012-12-18 11:36:37 -0800 Comment on attachment 179894 Fix a minor typo View in context: https://bugs.webkit.org/attachment.cgi?id=179894&action=review >> Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py:158 >> + if self._options.use_skipped_list and self._port.skips_perf_test(relative_path) and filesystem.normpath(path) in paths: > > I'm feeling stupid today, so I'm probably missing something, but why do you want the 'filesystem.normpath(path) in paths' part? won't that make you skip things that you listed on the command line? Huh, that seems right. I wonder why none of the tests caught that. 793968 12 rniwa 2012-12-18 13:08:25 -0800 Fixed in http://trac.webkit.org/changeset/138058. This was a pretty bad regression. It caused all perf. tests regardless of whether they're skipped or not are ran :( I've cancelled all perf bot jobs up until r138058. 793973 13 dpranke 2012-12-18 13:12:19 -0800 (In reply to comment #12) > Fixed in http://trac.webkit.org/changeset/138058. This was a pretty bad regression. It caused all perf. tests regardless of whether they're skipped or not are ran :( I've cancelled all perf bot jobs up until r138058. I'm a bit confused by that change ... you mean that it caused all the perf tests to be skipped, right? Also, Did changing the unit test name actually affect anything? 793977 14 rniwa 2012-12-18 13:16:18 -0800 (In reply to comment #13) > I'm a bit confused by that change ... you mean that it caused all the perf tests to be skipped, right? Also, Did changing the unit test name actually affect anything? No. It meant that Skipped was completely ignored because the last condition was always false. Renaming the unit test name was necessarily because there was another unit test of the same, which actually tested that skipped list worked. This particular test was about --foce, so skipped list was ignored there. As a result, we didn't have any unit test for skip list :( 179892 2012-12-18 00:06:50 -0800 2012-12-18 00:08:44 -0800 Fixes the bug fix105219 text/plain 5762 rniwa SW5kZXg6IFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvYmFzZS5weQo9 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09Ci0tLSBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L2Jh c2UucHkJKHJldmlzaW9uIDEzNzk3NikKKysrIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0 X3Rlc3RzL3BvcnQvYmFzZS5weQkod29ya2luZyBjb3B5KQpAQCAtODI4LDEyICs4MjgsNiBAQAog ICAgICAgICBlbHNlOgogICAgICAgICAgICAgcmV0dXJuIHNlbGYuaG9zdC5maWxlc3lzdGVtLmFi c3BhdGgoZmlsZW5hbWUpCiAKLSAgICBkZWYgcmVsYXRpdmVfcGVyZl90ZXN0X2ZpbGVuYW1lKHNl bGYsIGZpbGVuYW1lKToKLSAgICAgICAgaWYgZmlsZW5hbWUuc3RhcnRzd2l0aChzZWxmLnBlcmZf dGVzdHNfZGlyKCkpOgotICAgICAgICAgICAgcmV0dXJuIHNlbGYuaG9zdC5maWxlc3lzdGVtLnJl bHBhdGgoZmlsZW5hbWUsIHNlbGYucGVyZl90ZXN0c19kaXIoKSkKLSAgICAgICAgZWxzZToKLSAg ICAgICAgICAgIHJldHVybiBzZWxmLmhvc3QuZmlsZXN5c3RlbS5hYnNwYXRoKGZpbGVuYW1lKQot CiAgICAgQG1lbW9pemVkCiAgICAgZGVmIGFic3BhdGhfZm9yX3Rlc3Qoc2VsZiwgdGVzdF9uYW1l KToKICAgICAgICAgIiIiUmV0dXJucyB0aGUgZnVsbCBwYXRoIHRvIHRoZSBmaWxlIGZvciBhIGdp dmVuIHRlc3QgbmFtZS4gVGhpcyBpcyB0aGUKSW5kZXg6IFRvb2xzL1NjcmlwdHMvd2Via2l0cHkv cGVyZm9ybWFuY2VfdGVzdHMvcGVyZnRlc3RzcnVubmVyX3VuaXR0ZXN0LnB5Cj09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0K LS0tIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvcGVyZm9ybWFuY2VfdGVzdHMvcGVyZnRlc3RzcnVu bmVyX3VuaXR0ZXN0LnB5CShyZXZpc2lvbiAxMzc5NzYpCisrKyBUb29scy9TY3JpcHRzL3dlYmtp dHB5L3BlcmZvcm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1bm5lcl91bml0dGVzdC5weQkod29ya2lu ZyBjb3B5KQpAQCAtNTg4LDYgKzU4OCwxOSBAQAogICAgICAgICBwb3J0LnNraXBwZWRfcGVyZl90 ZXN0cyA9IGxhbWJkYTogWydpbnNwZWN0b3IvdW5zdXBwb3J0ZWRfdGVzdDEuaHRtbCcsICd1bnN1 cHBvcnRlZCddCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwoc2VsZi5fY29sbGVjdF90ZXN0c19h bmRfc29ydF90ZXN0X25hbWUocnVubmVyKSwgWydpbnNwZWN0b3IvdGVzdDEuaHRtbCcsICdpbnNw ZWN0b3IvdGVzdDIuaHRtbCddKQogCisgICAgZGVmIHRlc3RfY29sbGVjdF90ZXN0c193aXRoX3Nr aXBwZWRfbGlzdF9hbmRfZmlsZXMoc2VsZik6CisgICAgICAgIHJ1bm5lciwgcG9ydCA9IHNlbGYu Y3JlYXRlX3J1bm5lcihhcmdzPVsnU3VpdGUvVGVzdDEuaHRtbCcsICdTdWl0ZS9Ta2lwcGVkVGVz dDEuaHRtbCcsICdTa2lwcGVkU3VpdGUvVGVzdDEuaHRtbCddKQorCisgICAgICAgIHNlbGYuX2Fk ZF9maWxlKHJ1bm5lciwgJ1NraXBwZWRTdWl0ZScsICdUZXN0MS5odG1sJykKKyAgICAgICAgc2Vs Zi5fYWRkX2ZpbGUocnVubmVyLCAnU2tpcHBlZFN1aXRlJywgJ1Rlc3QyLmh0bWwnKQorICAgICAg ICBzZWxmLl9hZGRfZmlsZShydW5uZXIsICdTdWl0ZScsICdUZXN0MS5odG1sJykKKyAgICAgICAg c2VsZi5fYWRkX2ZpbGUocnVubmVyLCAnU3VpdGUnLCAnVGVzdDIuaHRtbCcpCisgICAgICAgIHNl bGYuX2FkZF9maWxlKHJ1bm5lciwgJ1N1aXRlJywgJ1NraXBwZWRUZXN0MS5odG1sJykKKyAgICAg ICAgc2VsZi5fYWRkX2ZpbGUocnVubmVyLCAnU3VpdGUnLCAnU2tpcHBlZFRlc3QyLmh0bWwnKQor ICAgICAgICBwb3J0LnNraXBwZWRfcGVyZl90ZXN0cyA9IGxhbWJkYTogWydTdWl0ZS9Ta2lwcGVk VGVzdDEuaHRtbCcsICdTdWl0ZS9Ta2lwcGVkVGVzdDEuaHRtbCcsICdTa2lwcGVkU3VpdGUnXQor ICAgICAgICBzZWxmLmFzc2VydEVxdWFsKHNlbGYuX2NvbGxlY3RfdGVzdHNfYW5kX3NvcnRfdGVz dF9uYW1lKHJ1bm5lciksCisgICAgICAgICAgICBbJ1NraXBwZWRTdWl0ZS9UZXN0MS5odG1sJywg J1N1aXRlL1NraXBwZWRUZXN0MS5odG1sJywgJ1N1aXRlL1Rlc3QxLmh0bWwnXSkKKwogICAgIGRl ZiB0ZXN0X2NvbGxlY3RfdGVzdHNfd2l0aF9za2lwcGVkX2xpc3Qoc2VsZik6CiAgICAgICAgIHJ1 bm5lciwgcG9ydCA9IHNlbGYuY3JlYXRlX3J1bm5lcihhcmdzPVsnLS1mb3JjZSddKQogCkluZGV4 OiBUb29scy9TY3JpcHRzL3dlYmtpdHB5L3BlcmZvcm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1bm5l ci5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09Ci0tLSBUb29scy9TY3JpcHRzL3dlYmtpdHB5L3BlcmZvcm1hbmNlX3Rl c3RzL3BlcmZ0ZXN0c3J1bm5lci5weQkocmV2aXNpb24gMTM3OTc2KQorKysgVG9vbHMvU2NyaXB0 cy93ZWJraXRweS9wZXJmb3JtYW5jZV90ZXN0cy9wZXJmdGVzdHNydW5uZXIucHkJKHdvcmtpbmcg Y29weSkKQEAgLTExOCw3ICsxMTgsNyBAQAogICAgICAgICAgICAgICAgIGhlbHA9IlVzZSBXZWJL aXRUZXN0UnVubmVyIHJhdGhlciB0aGFuIER1bXBSZW5kZXJUcmVlLiIpLAogICAgICAgICAgICAg b3B0cGFyc2UubWFrZV9vcHRpb24oIi0tcmVwbGF5IiwgZGVzdD0icmVwbGF5IiwgYWN0aW9uPSJz dG9yZV90cnVlIiwgZGVmYXVsdD1GYWxzZSwKICAgICAgICAgICAgICAgICBoZWxwPSJSdW4gcmVw bGF5IHRlc3RzLiIpLAotICAgICAgICAgICAgb3B0cGFyc2UubWFrZV9vcHRpb24oIi0tZm9yY2Ui LCBkZXN0PSJza2lwcGVkIiwgYWN0aW9uPSJzdG9yZV90cnVlIiwgZGVmYXVsdD1GYWxzZSwKKyAg ICAgICAgICAgIG9wdHBhcnNlLm1ha2Vfb3B0aW9uKCItLWZvcmNlIiwgZGVzdD0iaWdub3JlX3Nr aXBfbGlzdCIsIGFjdGlvbj0ic3RvcmVfZmFsc2UiLCBkZWZhdWx0PVRydWUsCiAgICAgICAgICAg ICAgICAgaGVscD0iUnVuIGFsbCB0ZXN0cywgaW5jbHVkaW5nIHRoZSBvbmVzIGluIHRoZSBTa2lw cGVkIGxpc3QuIiksCiAgICAgICAgICAgICBvcHRwYXJzZS5tYWtlX29wdGlvbigiLS1wcm9maWxl IiwgYWN0aW9uPSJzdG9yZV90cnVlIiwKICAgICAgICAgICAgICAgICBoZWxwPSJPdXRwdXQgcGVy LXRlc3QgcHJvZmlsZSBpbmZvcm1hdGlvbi4iKSwKQEAgLTE0MSwxNyArMTQxLDIxIEBACiAKICAg ICAgICAgcGF0aHMgPSBbXQogICAgICAgICBmb3IgYXJnIGluIHNlbGYuX2FyZ3M6Ci0gICAgICAg ICAgICBwYXRocy5hcHBlbmQoYXJnKQotICAgICAgICAgICAgcmVscGF0aCA9IGZpbGVzeXN0ZW0u cmVscGF0aChhcmcsIHNlbGYuX2Jhc2VfcGF0aCkKLSAgICAgICAgICAgIGlmIHJlbHBhdGg6Ci0g ICAgICAgICAgICAgICAgcGF0aHMuYXBwZW5kKHJlbHBhdGgpCisgICAgICAgICAgICBpZiBmaWxl c3lzdGVtLmV4aXN0cyhmaWxlc3lzdGVtLmpvaW4oc2VsZi5fYmFzZV9wYXRoLCBhcmcpKToKKyAg ICAgICAgICAgICAgICBwYXRocy5hcHBlbmQoYXJnKQorICAgICAgICAgICAgZWxzZToKKyAgICAg ICAgICAgICAgICByZWxwYXRoID0gZmlsZXN5c3RlbS5yZWxwYXRoKGFyZywgc2VsZi5fYmFzZV9w YXRoKQorICAgICAgICAgICAgICAgIGlmIGZpbGVzeXN0ZW0uZXhpc3RzKGZpbGVzeXN0ZW0uam9p bihzZWxmLl9iYXNlX3BhdGgsIHJlbHBhdGgpKToKKyAgICAgICAgICAgICAgICAgICAgcGF0aHMu YXBwZW5kKGZpbGVzeXN0ZW0ubm9ybXBhdGgocmVscGF0aCkpCisgICAgICAgICAgICAgICAgZWxz ZToKKyAgICAgICAgICAgICAgICAgICAgX2xvZy53YXJuKCdQYXRoIHdhcyBub3QgZm91bmQ6JyAr IGFyZykKIAogICAgICAgICBza2lwcGVkX2RpcmVjdG9yaWVzID0gc2V0KFsnLnN2bicsICdyZXNv dXJjZXMnXSkKICAgICAgICAgdGVzdF9maWxlcyA9IGZpbmRfZmlsZXMuZmluZChmaWxlc3lzdGVt LCBzZWxmLl9iYXNlX3BhdGgsIHBhdGhzLCBza2lwcGVkX2RpcmVjdG9yaWVzLCBfaXNfdGVzdF9m aWxlKQogICAgICAgICB0ZXN0cyA9IFtdCiAgICAgICAgIGZvciBwYXRoIGluIHRlc3RfZmlsZXM6 Ci0gICAgICAgICAgICByZWxhdGl2ZV9wYXRoID0gc2VsZi5fcG9ydC5yZWxhdGl2ZV9wZXJmX3Rl c3RfZmlsZW5hbWUocGF0aCkucmVwbGFjZSgnXFwnLCAnLycpCi0gICAgICAgICAgICBpZiBzZWxm Ll9wb3J0LnNraXBzX3BlcmZfdGVzdChyZWxhdGl2ZV9wYXRoKSBhbmQgbm90IHNlbGYuX29wdGlv bnMuc2tpcHBlZDoKKyAgICAgICAgICAgIHJlbGF0aXZlX3BhdGggPSBmaWxlc3lzdGVtLnJlbHBh dGgocGF0aCwgc2VsZi5fYmFzZV9wYXRoKS5yZXBsYWNlKCdcXCcsICcvJykKKyAgICAgICAgICAg IGlmIHNlbGYuX29wdGlvbnMuaWdub3JlX3NraXBfbGlzdCBhbmQgc2VsZi5fcG9ydC5za2lwc19w ZXJmX3Rlc3QocmVsYXRpdmVfcGF0aCkgYW5kIGZpbGVzeXN0ZW0ubm9ybXBhdGgocGF0aCkgaW4g cGF0aHM6CiAgICAgICAgICAgICAgICAgY29udGludWUKICAgICAgICAgICAgIHRlc3QgPSBQZXJm VGVzdEZhY3RvcnkuY3JlYXRlX3BlcmZfdGVzdChzZWxmLl9wb3J0LCByZWxhdGl2ZV9wYXRoLCBw YXRoKQogICAgICAgICAgICAgdGVzdHMuYXBwZW5kKHRlc3QpCkluZGV4OiBUb29scy9DaGFuZ2VM b2cKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PQotLS0gVG9vbHMvQ2hhbmdlTG9nCShyZXZpc2lvbiAxMzc5OTIpCisrKyBU b29scy9DaGFuZ2VMb2cJKHdvcmtpbmcgY29weSkKQEAgLTEsMyArMSwyMSBAQAorMjAxMi0xMi0x NyAgUnlvc3VrZSBOaXdhICA8cm5pd2FAd2Via2l0Lm9yZz4KKworICAgICAgICBSdW5uaW5nIGEg c2tpcHBlZCB0ZXN0IHdpdGggcnVuLXBlcmYtdGVzdHMgY291bGQgYWxlcnQgdGhlIHVzZXIKKyAg ICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTEwNTIxOQorCisg ICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIEZvcmNlIHJ1biBz a2lwcGVkIHBlcmZvcm1hbmNlIHRlc3RzIGV4cGxpY2l0bHkgc3BlY2lmaWVkIGluIHRoZSBjb21t YW5kIGxpbmUuCisgICAgICAgIEFsc28gd2FybiB0aGUgdXNlciB3aGVuIHRoZSBzcGVjaWZpZWQg cGF0aHMgZG9uJ3QgZXhpc3QuCisKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90 ZXN0cy9wb3J0L2Jhc2UucHk6CisgICAgICAgIChQb3J0LnJlbGF0aXZlX3Rlc3RfZmlsZW5hbWUp OgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvcGVyZm9ybWFuY2VfdGVzdHMvcGVyZnRlc3Rz cnVubmVyLnB5OgorICAgICAgICAoUGVyZlRlc3RzUnVubmVyLl9wYXJzZV9hcmdzKToKKyAgICAg ICAgKFBlcmZUZXN0c1J1bm5lci5fY29sbGVjdF90ZXN0cyk6CisgICAgICAgICogU2NyaXB0cy93 ZWJraXRweS9wZXJmb3JtYW5jZV90ZXN0cy9wZXJmdGVzdHNydW5uZXJfdW5pdHRlc3QucHk6Cisg ICAgICAgICh0ZXN0X2NvbGxlY3RfdGVzdHNfd2l0aF9za2lwcGVkX2xpc3RfYW5kX2ZpbGVzKToK KwogMjAxMi0xMi0xNyAgSm9jaGVuIEVpc2luZ2VyICA8am9jaGVuQGNocm9taXVtLm9yZz4KIAog ICAgICAgICBbY2hyb21pdW1dIEZpeCBtZXRob2RzIGluIFRlc3RSdW5uZXIgdGhhdCBkaXJlY3Rs eSBtb2RpZnkgdGhlIHNldHRpbmdzIGluc3RlYWQgb2YgZ29pbmcgdGhyb3VnaCBXZWJQcmVmZXJl bmNlcwo= 179894 2012-12-18 00:08:44 -0800 2012-12-18 11:36:37 -0800 Fix a minor typo fix105219b text/plain 5762 rniwa SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDEzNzk5MikKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDIxIEBACisyMDEyLTEyLTE3ICBSeW9zdWtlIE5pd2EgIDxybml3YUB3ZWJraXQub3JnPgor CisgICAgICAgIFJ1bm5pbmcgYSBza2lwcGVkIHRlc3Qgd2l0aCBydW4tcGVyZi10ZXN0cyBjb3Vs ZCBhbGVydCB0aGUgdXNlcgorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1 Zy5jZ2k/aWQ9MTA1MjE5CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisK KyAgICAgICAgRm9yY2UgcnVuIHNraXBwZWQgcGVyZm9ybWFuY2UgdGVzdHMgZXhwbGljaXRseSBz cGVjaWZpZWQgaW4gdGhlIGNvbW1hbmQgbGluZS4KKyAgICAgICAgQWxzbyB3YXJuIHRoZSB1c2Vy IHdoZW4gdGhlIHNwZWNpZmllZCBwYXRocyBkb24ndCBleGlzdC4KKworICAgICAgICAqIFNjcmlw dHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3BvcnQvYmFzZS5weToKKyAgICAgICAgKFBvcnQucmVs YXRpdmVfdGVzdF9maWxlbmFtZSk6CisgICAgICAgICogU2NyaXB0cy93ZWJraXRweS9wZXJmb3Jt YW5jZV90ZXN0cy9wZXJmdGVzdHNydW5uZXIucHk6CisgICAgICAgIChQZXJmVGVzdHNSdW5uZXIu X3BhcnNlX2FyZ3MpOgorICAgICAgICAoUGVyZlRlc3RzUnVubmVyLl9jb2xsZWN0X3Rlc3RzKToK KyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L3BlcmZvcm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1 bm5lcl91bml0dGVzdC5weToKKyAgICAgICAgKHRlc3RfY29sbGVjdF90ZXN0c193aXRoX3NraXBw ZWRfbGlzdF9hbmRfZmlsZXMpOgorCiAyMDEyLTEyLTE3ICBKb2NoZW4gRWlzaW5nZXIgIDxqb2No ZW5AY2hyb21pdW0ub3JnPgogCiAgICAgICAgIFtjaHJvbWl1bV0gRml4IG1ldGhvZHMgaW4gVGVz dFJ1bm5lciB0aGF0IGRpcmVjdGx5IG1vZGlmeSB0aGUgc2V0dGluZ3MgaW5zdGVhZCBvZiBnb2lu ZyB0aHJvdWdoIFdlYlByZWZlcmVuY2VzCkluZGV4OiBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xh eW91dF90ZXN0cy9wb3J0L2Jhc2UucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvU2NyaXB0cy93ZWJr aXRweS9sYXlvdXRfdGVzdHMvcG9ydC9iYXNlLnB5CShyZXZpc2lvbiAxMzc5NzYpCisrKyBUb29s cy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L2Jhc2UucHkJKHdvcmtpbmcgY29w eSkKQEAgLTgyOCwxMiArODI4LDYgQEAKICAgICAgICAgZWxzZToKICAgICAgICAgICAgIHJldHVy biBzZWxmLmhvc3QuZmlsZXN5c3RlbS5hYnNwYXRoKGZpbGVuYW1lKQogCi0gICAgZGVmIHJlbGF0 aXZlX3BlcmZfdGVzdF9maWxlbmFtZShzZWxmLCBmaWxlbmFtZSk6Ci0gICAgICAgIGlmIGZpbGVu YW1lLnN0YXJ0c3dpdGgoc2VsZi5wZXJmX3Rlc3RzX2RpcigpKToKLSAgICAgICAgICAgIHJldHVy biBzZWxmLmhvc3QuZmlsZXN5c3RlbS5yZWxwYXRoKGZpbGVuYW1lLCBzZWxmLnBlcmZfdGVzdHNf ZGlyKCkpCi0gICAgICAgIGVsc2U6Ci0gICAgICAgICAgICByZXR1cm4gc2VsZi5ob3N0LmZpbGVz eXN0ZW0uYWJzcGF0aChmaWxlbmFtZSkKLQogICAgIEBtZW1vaXplZAogICAgIGRlZiBhYnNwYXRo X2Zvcl90ZXN0KHNlbGYsIHRlc3RfbmFtZSk6CiAgICAgICAgICIiIlJldHVybnMgdGhlIGZ1bGwg cGF0aCB0byB0aGUgZmlsZSBmb3IgYSBnaXZlbiB0ZXN0IG5hbWUuIFRoaXMgaXMgdGhlCkluZGV4 OiBUb29scy9TY3JpcHRzL3dlYmtpdHB5L3BlcmZvcm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1bm5l cl91bml0dGVzdC5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9TY3JpcHRzL3dlYmtpdHB5L3BlcmZv cm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1bm5lcl91bml0dGVzdC5weQkocmV2aXNpb24gMTM3OTc2 KQorKysgVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wZXJmb3JtYW5jZV90ZXN0cy9wZXJmdGVzdHNy dW5uZXJfdW5pdHRlc3QucHkJKHdvcmtpbmcgY29weSkKQEAgLTU4OCw2ICs1ODgsMTkgQEAKICAg ICAgICAgcG9ydC5za2lwcGVkX3BlcmZfdGVzdHMgPSBsYW1iZGE6IFsnaW5zcGVjdG9yL3Vuc3Vw cG9ydGVkX3Rlc3QxLmh0bWwnLCAndW5zdXBwb3J0ZWQnXQogICAgICAgICBzZWxmLmFzc2VydEVx dWFsKHNlbGYuX2NvbGxlY3RfdGVzdHNfYW5kX3NvcnRfdGVzdF9uYW1lKHJ1bm5lciksIFsnaW5z cGVjdG9yL3Rlc3QxLmh0bWwnLCAnaW5zcGVjdG9yL3Rlc3QyLmh0bWwnXSkKIAorICAgIGRlZiB0 ZXN0X2NvbGxlY3RfdGVzdHNfd2l0aF9za2lwcGVkX2xpc3RfYW5kX2ZpbGVzKHNlbGYpOgorICAg ICAgICBydW5uZXIsIHBvcnQgPSBzZWxmLmNyZWF0ZV9ydW5uZXIoYXJncz1bJ1N1aXRlL1Rlc3Qx Lmh0bWwnLCAnU3VpdGUvU2tpcHBlZFRlc3QxLmh0bWwnLCAnU2tpcHBlZFN1aXRlL1Rlc3QxLmh0 bWwnXSkKKworICAgICAgICBzZWxmLl9hZGRfZmlsZShydW5uZXIsICdTa2lwcGVkU3VpdGUnLCAn VGVzdDEuaHRtbCcpCisgICAgICAgIHNlbGYuX2FkZF9maWxlKHJ1bm5lciwgJ1NraXBwZWRTdWl0 ZScsICdUZXN0Mi5odG1sJykKKyAgICAgICAgc2VsZi5fYWRkX2ZpbGUocnVubmVyLCAnU3VpdGUn LCAnVGVzdDEuaHRtbCcpCisgICAgICAgIHNlbGYuX2FkZF9maWxlKHJ1bm5lciwgJ1N1aXRlJywg J1Rlc3QyLmh0bWwnKQorICAgICAgICBzZWxmLl9hZGRfZmlsZShydW5uZXIsICdTdWl0ZScsICdT a2lwcGVkVGVzdDEuaHRtbCcpCisgICAgICAgIHNlbGYuX2FkZF9maWxlKHJ1bm5lciwgJ1N1aXRl JywgJ1NraXBwZWRUZXN0Mi5odG1sJykKKyAgICAgICAgcG9ydC5za2lwcGVkX3BlcmZfdGVzdHMg PSBsYW1iZGE6IFsnU3VpdGUvU2tpcHBlZFRlc3QxLmh0bWwnLCAnU3VpdGUvU2tpcHBlZFRlc3Qx Lmh0bWwnLCAnU2tpcHBlZFN1aXRlJ10KKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChzZWxmLl9j b2xsZWN0X3Rlc3RzX2FuZF9zb3J0X3Rlc3RfbmFtZShydW5uZXIpLAorICAgICAgICAgICAgWydT a2lwcGVkU3VpdGUvVGVzdDEuaHRtbCcsICdTdWl0ZS9Ta2lwcGVkVGVzdDEuaHRtbCcsICdTdWl0 ZS9UZXN0MS5odG1sJ10pCisKICAgICBkZWYgdGVzdF9jb2xsZWN0X3Rlc3RzX3dpdGhfc2tpcHBl ZF9saXN0KHNlbGYpOgogICAgICAgICBydW5uZXIsIHBvcnQgPSBzZWxmLmNyZWF0ZV9ydW5uZXIo YXJncz1bJy0tZm9yY2UnXSkKIApJbmRleDogVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wZXJmb3Jt YW5jZV90ZXN0cy9wZXJmdGVzdHNydW5uZXIucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvU2NyaXB0 cy93ZWJraXRweS9wZXJmb3JtYW5jZV90ZXN0cy9wZXJmdGVzdHNydW5uZXIucHkJKHJldmlzaW9u IDEzNzk3NikKKysrIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvcGVyZm9ybWFuY2VfdGVzdHMvcGVy ZnRlc3RzcnVubmVyLnB5CSh3b3JraW5nIGNvcHkpCkBAIC0xMTgsNyArMTE4LDcgQEAKICAgICAg ICAgICAgICAgICBoZWxwPSJVc2UgV2ViS2l0VGVzdFJ1bm5lciByYXRoZXIgdGhhbiBEdW1wUmVu ZGVyVHJlZS4iKSwKICAgICAgICAgICAgIG9wdHBhcnNlLm1ha2Vfb3B0aW9uKCItLXJlcGxheSIs IGRlc3Q9InJlcGxheSIsIGFjdGlvbj0ic3RvcmVfdHJ1ZSIsIGRlZmF1bHQ9RmFsc2UsCiAgICAg ICAgICAgICAgICAgaGVscD0iUnVuIHJlcGxheSB0ZXN0cy4iKSwKLSAgICAgICAgICAgIG9wdHBh cnNlLm1ha2Vfb3B0aW9uKCItLWZvcmNlIiwgZGVzdD0ic2tpcHBlZCIsIGFjdGlvbj0ic3RvcmVf dHJ1ZSIsIGRlZmF1bHQ9RmFsc2UsCisgICAgICAgICAgICBvcHRwYXJzZS5tYWtlX29wdGlvbigi LS1mb3JjZSIsIGRlc3Q9InVzZV9za2lwcGVkX2xpc3QiLCBhY3Rpb249InN0b3JlX2ZhbHNlIiwg ZGVmYXVsdD1UcnVlLAogICAgICAgICAgICAgICAgIGhlbHA9IlJ1biBhbGwgdGVzdHMsIGluY2x1 ZGluZyB0aGUgb25lcyBpbiB0aGUgU2tpcHBlZCBsaXN0LiIpLAogICAgICAgICAgICAgb3B0cGFy c2UubWFrZV9vcHRpb24oIi0tcHJvZmlsZSIsIGFjdGlvbj0ic3RvcmVfdHJ1ZSIsCiAgICAgICAg ICAgICAgICAgaGVscD0iT3V0cHV0IHBlci10ZXN0IHByb2ZpbGUgaW5mb3JtYXRpb24uIiksCkBA IC0xNDEsMTcgKzE0MSwyMSBAQAogCiAgICAgICAgIHBhdGhzID0gW10KICAgICAgICAgZm9yIGFy ZyBpbiBzZWxmLl9hcmdzOgotICAgICAgICAgICAgcGF0aHMuYXBwZW5kKGFyZykKLSAgICAgICAg ICAgIHJlbHBhdGggPSBmaWxlc3lzdGVtLnJlbHBhdGgoYXJnLCBzZWxmLl9iYXNlX3BhdGgpCi0g ICAgICAgICAgICBpZiByZWxwYXRoOgotICAgICAgICAgICAgICAgIHBhdGhzLmFwcGVuZChyZWxw YXRoKQorICAgICAgICAgICAgaWYgZmlsZXN5c3RlbS5leGlzdHMoZmlsZXN5c3RlbS5qb2luKHNl bGYuX2Jhc2VfcGF0aCwgYXJnKSk6CisgICAgICAgICAgICAgICAgcGF0aHMuYXBwZW5kKGFyZykK KyAgICAgICAgICAgIGVsc2U6CisgICAgICAgICAgICAgICAgcmVscGF0aCA9IGZpbGVzeXN0ZW0u cmVscGF0aChhcmcsIHNlbGYuX2Jhc2VfcGF0aCkKKyAgICAgICAgICAgICAgICBpZiBmaWxlc3lz dGVtLmV4aXN0cyhmaWxlc3lzdGVtLmpvaW4oc2VsZi5fYmFzZV9wYXRoLCByZWxwYXRoKSk6Cisg ICAgICAgICAgICAgICAgICAgIHBhdGhzLmFwcGVuZChmaWxlc3lzdGVtLm5vcm1wYXRoKHJlbHBh dGgpKQorICAgICAgICAgICAgICAgIGVsc2U6CisgICAgICAgICAgICAgICAgICAgIF9sb2cud2Fy bignUGF0aCB3YXMgbm90IGZvdW5kOicgKyBhcmcpCiAKICAgICAgICAgc2tpcHBlZF9kaXJlY3Rv cmllcyA9IHNldChbJy5zdm4nLCAncmVzb3VyY2VzJ10pCiAgICAgICAgIHRlc3RfZmlsZXMgPSBm aW5kX2ZpbGVzLmZpbmQoZmlsZXN5c3RlbSwgc2VsZi5fYmFzZV9wYXRoLCBwYXRocywgc2tpcHBl ZF9kaXJlY3RvcmllcywgX2lzX3Rlc3RfZmlsZSkKICAgICAgICAgdGVzdHMgPSBbXQogICAgICAg ICBmb3IgcGF0aCBpbiB0ZXN0X2ZpbGVzOgotICAgICAgICAgICAgcmVsYXRpdmVfcGF0aCA9IHNl bGYuX3BvcnQucmVsYXRpdmVfcGVyZl90ZXN0X2ZpbGVuYW1lKHBhdGgpLnJlcGxhY2UoJ1xcJywg Jy8nKQotICAgICAgICAgICAgaWYgc2VsZi5fcG9ydC5za2lwc19wZXJmX3Rlc3QocmVsYXRpdmVf cGF0aCkgYW5kIG5vdCBzZWxmLl9vcHRpb25zLnNraXBwZWQ6CisgICAgICAgICAgICByZWxhdGl2 ZV9wYXRoID0gZmlsZXN5c3RlbS5yZWxwYXRoKHBhdGgsIHNlbGYuX2Jhc2VfcGF0aCkucmVwbGFj ZSgnXFwnLCAnLycpCisgICAgICAgICAgICBpZiBzZWxmLl9vcHRpb25zLnVzZV9za2lwcGVkX2xp c3QgYW5kIHNlbGYuX3BvcnQuc2tpcHNfcGVyZl90ZXN0KHJlbGF0aXZlX3BhdGgpIGFuZCBmaWxl c3lzdGVtLm5vcm1wYXRoKHBhdGgpIGluIHBhdGhzOgogICAgICAgICAgICAgICAgIGNvbnRpbnVl CiAgICAgICAgICAgICB0ZXN0ID0gUGVyZlRlc3RGYWN0b3J5LmNyZWF0ZV9wZXJmX3Rlc3Qoc2Vs Zi5fcG9ydCwgcmVsYXRpdmVfcGF0aCwgcGF0aCkKICAgICAgICAgICAgIHRlc3RzLmFwcGVuZCh0 ZXN0KQo=