76594 2012-01-18 17:53:07 -0800 run-perf-tests should support Skipped list 2012-01-19 00:27:52 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 10266 1 rniwa rniwa abarth dpranke eric loislo morrita ojan webkit.review.bot oldest_to_newest 538365 0 rniwa 2012-01-18 17:53:07 -0800 Unfortunately, run-perf-tests don't support all kinds of tests in PerformanceTests. In fact, only directories we can successfully run tests in are BIndings, Parser, and inspector at the moment. We can mitigate the issue for now by adding Skipped list. 538371 1 abarth 2012-01-18 17:59:10 -0800 Do we want a whitelist or a blacklist? 538407 2 rniwa 2012-01-18 19:26:09 -0800 (In reply to comment #1) > Do we want a whitelist or a blacklist? For now I use the same format as Skipped list for layout tests so blacklist. 538411 3 123056 rniwa 2012-01-18 19:48:08 -0800 Created attachment 123056 Patch 538428 4 123056 rniwa 2012-01-18 20:38:57 -0800 Comment on attachment 123056 Patch Great! Thanks for the review. 538443 5 123056 dpranke 2012-01-18 21:08:10 -0800 Comment on attachment 123056 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=123056&action=review > Tools/Scripts/webkitpy/layout_tests/port/test.py:227 > + LAYOUT_TEST_DIR = '/mock-checkout/LayoutTests' Your fix ignores the comments on lines 220-222. I think Eric introduced the /mock-checkout directory in MockSCM at some point, and Adam (?) maybe added the path_to_module() to MockFileSystem to support something similar (although looking at the latter now, that's at least slightly broken because there's no guarantee that /mock-checkout/Tools/Scripts exists in the mock filesystem; we need to create the corresponding directories and files for consistency). At any rate, it is (was?) a feature of NRWT that it could support LayoutTests being located outside of the webkit tree if necessary (as the chromium port once used to do), and this LAYOUT_TEST_DIR partially tested for that; it would concern me if there were tests failing because they assumed the tests lived under $webkit_root/LayoutTests, because they shouldn't (IMO) be assuming that. It could be, of course, that there are better ways to test for the features/test cases I'm worried about. And, at the very least, if we do decide that my concerns above are more trouble than they're worth, you should update or delete the comments to comply. And I would like to document that discussion somewhere we can associate with such a CL so that if we have to do some spelunking in the future we might have a shot at finding it :). 538481 6 rniwa 2012-01-18 21:59:20 -0800 (In reply to comment #5) > Your fix ignores the comments on lines 220-222. I think Eric introduced the /mock-checkout directory in MockSCM at some point, and Adam (?) maybe added the path_to_module() to MockFileSystem to support something similar (although looking at the latter now, that's at least slightly broken because there's no guarantee that /mock-checkout/Tools/Scripts exists in the mock filesystem; we need to create the corresponding directories and files for consistency). > > At any rate, it is (was?) a feature of NRWT that it could support LayoutTests being located outside of the webkit tree if necessary (as the chromium port once used to do), and this LAYOUT_TEST_DIR partially tested for that; it would concern me if there were tests failing because they assumed the tests lived under $webkit_root/LayoutTests, because they shouldn't (IMO) be assuming that. > > It could be, of course, that there are better ways to test for the features/test cases I'm worried about. And, at the very least, if we do decide that my concerns above are more trouble than they're worth, you should update or delete the comments to comply. And I would like to document that discussion somewhere we can associate with such a CL so that if we have to do some spelunking in the future we might have a shot at finding it :). Okay. Thanks for the clarification. I've reverted that part of the change. 538482 7 rniwa 2012-01-18 22:00:44 -0800 Committed r105381: <http://trac.webkit.org/changeset/105381> 538486 8 dpranke 2012-01-18 22:19:00 -0800 Thanks! I'm definitely open to better ways to document / require / test those things. It's probably too subtle as-is. 538489 9 rniwa 2012-01-18 22:29:06 -0800 (In reply to comment #8) > Thanks! I'm definitely open to better ways to document / require / test those things. It's probably too subtle as-is. I think we need to simplify all these classes. Dependencies between host, filesystem, config, etc... are all too confusing. 123056 2012-01-18 19:48:08 -0800 2012-01-18 22:01:44 -0800 Patch bug-76594-20120118194807.patch text/plain 14429 rniwa SW5kZXg6IFBlcmZvcm1hbmNlVGVzdHMvQ2hhbmdlTG9nCj09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIFBlcmZvcm1h bmNlVGVzdHMvQ2hhbmdlTG9nCShyZXZpc2lvbiAxMDUzNzQpCisrKyBQZXJmb3JtYW5jZVRlc3Rz L0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwzICsxLDE1IEBACisyMDEyLTAxLTE4ICBS eW9zdWtlIE5pd2EgIDxybml3YUB3ZWJraXQub3JnPgorCisgICAgICAgIHJ1bi1wZXJmLXRlc3Rz IHNob3VsZCBzdXBwb3J0IFNraXBwZWQgbGlzdAorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0 Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9NzY1OTQKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkg KE9PUFMhKS4KKworICAgICAgICBBZGQgU2tpcHBlZCBsaXN0IHRvIGJlIHVzZWQgYnkgcnVuLXBl cmYtdGVzdHMgYXMgaXQgY2FuIG9ubHkgcnVucyB0ZXN0cyBpbgorICAgICAgICBCaW5kaW5ncywg UGFyc2VyLCBhbmQgaW5zcGVjdG9yIGF0IHRoZSBtb21lbnQuCisKKyAgICAgICAgKiBTa2lwcGVk OiBBZGRlZC4KKwogMjAxMi0wMS0xMiAgSWx5YSBUaWtob25vdnNreSAgPGxvaXNsb0BjaHJvbWl1 bS5vcmc+CiAKICAgICAgICAgV2ViIEluc3BlY3RvcjogcGVyZm9ybWFuY2U6IHJlc3RvcmUgJ2xv ZyAzMDAgbWVzc2FnZXMgaW50byBjb25zb2xlJyB0ZXN0LgpJbmRleDogUGVyZm9ybWFuY2VUZXN0 cy9Ta2lwcGVkCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT0KLS0tIFBlcmZvcm1hbmNlVGVzdHMvU2tpcHBlZAkocmV2aXNp b24gMCkKKysrIFBlcmZvcm1hbmNlVGVzdHMvU2tpcHBlZAkocmV2aXNpb24gMCkKQEAgLTAsMCAr MSw4IEBACisjIE5vdCBlbmFibGVkIGJ5IGRlZmF1bHQgb24gc29tZSBwb3J0cworTXV0YXRpb24K KworIyBEbyBub3QgY29uZm9ybSB0byBXZWJLaXQgb3IgQ2hyb21pdW0gcGVyZiB0ZXN0IHN0eWxl cworTGF5b3V0CitQYWdlTG9hZAorU3VuU3BpZGVyCitYU1NBdWRpdG9yCkluZGV4OiBUb29scy9D aGFuZ2VMb2cKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQ2hhbmdlTG9nCShyZXZpc2lvbiAxMDUzNzQp CisrKyBUb29scy9DaGFuZ2VMb2cJKHdvcmtpbmcgY29weSkKQEAgLTEsMyArMSwzOCBAQAorMjAx Mi0wMS0xOCAgUnlvc3VrZSBOaXdhICA8cm5pd2FAd2Via2l0Lm9yZz4KKworICAgICAgICBydW4t cGVyZi10ZXN0cyBzaG91bGQgc3VwcG9ydCBTa2lwcGVkIGxpc3QKKyAgICAgICAgaHR0cHM6Ly9i dWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTc2NTk0CisKKyAgICAgICAgUmV2aWV3ZWQg YnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgQWRkIGEgc3VwcG9ydCBmb3IgU2tpcHBlZCBs aXN0IGluIHJ1bi1wZXJmLXRlc3RzOyBhbHNvIHNraXAgZmlsZXMgaW4gcmVzb3VyY2VzIGRpcmVj dG9yaWVzLgorCisgICAgICAgIFRoaXMgcGF0Y2ggYWxzbyBmaXhlcyBhIGJ1ZyBpbiBUZXN0IHBv cnQgdGhhdCB3ZWJraXRfYmFzZSByZXR1cm5lZCBhIGRpcmVjdG9yeSAoL3Rlc3QuY2hlY2tvdXQp CisgICAgICAgIGluY29uc2lzdGVudCB3aXRoIE1vY2tGaWxlU3lzdGVtLCB3aGljaCB1c2VzICIv bW9jay1jaGVja291dCIuCisKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0 cy9wb3J0L2Jhc2UucHk6CisgICAgICAgIChQb3J0LnBlcmZfdGVzdHNfZGlyKToKKyAgICAgICAg KFBvcnQuc2tpcHBlZF9wZXJmX3Rlc3RzKToKKyAgICAgICAgKFBvcnQuc2tpcHNfcGVyZl90ZXN0 KToKKyAgICAgICAgKFBvcnQucmVsYXRpdmVfcGVyZl90ZXN0X2ZpbGVuYW1lKToKKyAgICAgICAg KiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L3Rlc3QucHk6CisgICAgICAgIChU ZXN0UG9ydC53ZWJraXRfYmFzZSk6CisgICAgICAgICogU2NyaXB0cy93ZWJraXRweS9sYXlvdXRf dGVzdHMvcG9ydC93ZWJraXQucHk6CisgICAgICAgIChXZWJLaXRQb3J0Ll9leHBlY3RhdGlvbnNf ZnJvbV9za2lwcGVkX2ZpbGVzKToKKyAgICAgICAgKFdlYktpdFBvcnQpOgorICAgICAgICAoV2Vi S2l0UG9ydC5za2lwcGVkX2xheW91dF90ZXN0cyk6CisgICAgICAgIChXZWJLaXRQb3J0LnNraXBw ZWRfcGVyZl90ZXN0cyk6CisgICAgICAgICogU2NyaXB0cy93ZWJraXRweS9wZXJmb3JtYW5jZV90 ZXN0cy9wZXJmdGVzdHNydW5uZXIucHk6CisgICAgICAgIChQZXJmVGVzdHNSdW5uZXIuX2NvbGxl Y3RfdGVzdHMpOgorICAgICAgICAoUGVyZlRlc3RzUnVubmVyLnJ1bik6CisgICAgICAgICogU2Ny aXB0cy93ZWJraXRweS9wZXJmb3JtYW5jZV90ZXN0cy9wZXJmdGVzdHNydW5uZXJfdW5pdHRlc3Qu cHk6CisgICAgICAgICh0ZXN0X3J1bl90ZXN0X3NldF93aXRoX2pzb25fb3V0cHV0KToKKyAgICAg ICAgKHRlc3RfcnVuX3Rlc3Rfc2V0X3dpdGhfanNvbl9zb3VyY2UpOgorICAgICAgICAodGVzdF9j b2xsZWN0X3Rlc3RzX3dpdGhfc2tpcHBlZF9saXN0KToKKyAgICAgICAgKHRlc3RfY29sbGVjdF90 ZXN0c193aXRoX3NraXBwZWRfbGlzdC5hZGRfZmlsZSk6CisgICAgICAgICogU2NyaXB0cy93ZWJr aXRweS90b19iZV9tb3ZlZC9yZWJhc2VsaW5lX2Nocm9taXVtX3dlYmtpdF90ZXN0c191bml0dGVz dC5weToKKyAgICAgICAgKFRlc3RSZWFsTWFpbi50ZXN0X2FsbF9wbGF0Zm9ybXMpOgorCiAyMDEy LTAxLTE4ICBEaXJrIFByYW5rZSAgPGRwcmFua2VAY2hyb21pdW0ub3JnPgogCiAgICAgICAgIEZp eCBwYXRoIHRvIGNocm9taXVtX3NyY19kaXIgaW50cm9kdWNlZCBpbiBwcmV2aW91cyBjaGFuZ2Uu CkluZGV4OiBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L2Jhc2UucHkK PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PQotLS0gVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC9i YXNlLnB5CShyZXZpc2lvbiAxMDUzNjUpCisrKyBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91 dF90ZXN0cy9wb3J0L2Jhc2UucHkJKHdvcmtpbmcgY29weSkKQEAgLTU2NCwxMiArNTY0LDE5IEBA IGNsYXNzIFBvcnQob2JqZWN0KToKICAgICAgICAgIiIiUmV0dXJuIHRoZSBhYnNvbHV0ZSBwYXRo IHRvIHRoZSB0b3Agb2YgdGhlIExheW91dFRlc3RzIGRpcmVjdG9yeS4iIiIKICAgICAgICAgcmV0 dXJuIHNlbGYucGF0aF9mcm9tX3dlYmtpdF9iYXNlKCdMYXlvdXRUZXN0cycpCiAKKyAgICBkZWYg cGVyZl90ZXN0c19kaXIoc2VsZik6CisgICAgICAgICIiIlJldHVybiB0aGUgYWJzb2x1dGUgcGF0 aCB0byB0aGUgdG9wIG9mIHRoZSBQZXJmb3JtYW5jZVRlc3RzIGRpcmVjdG9yeS4iIiIKKyAgICAg ICAgcmV0dXJuIHNlbGYucGF0aF9mcm9tX3dlYmtpdF9iYXNlKCdQZXJmb3JtYW5jZVRlc3RzJykK KwogICAgIGRlZiB3ZWJraXRfYmFzZShzZWxmKToKICAgICAgICAgcmV0dXJuIHNlbGYuX2ZpbGVz eXN0ZW0uYWJzcGF0aChzZWxmLnBhdGhfZnJvbV93ZWJraXRfYmFzZSgnLicpKQogCiAgICAgZGVm IHNraXBwZWRfbGF5b3V0X3Rlc3RzKHNlbGYpOgogICAgICAgICByZXR1cm4gW10KIAorICAgIGRl ZiBza2lwcGVkX3BlcmZfdGVzdHMoc2VsZik6CisgICAgICAgIHJldHVybiBbXQorCiAgICAgZGVm IHNraXBwZWRfdGVzdHMoc2VsZik6CiAgICAgICAgIHJldHVybiBbXQogCkBAIC01ODUsNiArNTky LDE1IEBAIGNsYXNzIFBvcnQob2JqZWN0KToKICAgICAgICAgICAgICAgICByZXR1cm4gVHJ1ZQog ICAgICAgICByZXR1cm4gRmFsc2UKIAorICAgIGRlZiBza2lwc19wZXJmX3Rlc3Qoc2VsZiwgdGVz dF9uYW1lKToKKyAgICAgICAgZm9yIHRlc3Rfb3JfY2F0ZWdvcnkgaW4gc2VsZi5za2lwcGVkX3Bl cmZfdGVzdHMoKToKKyAgICAgICAgICAgIGlmIHRlc3Rfb3JfY2F0ZWdvcnkgPT0gdGVzdF9uYW1l OgorICAgICAgICAgICAgICAgIHJldHVybiBUcnVlCisgICAgICAgICAgICBjYXRlZ29yeSA9IHNl bGYuX2ZpbGVzeXN0ZW0uam9pbihzZWxmLnBlcmZfdGVzdHNfZGlyKCksIHRlc3Rfb3JfY2F0ZWdv cnkpCisgICAgICAgICAgICBpZiBzZWxmLl9maWxlc3lzdGVtLmlzZGlyKGNhdGVnb3J5KSBhbmQg dGVzdF9uYW1lLnN0YXJ0c3dpdGgodGVzdF9vcl9jYXRlZ29yeSk6CisgICAgICAgICAgICAgICAg cmV0dXJuIFRydWUKKyAgICAgICAgcmV0dXJuIEZhbHNlCisKICAgICBkZWYgbWF5YmVfbWFrZV9k aXJlY3Rvcnkoc2VsZiwgKmNvbXBzKToKICAgICAgICAgIiIiQ3JlYXRlcyB0aGUgc3BlY2lmaWVk IGRpcmVjdG9yeSBpZiBpdCBkb2Vzbid0IGFscmVhZHkgZXhpc3QuIiIiCiAgICAgICAgIHNlbGYu X2ZpbGVzeXN0ZW0ubWF5YmVfbWFrZV9kaXJlY3RvcnkoKmNvbXBzKQpAQCAtNjU0LDYgKzY3MCwx MCBAQCBjbGFzcyBQb3J0KG9iamVjdCk6CiAgICAgICAgIGFzc2VydCBmaWxlbmFtZS5zdGFydHN3 aXRoKHNlbGYubGF5b3V0X3Rlc3RzX2RpcigpKSwgIiVzIGRpZCBub3Qgc3RhcnQgd2l0aCAlcyIg JSAoZmlsZW5hbWUsIHNlbGYubGF5b3V0X3Rlc3RzX2RpcigpKQogICAgICAgICByZXR1cm4gZmls ZW5hbWVbbGVuKHNlbGYubGF5b3V0X3Rlc3RzX2RpcigpKSArIDE6XQogCisgICAgZGVmIHJlbGF0 aXZlX3BlcmZfdGVzdF9maWxlbmFtZShzZWxmLCBmaWxlbmFtZSk6CisgICAgICAgIGFzc2VydCBm aWxlbmFtZS5zdGFydHN3aXRoKHNlbGYucGVyZl90ZXN0c19kaXIoKSksICIlcyBkaWQgbm90IHN0 YXJ0IHdpdGggJXMiICUgKGZpbGVuYW1lLCBzZWxmLnBlcmZfdGVzdHNfZGlyKCkpCisgICAgICAg IHJldHVybiBmaWxlbmFtZVtsZW4oc2VsZi5wZXJmX3Rlc3RzX2RpcigpKSArIDE6XQorCiAgICAg ZGVmIGFic3BhdGhfZm9yX3Rlc3Qoc2VsZiwgdGVzdF9uYW1lKToKICAgICAgICAgIiIiUmV0dXJu cyB0aGUgZnVsbCBwYXRoIHRvIHRoZSBmaWxlIGZvciBhIGdpdmVuIHRlc3QgbmFtZS4gVGhpcyBp cyB0aGUKICAgICAgICAgaW52ZXJzZSBvZiByZWxhdGl2ZV90ZXN0X2ZpbGVuYW1lKCkuIiIiCklu ZGV4OiBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9wb3J0L3Rlc3QucHkKPT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PQotLS0gVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC90ZXN0 LnB5CShyZXZpc2lvbiAxMDUzNjUpCisrKyBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90 ZXN0cy9wb3J0L3Rlc3QucHkJKHdvcmtpbmcgY29weSkKQEAgLTIyMiw5ICsyMjIsOSBAQCBsYXll ciBhdCAoMCwwKSBzaXplIDgwMHgzNAogIyByZWxhdGVkIHRvIHRoaXMgYmVmb3JlLgogCiBpZiBz eXMucGxhdGZvcm0gPT0gJ3dpbjMyJzoKLSAgICBMQVlPVVRfVEVTVF9ESVIgPSAnYzovdGVzdC5j aGVja291dC9MYXlvdXRUZXN0cycKKyAgICBMQVlPVVRfVEVTVF9ESVIgPSAnYzovbW9jay1jaGVj a291dC9MYXlvdXRUZXN0cycKIGVsc2U6Ci0gICAgTEFZT1VUX1RFU1RfRElSID0gJy90ZXN0LmNo ZWNrb3V0L0xheW91dFRlc3RzJworICAgIExBWU9VVF9URVNUX0RJUiA9ICcvbW9jay1jaGVja291 dC9MYXlvdXRUZXN0cycKIAogCiAjIEhlcmUgd2Ugc3ludGhlc2l6ZSBhbiBpbi1tZW1vcnkgZmls ZXN5c3RlbSBmcm9tIHRoZSB0ZXN0IGxpc3QKQEAgLTM4NSw3ICszODUsNyBAQCBjbGFzcyBUZXN0 UG9ydChQb3J0KToKICAgICAgICAgcmV0dXJuIExBWU9VVF9URVNUX0RJUgogCiAgICAgZGVmIHdl YmtpdF9iYXNlKHNlbGYpOgotICAgICAgICByZXR1cm4gJy90ZXN0LmNoZWNrb3V0JworICAgICAg ICByZXR1cm4gJy9tb2NrLWNoZWNrb3V0JwogCiAgICAgZGVmIG5hbWUoc2VsZik6CiAgICAgICAg IHJldHVybiBzZWxmLl9uYW1lCkluZGV4OiBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90 ZXN0cy9wb3J0L3dlYmtpdC5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9TY3JpcHRzL3dlYmtpdHB5 L2xheW91dF90ZXN0cy9wb3J0L3dlYmtpdC5weQkocmV2aXNpb24gMTA1MzY1KQorKysgVG9vbHMv U2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvcG9ydC93ZWJraXQucHkJKHdvcmtpbmcgY29w eSkKQEAgLTM1MSw5ICszNTEsOSBAQCBjbGFzcyBXZWJLaXRQb3J0KFBvcnQpOgogICAgICAgICAg ICAgc2VhcmNoX3BhdGhzLnVwZGF0ZShbc2VsZi5fd2syX3BvcnRfbmFtZSgpLCAid2syIl0pCiAg ICAgICAgIHJldHVybiBzZWFyY2hfcGF0aHMKIAotICAgIGRlZiBfZXhwZWN0YXRpb25zX2Zyb21f c2tpcHBlZF9maWxlcyhzZWxmKToKKyAgICBkZWYgX2V4cGVjdGF0aW9uc19mcm9tX3NraXBwZWRf ZmlsZXMoc2VsZiwgc2tpcHBlZF9maWxlX3BhdGhzKToKICAgICAgICAgdGVzdHNfdG9fc2tpcCA9 IFtdCi0gICAgICAgIGZvciBzZWFyY2hfcGF0aCBpbiBzZWxmLl9za2lwcGVkX2ZpbGVfc2VhcmNo X3BhdGhzKCk6CisgICAgICAgIGZvciBzZWFyY2hfcGF0aCBpbiBza2lwcGVkX2ZpbGVfcGF0aHM6 CiAgICAgICAgICAgICBmaWxlbmFtZSA9IHNlbGYuX2ZpbGVzeXN0ZW0uam9pbihzZWxmLl93ZWJr aXRfYmFzZWxpbmVfcGF0aChzZWFyY2hfcGF0aCksICJTa2lwcGVkIikKICAgICAgICAgICAgIGlm IG5vdCBzZWxmLl9maWxlc3lzdGVtLmV4aXN0cyhmaWxlbmFtZSk6CiAgICAgICAgICAgICAgICAg X2xvZy5kZWJ1ZygiU2tpcHBlZCBkb2VzIG5vdCBleGlzdDogJXMiICUgZmlsZW5hbWUpCkBAIC0z NzIsMTMgKzM3MiwxOCBAQCBjbGFzcyBXZWJLaXRQb3J0KFBvcnQpOgogICAgICAgICAgICAgZXhw ZWN0YXRpb25zID0gc2VsZi5fZmlsZXN5c3RlbS5yZWFkX3RleHRfZmlsZShleHBlY3RhdGlvbnNf cGF0aCkKICAgICAgICAgcmV0dXJuIGV4cGVjdGF0aW9ucwogCisgICAgQG1lbW9pemVkCiAgICAg ZGVmIHNraXBwZWRfbGF5b3V0X3Rlc3RzKHNlbGYpOgogICAgICAgICAjIFVzZSBhIHNldCB0byBh bGxvdyBkdXBsaWNhdGVzCi0gICAgICAgIHRlc3RzX3RvX3NraXAgPSBzZXQoc2VsZi5fZXhwZWN0 YXRpb25zX2Zyb21fc2tpcHBlZF9maWxlcygpKQorICAgICAgICB0ZXN0c190b19za2lwID0gc2V0 KHNlbGYuX2V4cGVjdGF0aW9uc19mcm9tX3NraXBwZWRfZmlsZXMoc2VsZi5fc2tpcHBlZF9maWxl X3NlYXJjaF9wYXRocygpKSkKICAgICAgICAgdGVzdHNfdG9fc2tpcC51cGRhdGUoc2VsZi5fdGVz dHNfZm9yX290aGVyX3BsYXRmb3JtcygpKQogICAgICAgICB0ZXN0c190b19za2lwLnVwZGF0ZShz ZWxmLl9za2lwcGVkX3Rlc3RzX2Zvcl91bnN1cHBvcnRlZF9mZWF0dXJlcygpKQogICAgICAgICBy ZXR1cm4gdGVzdHNfdG9fc2tpcAogCisgICAgQG1lbW9pemVkCisgICAgZGVmIHNraXBwZWRfcGVy Zl90ZXN0cyhzZWxmKToKKyAgICAgICAgcmV0dXJuIHNlbGYuX2V4cGVjdGF0aW9uc19mcm9tX3Nr aXBwZWRfZmlsZXMoW3NlbGYucGVyZl90ZXN0c19kaXIoKV0pCisKICAgICBkZWYgc2tpcHBlZF90 ZXN0cyhzZWxmKToKICAgICAgICAgcmV0dXJuIHNlbGYuc2tpcHBlZF9sYXlvdXRfdGVzdHMoKQog CkluZGV4OiBUb29scy9TY3JpcHRzL3dlYmtpdHB5L3BlcmZvcm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0 c3J1bm5lci5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9TY3JpcHRzL3dlYmtpdHB5L3BlcmZvcm1h bmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1bm5lci5weQkocmV2aXNpb24gMTA1MzY1KQorKysgVG9vbHMv U2NyaXB0cy93ZWJraXRweS9wZXJmb3JtYW5jZV90ZXN0cy9wZXJmdGVzdHNydW5uZXIucHkJKHdv cmtpbmcgY29weSkKQEAgLTk2LDcgKzk2LDkgQEAgY2xhc3MgUGVyZlRlc3RzUnVubmVyKG9iamVj dCk6CiAgICAgICAgIGRlZiBfaXNfdGVzdF9maWxlKGZpbGVzeXN0ZW0sIGRpcm5hbWUsIGZpbGVu YW1lKToKICAgICAgICAgICAgIHJldHVybiBmaWxlbmFtZS5lbmRzd2l0aCgnLmh0bWwnKQogCi0g ICAgICAgIHJldHVybiBmaW5kX2ZpbGVzLmZpbmQoc2VsZi5faG9zdC5maWxlc3lzdGVtLCBzZWxm Ll9iYXNlX3BhdGgsIHBhdGhzPXNlbGYuX2FyZ3MsIGZpbGVfZmlsdGVyPV9pc190ZXN0X2ZpbGUp CisgICAgICAgIHNraXBwZWRfZGlyZWN0b3JpZXMgPSBzZXQoWycuc3ZuJywgJ3Jlc291cmNlcydd KQorICAgICAgICB0ZXN0cyA9IGZpbmRfZmlsZXMuZmluZChzZWxmLl9ob3N0LmZpbGVzeXN0ZW0s IHNlbGYuX2Jhc2VfcGF0aCwgc2VsZi5fYXJncywgc2tpcHBlZF9kaXJlY3RvcmllcywgX2lzX3Rl c3RfZmlsZSkKKyAgICAgICAgcmV0dXJuIFt0ZXN0IGZvciB0ZXN0IGluIHRlc3RzIGlmIG5vdCBz ZWxmLl9wb3J0LnNraXBzX3BlcmZfdGVzdChzZWxmLl9wb3J0LnJlbGF0aXZlX3BlcmZfdGVzdF9m aWxlbmFtZSh0ZXN0KSldCiAKICAgICBkZWYgcnVuKHNlbGYpOgogICAgICAgICBpZiBzZWxmLl9v cHRpb25zLmhlbHBfcHJpbnRpbmc6CkBAIC0xMTMsNyArMTE1LDcgQEAgY2xhc3MgUGVyZlRlc3Rz UnVubmVyKG9iamVjdCk6CiAgICAgICAgIHVuZXhwZWN0ZWQgPSAtMQogICAgICAgICB0cnk6CiAg ICAgICAgICAgICB0ZXN0cyA9IHNlbGYuX2NvbGxlY3RfdGVzdHMoKQotICAgICAgICAgICAgdW5l eHBlY3RlZCA9IHNlbGYuX3J1bl90ZXN0c19zZXQodGVzdHMsIHNlbGYuX3BvcnQpCisgICAgICAg ICAgICB1bmV4cGVjdGVkID0gc2VsZi5fcnVuX3Rlc3RzX3NldChzb3J0ZWQobGlzdCh0ZXN0cykp LCBzZWxmLl9wb3J0KQogICAgICAgICBmaW5hbGx5OgogICAgICAgICAgICAgc2VsZi5fcHJpbnRl ci5jbGVhbnVwKCkKIApJbmRleDogVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wZXJmb3JtYW5jZV90 ZXN0cy9wZXJmdGVzdHNydW5uZXJfdW5pdHRlc3QucHkKPT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvU2Ny aXB0cy93ZWJraXRweS9wZXJmb3JtYW5jZV90ZXN0cy9wZXJmdGVzdHNydW5uZXJfdW5pdHRlc3Qu cHkJKHJldmlzaW9uIDEwNTM2NSkKKysrIFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvcGVyZm9ybWFu Y2VfdGVzdHMvcGVyZnRlc3RzcnVubmVyX3VuaXR0ZXN0LnB5CSh3b3JraW5nIGNvcHkpCkBAIC0x NzUsNyArMTc1LDcgQEAgbWF4IDExMjAKIAogICAgIGRlZiB0ZXN0X3J1bl90ZXN0X3NldF93aXRo X2pzb25fb3V0cHV0KHNlbGYpOgogICAgICAgICBidWlsZGJvdF9vdXRwdXQgPSBhcnJheV9zdHJl YW0uQXJyYXlTdHJlYW0oKQotICAgICAgICBydW5uZXIgPSBzZWxmLmNyZWF0ZV9ydW5uZXIoYnVp bGRib3Rfb3V0cHV0LCBhcmdzPVsnLS1vdXRwdXQtanNvbi1wYXRoPS90ZXN0LmNoZWNrb3V0L291 dHB1dC5qc29uJ10pCisgICAgICAgIHJ1bm5lciA9IHNlbGYuY3JlYXRlX3J1bm5lcihidWlsZGJv dF9vdXRwdXQsIGFyZ3M9WyctLW91dHB1dC1qc29uLXBhdGg9L21vY2stY2hlY2tvdXQvb3V0cHV0 Lmpzb24nXSkKICAgICAgICAgcnVubmVyLl9ob3N0LmZpbGVzeXN0ZW0uZmlsZXNbcnVubmVyLl9i YXNlX3BhdGggKyAnL2luc3BlY3Rvci9wYXNzLmh0bWwnXSA9IFRydWUKICAgICAgICAgcnVubmVy Ll9ob3N0LmZpbGVzeXN0ZW0uZmlsZXNbcnVubmVyLl9iYXNlX3BhdGggKyAnL0JpbmRpbmdzL2V2 ZW50LXRhcmdldC13cmFwcGVyLmh0bWwnXSA9IFRydWUKICAgICAgICAgcnVubmVyLl90aW1lc3Rh bXAgPSAxMjM0NTY3ODkKQEAgLTE4NSw3ICsxODUsNyBAQCBtYXggMTEyMAogICAgICAgICBzZWxm LmFzc2VydEVxdWFsKGJ1aWxkYm90X291dHB1dC5nZXQoKVsxXSwgJ21lZGlhbj0gMTQ4NyBtcywg c3RkZXY9IDE0LjQ2IG1zLCBtaW49IDE0NzEgbXMsIG1heD0gMTUxMCBtc1xuJykKICAgICAgICAg c2VsZi5hc3NlcnRFcXVhbChidWlsZGJvdF9vdXRwdXQuZ2V0KClbMl0sICdSRVNVTFQgZ3JvdXBf bmFtZTogdGVzdF9uYW1lPSA0MiBtc1xuJykKIAotICAgICAgICBzZWxmLmFzc2VydEVxdWFsKGpz b24ubG9hZHMocnVubmVyLl9ob3N0LmZpbGVzeXN0ZW0uZmlsZXNbJy90ZXN0LmNoZWNrb3V0L291 dHB1dC5qc29uJ10pLCB7CisgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwoanNvbi5sb2FkcyhydW5u ZXIuX2hvc3QuZmlsZXN5c3RlbS5maWxlc1snL21vY2stY2hlY2tvdXQvb3V0cHV0Lmpzb24nXSks IHsKICAgICAgICAgICAgICJ0aW1lc3RhbXAiOiAxMjM0NTY3ODksICJyZXN1bHRzIjoKICAgICAg ICAgICAgIHsiZXZlbnQtdGFyZ2V0LXdyYXBwZXIiOiB7Im1heCI6ICIxNTEwIiwgImF2ZyI6ICIx NDg5LjA1IiwgIm1lZGlhbiI6ICIxNDg3IiwgIm1pbiI6ICIxNDcxIiwgInN0ZGV2IjogIjE0LjQ2 In0sCiAgICAgICAgICAgICAiZ3JvdXBfbmFtZTp0ZXN0X25hbWUiOiA0Mn0sCkBAIC0xOTMsOSAr MTkzLDkgQEAgbWF4IDExMjAKIAogICAgIGRlZiB0ZXN0X3J1bl90ZXN0X3NldF93aXRoX2pzb25f c291cmNlKHNlbGYpOgogICAgICAgICBidWlsZGJvdF9vdXRwdXQgPSBhcnJheV9zdHJlYW0uQXJy YXlTdHJlYW0oKQotICAgICAgICBydW5uZXIgPSBzZWxmLmNyZWF0ZV9ydW5uZXIoYnVpbGRib3Rf b3V0cHV0LCBhcmdzPVsnLS1vdXRwdXQtanNvbi1wYXRoPS90ZXN0LmNoZWNrb3V0L291dHB1dC5q c29uJywKLSAgICAgICAgICAgICctLXNvdXJjZS1qc29uLXBhdGg9L3Rlc3QuY2hlY2tvdXQvc291 cmNlLmpzb24nXSkKLSAgICAgICAgcnVubmVyLl9ob3N0LmZpbGVzeXN0ZW0uZmlsZXNbJy90ZXN0 LmNoZWNrb3V0L3NvdXJjZS5qc29uJ10gPSAneyJrZXkiOiAidmFsdWUifScKKyAgICAgICAgcnVu bmVyID0gc2VsZi5jcmVhdGVfcnVubmVyKGJ1aWxkYm90X291dHB1dCwgYXJncz1bJy0tb3V0cHV0 LWpzb24tcGF0aD0vbW9jay1jaGVja291dC9vdXRwdXQuanNvbicsCisgICAgICAgICAgICAnLS1z b3VyY2UtanNvbi1wYXRoPS9tb2NrLWNoZWNrb3V0L3NvdXJjZS5qc29uJ10pCisgICAgICAgIHJ1 bm5lci5faG9zdC5maWxlc3lzdGVtLmZpbGVzWycvbW9jay1jaGVja291dC9zb3VyY2UuanNvbidd ID0gJ3sia2V5IjogInZhbHVlIn0nCiAgICAgICAgIHJ1bm5lci5faG9zdC5maWxlc3lzdGVtLmZp bGVzW3J1bm5lci5fYmFzZV9wYXRoICsgJy9pbnNwZWN0b3IvcGFzcy5odG1sJ10gPSBUcnVlCiAg ICAgICAgIHJ1bm5lci5faG9zdC5maWxlc3lzdGVtLmZpbGVzW3J1bm5lci5fYmFzZV9wYXRoICsg Jy9CaW5kaW5ncy9ldmVudC10YXJnZXQtd3JhcHBlci5odG1sJ10gPSBUcnVlCiAgICAgICAgIHJ1 bm5lci5fdGltZXN0YW1wID0gMTIzNDU2Nzg5CkBAIC0yMDUsNyArMjA1LDcgQEAgbWF4IDExMjAK ICAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChidWlsZGJvdF9vdXRwdXQuZ2V0KClbMV0sICdtZWRp YW49IDE0ODcgbXMsIHN0ZGV2PSAxNC40NiBtcywgbWluPSAxNDcxIG1zLCBtYXg9IDE1MTAgbXNc bicpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwoYnVpbGRib3Rfb3V0cHV0LmdldCgpWzJdLCAn UkVTVUxUIGdyb3VwX25hbWU6IHRlc3RfbmFtZT0gNDIgbXNcbicpCiAKLSAgICAgICAgc2VsZi5h c3NlcnRFcXVhbChqc29uLmxvYWRzKHJ1bm5lci5faG9zdC5maWxlc3lzdGVtLmZpbGVzWycvdGVz dC5jaGVja291dC9vdXRwdXQuanNvbiddKSwgeworICAgICAgICBzZWxmLmFzc2VydEVxdWFsKGpz b24ubG9hZHMocnVubmVyLl9ob3N0LmZpbGVzeXN0ZW0uZmlsZXNbJy9tb2NrLWNoZWNrb3V0L291 dHB1dC5qc29uJ10pLCB7CiAgICAgICAgICAgICAidGltZXN0YW1wIjogMTIzNDU2Nzg5LCAicmVz dWx0cyI6CiAgICAgICAgICAgICB7ImV2ZW50LXRhcmdldC13cmFwcGVyIjogeyJtYXgiOiAiMTUx MCIsICJhdmciOiAiMTQ4OS4wNSIsICJtZWRpYW4iOiAiMTQ4NyIsICJtaW4iOiAiMTQ3MSIsICJz dGRldiI6ICIxNC40NiJ9LAogICAgICAgICAgICAgImdyb3VwX25hbWU6dGVzdF9uYW1lIjogNDJ9 LApAQCAtMjE5LDYgKzIxOSwyMyBAQCBtYXggMTEyMAogICAgICAgICB0ZXN0cyA9IHJ1bm5lci5f Y29sbGVjdF90ZXN0cygpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwobGVuKHRlc3RzKSwgMSkK IAorICAgIGRlZiB0ZXN0X2NvbGxlY3RfdGVzdHNfd2l0aF9za2lwcGVkX2xpc3Qoc2VsZik6Cisg ICAgICAgIHJ1bm5lciA9IHNlbGYuY3JlYXRlX3J1bm5lcigpCisKKyAgICAgICAgZGVmIGFkZF9m aWxlKGRpcm5hbWUsIGZpbGVuYW1lLCBjb250ZW50PVRydWUpOgorICAgICAgICAgICAgZGlybmFt ZSA9IHJ1bm5lci5faG9zdC5maWxlc3lzdGVtLmpvaW4ocnVubmVyLl9iYXNlX3BhdGgsIGRpcm5h bWUpIGlmIGRpcm5hbWUgZWxzZSBydW5uZXIuX2Jhc2VfcGF0aAorICAgICAgICAgICAgcnVubmVy Ll9ob3N0LmZpbGVzeXN0ZW0ubWF5YmVfbWFrZV9kaXJlY3RvcnkoZGlybmFtZSkKKyAgICAgICAg ICAgIHJ1bm5lci5faG9zdC5maWxlc3lzdGVtLmZpbGVzW3J1bm5lci5faG9zdC5maWxlc3lzdGVt LmpvaW4oZGlybmFtZSwgZmlsZW5hbWUpXSA9IGNvbnRlbnQKKworICAgICAgICBhZGRfZmlsZSgn aW5zcGVjdG9yJywgJ3Rlc3QxLmh0bWwnKQorICAgICAgICBhZGRfZmlsZSgnaW5zcGVjdG9yJywg J3Vuc3VwcG9ydGVkX3Rlc3QxLmh0bWwnKQorICAgICAgICBhZGRfZmlsZSgnaW5zcGVjdG9yJywg J3Rlc3QyLmh0bWwnKQorICAgICAgICBhZGRfZmlsZSgnaW5zcGVjdG9yL3Jlc291cmNlcycsICdy ZXNvdXJjZV9maWxlLmh0bWwnKQorICAgICAgICBhZGRfZmlsZSgndW5zdXBwb3J0ZWQnLCAndW5z dXBwb3J0ZWRfdGVzdDIuaHRtbCcpCisgICAgICAgIHJ1bm5lci5fcG9ydC5za2lwcGVkX3BlcmZf dGVzdHMgPSBsYW1iZGE6IFsnaW5zcGVjdG9yL3Vuc3VwcG9ydGVkX3Rlc3QxLmh0bWwnLCAndW5z dXBwb3J0ZWQnXQorICAgICAgICB0ZXN0cyA9IFtydW5uZXIuX3BvcnQucmVsYXRpdmVfcGVyZl90 ZXN0X2ZpbGVuYW1lKHRlc3QpIGZvciB0ZXN0IGluIHJ1bm5lci5fY29sbGVjdF90ZXN0cygpXQor ICAgICAgICBzZWxmLmFzc2VydEVxdWFsKHNvcnRlZCh0ZXN0cyksIFsnaW5zcGVjdG9yL3Rlc3Qx Lmh0bWwnLCAnaW5zcGVjdG9yL3Rlc3QyLmh0bWwnXSkKKwogICAgIGRlZiB0ZXN0X3BhcnNlX2Fy Z3Moc2VsZik6CiAgICAgICAgIHJ1bm5lciA9IHNlbGYuY3JlYXRlX3J1bm5lcigpCiAgICAgICAg IG9wdGlvbnMsIGFyZ3MgPSBQZXJmVGVzdHNSdW5uZXIuX3BhcnNlX2FyZ3MoWwpJbmRleDogVG9v bHMvU2NyaXB0cy93ZWJraXRweS90b19iZV9tb3ZlZC9yZWJhc2VsaW5lX2Nocm9taXVtX3dlYmtp dF90ZXN0c191bml0dGVzdC5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9TY3JpcHRzL3dlYmtpdHB5 L3RvX2JlX21vdmVkL3JlYmFzZWxpbmVfY2hyb21pdW1fd2Via2l0X3Rlc3RzX3VuaXR0ZXN0LnB5 CShyZXZpc2lvbiAxMDUzNjUpCisrKyBUb29scy9TY3JpcHRzL3dlYmtpdHB5L3RvX2JlX21vdmVk L3JlYmFzZWxpbmVfY2hyb21pdW1fd2Via2l0X3Rlc3RzX3VuaXR0ZXN0LnB5CSh3b3JraW5nIGNv cHkpCkBAIC0zNDYsOCArMzQ2LDggQEAgY2xhc3MgVGVzdFJlYWxNYWluKHVuaXR0ZXN0LlRlc3RD YXNlKToKIAogICAgICAgICAjIFdlIGV4cGVjdCB0byBoYXZlIHdyaXR0ZW4gMzggZmlsZXMgb3Zl ciB0aGUgY291cnNlIG9mIHRoaXMgcmViYXNlbGluZToKICAgICAgICAgIyAqKSA2KjMgZmlsZXMg aW4gL19faW1fdG1wLyBmb3IgdGhlIGFyY2hpdmVkIG1lbWJlcnMgb2YgdGhlIDYgcG9ydHMKLSAg ICAgICAgIyAqKSAyKjMgZmlsZXMgaW4gL3Rlc3QuY2hlY2tvdXQgZm9yIGFjdHVhbGx5IGRpZmZl cmluZyBiYXNlbGluZXMKLSAgICAgICAgIyAqKSAxIGZpbGUgaW4gL3Rlc3QuY2hlY2tvdXQgZm9y IHRoZSB1cGRhdGVkIHRlc3RfZXhwZWN0YXRpb25zIGZpbGUKKyAgICAgICAgIyAqKSAyKjMgZmls ZXMgaW4gL21vY2stY2hlY2tvdXQgZm9yIGFjdHVhbGx5IGRpZmZlcmluZyBiYXNlbGluZXMKKyAg ICAgICAgIyAqKSAxIGZpbGUgaW4gL21vY2stY2hlY2tvdXQgZm9yIHRoZSB1cGRhdGVkIHRlc3Rf ZXhwZWN0YXRpb25zIGZpbGUKICAgICAgICAgIyAqKSAyKjQgZmlsZXMgaW4gL3RtcCBmb3IgdGhl IG9sZC9uZXcgYmFzZWxpbmVzIGZvciB0aGUgdHdvIGFjdHVhbCBwb3J0cwogICAgICAgICAjICop IDIgZmlsZXMgaW4gL3RtcCBmb3IgdGhlIHRleHQgZGlmZnMgZm9yIHRoZSB0d28gcG9ydHMKICAg ICAgICAgIyAqKSAyIGZpbGVzIGluIC90bXAgZm9yIHRoZSBpbWFnZSBkaWZmcyBmb3IgdGhlIHR3 byBwb3J0cwo=