117831 2013-06-20 05:47:35 -0700 Develop rebase info tool for EWS 2014-12-03 03:54:32 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED WONTFIX P2 Normal --- 118294 92033 118128 118293 1 jbadics webkit-unassigned commit-queue dpranke galpeter glenn kadam ossy rniwa tczene oldest_to_newest 902104 0 jbadics 2013-06-20 05:47:35 -0700 When a patch that affects test behavior on a specific platform is uploaded, rebasing is needed on other platforms as well. In most of the cases, this does not happen on uploading, and the bots stays red for a long time, or some discussion with the author is needed. This tool would get the list of modified tests on the EWS phase after uploading the patch, and notify the developers about ports/tests that would need some revision in a Bugzilla comment. In this way, the author (and the gardeners) can see what tests would need additional expected files, immediately. Another improvement is to run this rebase automatically and upload this rebase patch to the bug, or maybe unifying the rebase patch with the original one. 902107 1 205078 jbadics 2013-06-20 05:53:56 -0700 Created attachment 205078 working progress patch At the current state, EWS gets the list of new expected files in a given patch for a given port (e.g. Qt). Then, after applying the patch, it runs these tests on its own port (e.g. GTK) and the results/diffs can be found in the layout-test-results directory. A further goal would be to use these results/diffs to create an instant rebase patch, or maybe to unify it with the original patch. 903749 2 205078 rniwa 2013-06-26 08:58:06 -0700 Comment on attachment 205078 working progress patch View in context: https://bugs.webkit.org/attachment.cgi?id=205078&action=review Thanks for working on this feature but I think some methods and variable aren't named properly. > Tools/Scripts/webkitpy/tool/bot/earlywarningsystemtask.py:61 > + return self._run_rebased_tests() I don't think run_rebased_tests makes sense as "rebased" is an adjective. I would call it regenerate_expected_results instead. > Tools/Scripts/webkitpy/tool/steps/rebased_tests.py:55 > + def _get_test_name_stub(self, expected): We don't normally use get_ prefixes. > Tools/Scripts/webkitpy/tool/steps/rebased_tests.py:61 > + match = re.match(os.path.join('LayoutTests', 'platform', '(.+?)', '(.*)'), absdir) Why don't we use port.baseline_platform_dir or port.baseline_version_dir instead? > Tools/Scripts/webkitpy/tool/steps/rebased_tests.py:70 > + def _get_tests_by_expected(self, port, expected): I don't understand why this function exits at all. Why can't the caller just call _get_tests_in_dir? Also, I don't understand why the third argument is called expected. > Tools/Scripts/webkitpy/tool/steps/rebased_tests.py:83 > + def _get_first_generic_by_filename(self, port, filename): > + paths = self._get_tests_by_expected(port, filename) > + filestub = self._get_test_name_stub(filename) > + ret = None > + for path in paths: > + hit = re.match('.*' + filestub + '\..*', path) > + if hit: > + ret = hit.group(0) > + break > + return ret What is this function trying to do? 904154 3 205078 ossy 2013-06-27 03:49:37 -0700 Comment on attachment 205078 working progress patch View in context: https://bugs.webkit.org/attachment.cgi?id=205078&action=review >> Tools/Scripts/webkitpy/tool/bot/earlywarningsystemtask.py:61 >> + if self._delegate.run_tests: >> + return self._run_rebased_tests() > > I don't think run_rebased_tests makes sense as "rebased" is an adjective. > I would call it regenerate_expected_results instead. I would make this change separated to the new "RunTestsFromPatch" command. Additionally EWS bots should have three different option in ews.json: - don't run tests at all (now: GTK, GTK-WK2, EFL, EFL-WK2, Qt, Qt-WK2) - run only touched tests to help rebasing tasks (if they have limited hw resources) - run all tests (if they have unlimited hw resources) (now: Win, Mac, Mac-WK2) > Tools/Scripts/webkitpy/tool/commands/download.py:266 > + help_text = "Get the list of rebased patches from patch and make new baselines for own" I prefer the name run-tests-touched-by-attachment and help text: "Run only tests touched by the given attachment" > Tools/Scripts/webkitpy/tool/steps/forcetests.py:47 > +class ForceTests(AbstractStep): > + @classmethod > + def options(cls): > + return AbstractStep.options() > + > + def run(self, state): > + self._options.test = True > + self._options.layout_testlist_only = True > + self._options.non_interactive = True I don't like the idea to have a new ForceTests class to make RunTestsFromPatch command really run tests independently of --test command line option. Additionally it still doesn't build the attachment before trying to run the tests. There are similar strangeness with many webkit-patch commands, for example: - build-and-test doesn't build and doesn't test ... only if --build and --test command line options are passed - build-attachment doesn't build ... only if --build passed ... I prefer if AbstractPatchSequencingCommand would have _prepare_state function similar to AbstractSequencedCommand. > Tools/Scripts/webkitpy/tool/steps/runtests.py:113 > + if state["testlist"]: > + args.extend(state["testlist"]) What if the testlist is empty? In this case all tests would run, but we don't want to run any layout test. 904156 4 ossy 2013-06-27 03:56:05 -0700 I think we should change this bug to a meta bug with depending bugs like: - [webkitpy] AbstractPatchSequencingCommand should have _prepare_state - [webkitpy] Add run-tests-touched-by-attachment command - Let EWS bots run only touched tests - webkit-patch needs a apply-ews-results-from-bug command - bug92033 904225 5 ossy 2013-06-27 07:12:08 -0700 (In reply to comment #4) > - [webkitpy] AbstractPatchSequencingCommand should have _prepare_state new bug with proposed patch: https://bugs.webkit.org/show_bug.cgi?id=118128 906155 6 jbadics 2013-07-04 06:16:56 -0700 (In reply to comment #2) > > > Tools/Scripts/webkitpy/tool/steps/rebased_tests.py:61 > > + match = re.match(os.path.join('LayoutTests', 'platform', '(.+?)', '(.*)'), absdir) > > Why don't we use port.baseline_platform_dir or port.baseline_version_dir instead? The goal of this function is to return the generic directory path without the LayoutTests prefix. So, if the last parameter is LayoutTests/platform/efl/css1/basic/containment-expected.txt, then match is (u'efl', u'css1/basic'). This function returns css1/basic in this case. However, port.baseline_platform_dir returns /path/to/WebKit/repository/LayoutTests/platform/qt, if the script runs on a Qt EWS. But I can't use it to get css1/basic from 'LayoutTests/platform/efl/css1/basic/containment-expected.txt'. > > > Tools/Scripts/webkitpy/tool/steps/rebased_tests.py:83 > > + def _get_first_generic_by_filename(self, port, filename): > > + paths = self._get_tests_by_expected(port, filename) > > + filestub = self._get_test_name_stub(filename) > > + ret = None > > + for path in paths: > > + hit = re.match('.*' + filestub + '\..*', path) > > + if hit: > > + ret = hit.group(0) > > + break > > + return ret > > What is this function trying to do? According to the description at https://bugs.webkit.org/show_bug.cgi?id=118293 : _test_relative_dir() determines the relative generic directory path for the given test, e.g. LayoutTests/platform/qt-5.0-wk1/editing/deleting/delete-cell-contents-expected.txt -> editing/deleting/ Then _tests_in_dir gets the paths of all the tests found in this directory. As a last step, the loop in _first_generic_by_filename() gets the first match to the given file name. I am sure it can be solved in a more elegant way; currently this is the only way I can do it. 1052118 7 ossy 2014-12-03 03:54:32 -0800 no interest in it anymore 205078 2013-06-20 05:53:56 -0700 2013-06-27 03:49:37 -0700 working progress patch rebwithews3.patch text/plain 11596 jbadics ZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvdG9vbC9ib3QvZWFybHl3YXJuaW5n c3lzdGVtdGFzay5weSBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvdG9vbC9ib3QvZWFybHl3YXJu aW5nc3lzdGVtdGFzay5weQppbmRleCBiNjZjZmJjLi40ZjM3ZGM4IDEwMDY0NAotLS0gYS9Ub29s cy9TY3JpcHRzL3dlYmtpdHB5L3Rvb2wvYm90L2Vhcmx5d2FybmluZ3N5c3RlbXRhc2sucHkKKysr IGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS90b29sL2JvdC9lYXJseXdhcm5pbmdzeXN0ZW10YXNr LnB5CkBAIC01Nyw2ICs1Nyw4IEBAIGNsYXNzIEVhcmx5V2FybmluZ1N5c3RlbVRhc2soUGF0Y2hB bmFseXNpc1Rhc2spOgogICAgICAgICAgICAgcmV0dXJuIEZhbHNlCiAgICAgICAgIGlmIG5vdCBz ZWxmLl9hcHBseSgpOgogICAgICAgICAgICAgcmFpc2UgVW5hYmxlVG9BcHBseVBhdGNoKHNlbGYu X3BhdGNoKQorICAgICAgICBpZiBzZWxmLl9kZWxlZ2F0ZS5ydW5fdGVzdHM6CisgICAgICAgICAg ICByZXR1cm4gc2VsZi5fcnVuX3JlYmFzZWRfdGVzdHMoKQogICAgICAgICBpZiBub3Qgc2VsZi5f YnVpbGQoKToKICAgICAgICAgICAgIGlmIG5vdCBzZWxmLl9idWlsZF93aXRob3V0X3BhdGNoKCk6 CiAgICAgICAgICAgICAgICAgcmV0dXJuIEZhbHNlCmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRz L3dlYmtpdHB5L3Rvb2wvYm90L3BhdGNoYW5hbHlzaXN0YXNrLnB5IGIvVG9vbHMvU2NyaXB0cy93 ZWJraXRweS90b29sL2JvdC9wYXRjaGFuYWx5c2lzdGFzay5weQppbmRleCBiMDFjNmM3Li5lZWRi ZTI1IDEwMDY0NAotLS0gYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3Rvb2wvYm90L3BhdGNoYW5h bHlzaXN0YXNrLnB5CisrKyBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvdG9vbC9ib3QvcGF0Y2hh bmFseXNpc3Rhc2sucHkKQEAgLTE2OSw2ICsxNjksMTMgQEAgY2xhc3MgUGF0Y2hBbmFseXNpc1Rh c2sob2JqZWN0KToKICAgICAgICAgIkxhbmRlZCBwYXRjaCIsCiAgICAgICAgICJVbmFibGUgdG8g bGFuZCBwYXRjaCIpCiAKKyAgICBkZWYgX3J1bl9yZWJhc2VkX3Rlc3RzKHNlbGYpOgorICAgICAg ICByZXR1cm4gc2VsZi5fcnVuX2NvbW1hbmQoWworICAgICAgICAgICAgInJ1bi10ZXN0cy1mcm9t LXBhdGNoIiwKKyAgICAgICAgXSwKKyAgICAgICAgIlJlYmFzZWQgdGVzdHMgaGF2ZSBiZWVuIHJ1 bi4iLAorICAgICAgICAiQ291bGRuJ3QgcnVuIHJlYmFzZWQgdGVzdHMuIikKKwogICAgIGRlZiBf cmVwb3J0X2ZsYWt5X3Rlc3RzKHNlbGYsIGZsYWt5X3Rlc3RfcmVzdWx0cywgcmVzdWx0c19hcmNo aXZlKToKICAgICAgICAgc2VsZi5fZGVsZWdhdGUucmVwb3J0X2ZsYWt5X3Rlc3RzKHNlbGYuX3Bh dGNoLCBmbGFreV90ZXN0X3Jlc3VsdHMsIHJlc3VsdHNfYXJjaGl2ZSkKIApkaWZmIC0tZ2l0IGEv VG9vbHMvU2NyaXB0cy93ZWJraXRweS90b29sL2NvbW1hbmRzL2Rvd25sb2FkLnB5IGIvVG9vbHMv U2NyaXB0cy93ZWJraXRweS90b29sL2NvbW1hbmRzL2Rvd25sb2FkLnB5CmluZGV4IGQzMTY1Mzgu LjA0NTBjZWMgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvdG9vbC9jb21tYW5k cy9kb3dubG9hZC5weQorKysgYi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3Rvb2wvY29tbWFuZHMv ZG93bmxvYWQucHkKQEAgLTQwLDYgKzQwLDkgQEAgZnJvbSB3ZWJraXRweS50b29sLmNvbW1lbnRz IGltcG9ydCBidWdfY29tbWVudF9mcm9tX2NvbW1pdF90ZXh0CiBmcm9tIHdlYmtpdHB5LnRvb2wu Z3JhbW1hciBpbXBvcnQgcGx1cmFsaXplCiBmcm9tIHdlYmtpdHB5LnRvb2wubXVsdGljb21tYW5k dG9vbCBpbXBvcnQgQ29tbWFuZAogCitmcm9tIHdlYmtpdHB5LnRvb2wuc3RlcHMuYWJzdHJhY3Rz dGVwIGltcG9ydCBBYnN0cmFjdFN0ZXAKK2Zyb20gd2Via2l0cHkudG9vbC5zdGVwcy5vcHRpb25z IGltcG9ydCBPcHRpb25zCisKIF9sb2cgPSBsb2dnaW5nLmdldExvZ2dlcihfX25hbWVfXykKIAog CkBAIC0yNTgsNiArMjYxLDI0IEBAIGNsYXNzIEJ1aWxkQXR0YWNobWVudChBYnN0cmFjdFBhdGNo U2VxdWVuY2luZ0NvbW1hbmQsIFByb2Nlc3NBdHRhY2htZW50c01peGluKToKICAgICBdCiAKIAor Y2xhc3MgUnVuVGVzdHNGcm9tUGF0Y2goQWJzdHJhY3RQYXRjaFNlcXVlbmNpbmdDb21tYW5kLCBQ cm9jZXNzQXR0YWNobWVudHNNaXhpbik6CisgICAgbmFtZSA9ICJydW4tdGVzdHMtZnJvbS1wYXRj aCIKKyAgICBoZWxwX3RleHQgPSAiR2V0IHRoZSBsaXN0IG9mIHJlYmFzZWQgcGF0Y2hlcyBmcm9t IHBhdGNoIGFuZCBtYWtlIG5ldyBiYXNlbGluZXMgZm9yIG93biIKKyAgICBhcmd1bWVudF9uYW1l cyA9ICJBVFRBQ0hNRU5UX0lEIFtBVFRBQ0hNRU5UX0lEU10iCisgICAgcHJlcGFyZV9zdGVwcyA9 IFsKKyAgICAgICAgc3RlcHMuRm9yY2VUZXN0cywKKyAgICBdCisgICAgbWFpbl9zdGVwcyA9IFsK KyAgICAgICAgc3RlcHMuRGlzY2FyZExvY2FsQ2hhbmdlcywKKyAgICAgICAgc3RlcHMuVXBkYXRl LAorICAgICAgICBzdGVwcy5CdWlsZCwKKyAgICAgICAgc3RlcHMuQXBwbHlQYXRjaCwKKyAgICAg ICAgc3RlcHMuUmViYXNlZFRlc3RzLAorICAgICAgICBzdGVwcy5SdW5UZXN0cywKKyAgICBdCisK KworCiBjbGFzcyBCdWlsZEFuZFRlc3RBdHRhY2htZW50KEFic3RyYWN0UGF0Y2hTZXF1ZW5jaW5n Q29tbWFuZCwgUHJvY2Vzc0F0dGFjaG1lbnRzTWl4aW4pOgogICAgIG5hbWUgPSAiYnVpbGQtYW5k LXRlc3QtYXR0YWNobWVudCIKICAgICBoZWxwX3RleHQgPSAiQXBwbHksIGJ1aWxkLCBhbmQgdGVz dCBwYXRjaGVzIGZyb20gYnVnemlsbGEiCmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRzL3dlYmtp dHB5L3Rvb2wvc3RlcHMvX19pbml0X18ucHkgYi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3Rvb2wv c3RlcHMvX19pbml0X18ucHkKaW5kZXggNjU1ZjdkNS4uOGMxY2FhNyAxMDA2NDQKLS0tIGEvVG9v bHMvU2NyaXB0cy93ZWJraXRweS90b29sL3N0ZXBzL19faW5pdF9fLnB5CisrKyBiL1Rvb2xzL1Nj cmlwdHMvd2Via2l0cHkvdG9vbC9zdGVwcy9fX2luaXRfXy5weQpAQCAtNjIsMyArNjIsNSBAQCBm cm9tIHdlYmtpdHB5LnRvb2wuc3RlcHMudXBkYXRlIGltcG9ydCBVcGRhdGUKIGZyb20gd2Via2l0 cHkudG9vbC5zdGVwcy51cGRhdGVjaGFuZ2Vsb2dzd2l0aHJldmlld2VyIGltcG9ydCBVcGRhdGVD aGFuZ2VMb2dzV2l0aFJldmlld2VyCiBmcm9tIHdlYmtpdHB5LnRvb2wuc3RlcHMudmFsaWRhdGVj aGFuZ2Vsb2dzIGltcG9ydCBWYWxpZGF0ZUNoYW5nZUxvZ3MKIGZyb20gd2Via2l0cHkudG9vbC5z dGVwcy52YWxpZGF0ZXJldmlld2VyIGltcG9ydCBWYWxpZGF0ZVJldmlld2VyCitmcm9tIHdlYmtp dHB5LnRvb2wuc3RlcHMucmViYXNlZF90ZXN0cyBpbXBvcnQgUmViYXNlZFRlc3RzCitmcm9tIHdl YmtpdHB5LnRvb2wuc3RlcHMuZm9yY2V0ZXN0cyBpbXBvcnQgRm9yY2VUZXN0cwpkaWZmIC0tZ2l0 IGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS90b29sL3N0ZXBzL2ZvcmNldGVzdHMucHkgYi9Ub29s cy9TY3JpcHRzL3dlYmtpdHB5L3Rvb2wvc3RlcHMvZm9yY2V0ZXN0cy5weQpuZXcgZmlsZSBtb2Rl IDEwMDY0NAppbmRleCAwMDAwMDAwLi45OWVmZWJjCi0tLSAvZGV2L251bGwKKysrIGIvVG9vbHMv U2NyaXB0cy93ZWJraXRweS90b29sL3N0ZXBzL2ZvcmNldGVzdHMucHkKQEAgLTAsMCArMSw0NyBA QAorIyBDb3B5cmlnaHQgKEMpIDIwMTMgSmFub3MgQmFkaWNzIChqYmFkaWNzQGluZi51LXN6ZWdl ZC5odSksCisjIFVuaXZlcnNpdHkgb2YgU3plZ2VkLiBBbGwgcmlnaHRzIHJlc2VydmVkLgorIwor IyBSZWRpc3RyaWJ1dGlvbiBhbmQgdXNlIGluIHNvdXJjZSBhbmQgYmluYXJ5IGZvcm1zLCB3aXRo IG9yIHdpdGhvdXQKKyMgbW9kaWZpY2F0aW9uLCBhcmUgcGVybWl0dGVkIHByb3ZpZGVkIHRoYXQg dGhlIGZvbGxvd2luZyBjb25kaXRpb25zIGFyZQorIyBtZXQ6CisjCisjICAgICAqIFJlZGlzdHJp YnV0aW9ucyBvZiBzb3VyY2UgY29kZSBtdXN0IHJldGFpbiB0aGUgYWJvdmUgY29weXJpZ2h0Cisj IG5vdGljZSwgdGhpcyBsaXN0IG9mIGNvbmRpdGlvbnMgYW5kIHRoZSBmb2xsb3dpbmcgZGlzY2xh aW1lci4KKyMgICAgICogUmVkaXN0cmlidXRpb25zIGluIGJpbmFyeSBmb3JtIG11c3QgcmVwcm9k dWNlIHRoZSBhYm92ZQorIyBjb3B5cmlnaHQgbm90aWNlLCB0aGlzIGxpc3Qgb2YgY29uZGl0aW9u cyBhbmQgdGhlIGZvbGxvd2luZyBkaXNjbGFpbWVyCisjIGluIHRoZSBkb2N1bWVudGF0aW9uIGFu ZC9vciBvdGhlciBtYXRlcmlhbHMgcHJvdmlkZWQgd2l0aCB0aGUKKyMgZGlzdHJpYnV0aW9uLgor IyAgICAgKiBOZWl0aGVyIHRoZSBuYW1lIG9mIEdvb2dsZSBJbmMuIG5vciB0aGUgbmFtZXMgb2Yg aXRzCisjIGNvbnRyaWJ1dG9ycyBtYXkgYmUgdXNlZCB0byBlbmRvcnNlIG9yIHByb21vdGUgcHJv ZHVjdHMgZGVyaXZlZCBmcm9tCisjIHRoaXMgc29mdHdhcmUgd2l0aG91dCBzcGVjaWZpYyBwcmlv ciB3cml0dGVuIHBlcm1pc3Npb24uCisjCisjIFRISVMgU09GVFdBUkUgSVMgUFJPVklERUQgQlkg VEhFIENPUFlSSUdIVCBIT0xERVJTIEFORCBDT05UUklCVVRPUlMKKyMgIkFTIElTIiBBTkQgQU5Z IEVYUFJFU1MgT1IgSU1QTElFRCBXQVJSQU5USUVTLCBJTkNMVURJTkcsIEJVVCBOT1QKKyMgTElN SVRFRCBUTywgVEhFIElNUExJRUQgV0FSUkFOVElFUyBPRiBNRVJDSEFOVEFCSUxJVFkgQU5EIEZJ VE5FU1MgRk9SCisjIEEgUEFSVElDVUxBUiBQVVJQT1NFIEFSRSBESVNDTEFJTUVELiBJTiBOTyBF VkVOVCBTSEFMTCBUSEUgQ09QWVJJR0hUCisjIE9XTkVSIE9SIENPTlRSSUJVVE9SUyBCRSBMSUFC TEUgRk9SIEFOWSBESVJFQ1QsIElORElSRUNULCBJTkNJREVOVEFMLAorIyBTUEVDSUFMLCBFWEVN UExBUlksIE9SIENPTlNFUVVFTlRJQUwgREFNQUdFUyAoSU5DTFVESU5HLCBCVVQgTk9UCisjIExJ TUlURUQgVE8sIFBST0NVUkVNRU5UIE9GIFNVQlNUSVRVVEUgR09PRFMgT1IgU0VSVklDRVM7IExP U1MgT0YgVVNFLAorIyBEQVRBLCBPUiBQUk9GSVRTOyBPUiBCVVNJTkVTUyBJTlRFUlJVUFRJT04p IEhPV0VWRVIgQ0FVU0VEIEFORCBPTiBBTlkKKyMgVEhFT1JZIE9GIExJQUJJTElUWSwgV0hFVEhF UiBJTiBDT05UUkFDVCwgU1RSSUNUIExJQUJJTElUWSwgT1IgVE9SVAorIyAoSU5DTFVESU5HIE5F R0xJR0VOQ0UgT1IgT1RIRVJXSVNFKSBBUklTSU5HIElOIEFOWSBXQVkgT1VUIE9GIFRIRSBVU0UK KyMgT0YgVEhJUyBTT0ZUV0FSRSwgRVZFTiBJRiBBRFZJU0VEIE9GIFRIRSBQT1NTSUJJTElUWSBP RiBTVUNIIERBTUFHRS4KKworaW1wb3J0IGxvZ2dpbmcKK2ltcG9ydCBvcworaW1wb3J0IHBsYXRm b3JtCitpbXBvcnQgc3lzCitmcm9tIHdlYmtpdHB5LnRvb2wuc3RlcHMuYWJzdHJhY3RzdGVwIGlt cG9ydCBBYnN0cmFjdFN0ZXAKK2Zyb20gd2Via2l0cHkudG9vbC5zdGVwcy5vcHRpb25zIGltcG9y dCBPcHRpb25zCitmcm9tIHdlYmtpdHB5LmNvbW1vbi5zeXN0ZW0uZXhlY3V0aXZlIGltcG9ydCBT Y3JpcHRFcnJvcgorCisKK2NsYXNzIEZvcmNlVGVzdHMoQWJzdHJhY3RTdGVwKToKKyAgICBAY2xh c3NtZXRob2QKKyAgICBkZWYgb3B0aW9ucyhjbHMpOgorICAgICAgICByZXR1cm4gQWJzdHJhY3RT dGVwLm9wdGlvbnMoKQorCisgICAgZGVmIHJ1bihzZWxmLCBzdGF0ZSk6CisgICAgICAgIHNlbGYu X29wdGlvbnMudGVzdCA9IFRydWUKKyAgICAgICAgc2VsZi5fb3B0aW9ucy5sYXlvdXRfdGVzdGxp c3Rfb25seSA9IFRydWUKKyAgICAgICAgc2VsZi5fb3B0aW9ucy5ub25faW50ZXJhY3RpdmUgPSBU cnVlCmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3Rvb2wvc3RlcHMvcmViYXNl ZF90ZXN0cy5weSBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvdG9vbC9zdGVwcy9yZWJhc2VkX3Rl c3RzLnB5Cm5ldyBmaWxlIG1vZGUgMTAwNjQ0CmluZGV4IDAwMDAwMDAuLmRlMDEyOTUKLS0tIC9k ZXYvbnVsbAorKysgYi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3Rvb2wvc3RlcHMvcmViYXNlZF90 ZXN0cy5weQpAQCAtMCwwICsxLDk5IEBACisjIENvcHlyaWdodCAoQykgMjAxMyBKYW5vcyBCYWRp Y3MgKGpiYWRpY3NAaW5mLnUtc3plZ2VkLmh1KSwKKyMgVW5pdmVyc2l0eSBvZiBTemVnZWQuIEFs bCByaWdodHMgcmVzZXJ2ZWQuCisjCisjIFJlZGlzdHJpYnV0aW9uIGFuZCB1c2UgaW4gc291cmNl IGFuZCBiaW5hcnkgZm9ybXMsIHdpdGggb3Igd2l0aG91dAorIyBtb2RpZmljYXRpb24sIGFyZSBw ZXJtaXR0ZWQgcHJvdmlkZWQgdGhhdCB0aGUgZm9sbG93aW5nIGNvbmRpdGlvbnMgYXJlCisjIG1l dDoKKyMKKyMgICAgICogUmVkaXN0cmlidXRpb25zIG9mIHNvdXJjZSBjb2RlIG11c3QgcmV0YWlu IHRoZSBhYm92ZSBjb3B5cmlnaHQKKyMgbm90aWNlLCB0aGlzIGxpc3Qgb2YgY29uZGl0aW9ucyBh bmQgdGhlIGZvbGxvd2luZyBkaXNjbGFpbWVyLgorIyAgICAgKiBSZWRpc3RyaWJ1dGlvbnMgaW4g YmluYXJ5IGZvcm0gbXVzdCByZXByb2R1Y2UgdGhlIGFib3ZlCisjIGNvcHlyaWdodCBub3RpY2Us IHRoaXMgbGlzdCBvZiBjb25kaXRpb25zIGFuZCB0aGUgZm9sbG93aW5nIGRpc2NsYWltZXIKKyMg aW4gdGhlIGRvY3VtZW50YXRpb24gYW5kL29yIG90aGVyIG1hdGVyaWFscyBwcm92aWRlZCB3aXRo IHRoZQorIyBkaXN0cmlidXRpb24uCisjICAgICAqIE5laXRoZXIgdGhlIG5hbWUgb2YgR29vZ2xl IEluYy4gbm9yIHRoZSBuYW1lcyBvZiBpdHMKKyMgY29udHJpYnV0b3JzIG1heSBiZSB1c2VkIHRv IGVuZG9yc2Ugb3IgcHJvbW90ZSBwcm9kdWN0cyBkZXJpdmVkIGZyb20KKyMgdGhpcyBzb2Z0d2Fy ZSB3aXRob3V0IHNwZWNpZmljIHByaW9yIHdyaXR0ZW4gcGVybWlzc2lvbi4KKyMKKyMgVEhJUyBT T0ZUV0FSRSBJUyBQUk9WSURFRCBCWSBUSEUgQ09QWVJJR0hUIEhPTERFUlMgQU5EIENPTlRSSUJV VE9SUworIyAiQVMgSVMiIEFORCBBTlkgRVhQUkVTUyBPUiBJTVBMSUVEIFdBUlJBTlRJRVMsIElO Q0xVRElORywgQlVUIE5PVAorIyBMSU1JVEVEIFRPLCBUSEUgSU1QTElFRCBXQVJSQU5USUVTIE9G IE1FUkNIQU5UQUJJTElUWSBBTkQgRklUTkVTUyBGT1IKKyMgQSBQQVJUSUNVTEFSIFBVUlBPU0Ug QVJFIERJU0NMQUlNRUQuIElOIE5PIEVWRU5UIFNIQUxMIFRIRSBDT1BZUklHSFQKKyMgT1dORVIg T1IgQ09OVFJJQlVUT1JTIEJFIExJQUJMRSBGT1IgQU5ZIERJUkVDVCwgSU5ESVJFQ1QsIElOQ0lE RU5UQUwsCisjIFNQRUNJQUwsIEVYRU1QTEFSWSwgT1IgQ09OU0VRVUVOVElBTCBEQU1BR0VTIChJ TkNMVURJTkcsIEJVVCBOT1QKKyMgTElNSVRFRCBUTywgUFJPQ1VSRU1FTlQgT0YgU1VCU1RJVFVU RSBHT09EUyBPUiBTRVJWSUNFUzsgTE9TUyBPRiBVU0UsCisjIERBVEEsIE9SIFBST0ZJVFM7IE9S IEJVU0lORVNTIElOVEVSUlVQVElPTikgSE9XRVZFUiBDQVVTRUQgQU5EIE9OIEFOWQorIyBUSEVP UlkgT0YgTElBQklMSVRZLCBXSEVUSEVSIElOIENPTlRSQUNULCBTVFJJQ1QgTElBQklMSVRZLCBP UiBUT1JUCisjIChJTkNMVURJTkcgTkVHTElHRU5DRSBPUiBPVEhFUldJU0UpIEFSSVNJTkcgSU4g QU5ZIFdBWSBPVVQgT0YgVEhFIFVTRQorIyBPRiBUSElTIFNPRlRXQVJFLCBFVkVOIElGIEFEVklT RUQgT0YgVEhFIFBPU1NJQklMSVRZIE9GIFNVQ0ggREFNQUdFLgorCitpbXBvcnQgbG9nZ2luZwor aW1wb3J0IHJlCitpbXBvcnQgb3MKKworZnJvbSB3ZWJraXRweS50b29sLnN0ZXBzLmFic3RyYWN0 c3RlcCBpbXBvcnQgQWJzdHJhY3RTdGVwCitmcm9tIHdlYmtpdHB5LnRvb2wuc3RlcHMub3B0aW9u cyBpbXBvcnQgT3B0aW9ucworCitmcm9tIHdlYmtpdHB5LnBvcnQgaW1wb3J0IFBvcnQKK2Zyb20g d2Via2l0cHkuY29tbW9uLmhvc3QgaW1wb3J0IEhvc3QKK2Zyb20gd2Via2l0cHkubGF5b3V0X3Rl c3RzLmNvbnRyb2xsZXJzLmxheW91dF90ZXN0X2ZpbmRlciBpbXBvcnQgTGF5b3V0VGVzdEZpbmRl cgorZnJvbSB3ZWJraXRweS5sYXlvdXRfdGVzdHMuY29udHJvbGxlcnMubGF5b3V0X3Rlc3RfcnVu bmVyIGltcG9ydCBMYXlvdXRUZXN0UnVubmVyCitmcm9tIHdlYmtpdHB5LnRvb2wubW9ja3Rvb2wg aW1wb3J0IE1vY2tPcHRpb25zCitmcm9tIHdlYmtpdHB5LnBvcnQuZmFjdG9yeSBpbXBvcnQgUG9y dEZhY3RvcnkKKworCitfbG9nID0gbG9nZ2luZy5nZXRMb2dnZXIoX19uYW1lX18pCisKKworY2xh c3MgUmViYXNlZFRlc3RzKEFic3RyYWN0U3RlcCk6CisgICAgQGNsYXNzbWV0aG9kCisgICAgZGVm IG9wdGlvbnMoY2xzKToKKyAgICAgICAgcmV0dXJuIEFic3RyYWN0U3RlcC5vcHRpb25zKCkgKyBb CisgICAgICAgICAgICBPcHRpb25zLm5vbl9pbnRlcmFjdGl2ZSwKKyAgICAgICAgXQorCisgICAg ZGVmIF9nZXRfdGVzdF9uYW1lX3N0dWIoc2VsZiwgZXhwZWN0ZWQpOgorICAgICAgICBleHBuYW1l ID0gb3MucGF0aC5zcGxpdGV4dChvcy5wYXRoLmJhc2VuYW1lKGV4cGVjdGVkKSlbMF0KKyAgICAg ICAgcmV0dXJuIGV4cG5hbWVbOi1sZW4oJy1leHBlY3RlZCcpXQorCisgICAgZGVmIF9nZXRfdGVz dF9yZWxhdGl2ZV9kaXIoc2VsZiwgZGlyKToKKyAgICAgICAgYWJzZGlyID0gb3MucGF0aC5kaXJu YW1lKGRpcikKKyAgICAgICAgbWF0Y2ggPSByZS5tYXRjaChvcy5wYXRoLmpvaW4oJ0xheW91dFRl c3RzJywgJ3BsYXRmb3JtJywgJyguKz8pJywgJyguKiknKSwgYWJzZGlyKQorICAgICAgICBpZiBt YXRjaDoKKyAgICAgICAgICAgIHJldHVybiBtYXRjaC5ncm91cHMoKVsxXQorICAgICAgICBlbHNl OgorICAgICAgICAgICAgcmV0dXJuIGFic2RpcgorCisgICAgZGVmIF9nZXRfdGVzdHNfaW5fZGly KHNlbGYsIHBvcnQsIHJlbGRpcik6CisgICAgICAgIHJldHVybiBwb3J0LnRlc3RzKFtzZWxmLl9n ZXRfdGVzdF9yZWxhdGl2ZV9kaXIocmVsZGlyKV0pCisKKyAgICBkZWYgX2dldF90ZXN0c19ieV9l eHBlY3RlZChzZWxmLCBwb3J0LCBleHBlY3RlZCk6CisgICAgICAgIGRpciA9IHNlbGYuX2dldF90 ZXN0c19pbl9kaXIocG9ydCwgZXhwZWN0ZWQpCisgICAgICAgIHJldHVybiBkaXIKKworICAgIGRl ZiBfZ2V0X2ZpcnN0X2dlbmVyaWNfYnlfZmlsZW5hbWUoc2VsZiwgcG9ydCwgZmlsZW5hbWUpOgor ICAgICAgICBwYXRocyA9IHNlbGYuX2dldF90ZXN0c19ieV9leHBlY3RlZChwb3J0LCBmaWxlbmFt ZSkKKyAgICAgICAgZmlsZXN0dWIgPSBzZWxmLl9nZXRfdGVzdF9uYW1lX3N0dWIoZmlsZW5hbWUp CisgICAgICAgIHJldCA9IE5vbmUKKyAgICAgICAgZm9yIHBhdGggaW4gcGF0aHM6CisgICAgICAg ICAgICBoaXQgPSByZS5tYXRjaCgnLionICsgZmlsZXN0dWIgKyAnXC4uKicsIHBhdGgpCisgICAg ICAgICAgICBpZiBoaXQ6CisgICAgICAgICAgICAgICAgcmV0ID0gaGl0Lmdyb3VwKDApCisgICAg ICAgICAgICAgICAgYnJlYWsKKyAgICAgICAgcmV0dXJuIHJldAorCisgICAgZGVmIHJ1bihzZWxm LCBzdGF0ZSk6CisgICAgICAgIF9sb2cuaW5mbygiUHJvY2Vzc2luZyBwYXRjaCAlcyBmcm9tIGJ1 ZyAlcy4iICUgKHN0YXRlWyJwYXRjaCJdLmlkKCksIHN0YXRlWyJwYXRjaCJdLmJ1Z19pZCgpKSkK KworICAgICAgICBob3N0ID0gSG9zdCgpCisgICAgICAgIGhvc3QuaW5pdGlhbGl6ZV9zY20oKQor ICAgICAgICBmYWN0b3J5ID0gUG9ydEZhY3RvcnkoaG9zdCkKKyAgICAgICAgcG9ydCA9IGZhY3Rv cnkuZ2V0KCkKKyAgICAgICAgZ2VuZXJpY19wYXRocyA9IFtdCisKKyAgICAgICAgZm9yIGZpbGVu YW1lIGluIHNlbGYuX2NoYW5nZWRfZmlsZXMoc3RhdGUpOgorICAgICAgICAgICAgaGl0cyA9IHJl Lm1hdGNoKHInKCguKi1leHBlY3RlZFwuKSh0eHR8cG5nKSknLCBmaWxlbmFtZSkKKyAgICAgICAg ICAgIGlmIGhpdHM6CisgICAgICAgICAgICAgICAgZ2VuZXJpY19yZWxwYXRoID0gc2VsZi5fZ2V0 X2ZpcnN0X2dlbmVyaWNfYnlfZmlsZW5hbWUocG9ydCwgZmlsZW5hbWUpCisgICAgICAgICAgICAg ICAgZ2VuZXJpY19wYXRocy5hcHBlbmQoZ2VuZXJpY19yZWxwYXRoKQorICAgICAgICBzdGF0ZVsi dGVzdGxpc3QiXSA9IGdlbmVyaWNfcGF0aHMKZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Vi a2l0cHkvdG9vbC9zdGVwcy9ydW50ZXN0cy5weSBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvdG9v bC9zdGVwcy9ydW50ZXN0cy5weQppbmRleCBhNDU2MjhiLi4wNzY2OGI2IDEwMDY0NAotLS0gYS9U b29scy9TY3JpcHRzL3dlYmtpdHB5L3Rvb2wvc3RlcHMvcnVudGVzdHMucHkKKysrIGIvVG9vbHMv U2NyaXB0cy93ZWJraXRweS90b29sL3N0ZXBzL3J1bnRlc3RzLnB5CkBAIC03Miw3ICs3Miw3IEBA IGNsYXNzIFJ1blRlc3RzKEFic3RyYWN0U3RlcCk6CiAgICAgICAgICAgICAgICAgc2VsZi5fdG9v bC5leGVjdXRpdmUucnVuX2FuZF90aHJvd19pZl9mYWlsKGphdmFzY3JpcHRjb3JlX3Rlc3RzX2Nv bW1hbmQsIHF1aWV0PVRydWUsIGN3ZD1zZWxmLl90b29sLnNjbSgpLmNoZWNrb3V0X3Jvb3QpCiAK ICAgICAgICAgYmluZGluZ3NfdGVzdHNfY29tbWFuZCA9IHNlbGYuX3Rvb2wuZGVwcmVjYXRlZF9w b3J0KCkucnVuX2JpbmRpbmdzX3Rlc3RzX2NvbW1hbmQoKQotICAgICAgICBpZiBiaW5kaW5nc190 ZXN0c19jb21tYW5kOgorICAgICAgICBpZiBiaW5kaW5nc190ZXN0c19jb21tYW5kIGFuZCBub3Qg c2VsZi5fb3B0aW9ucy5sYXlvdXRfdGVzdGxpc3Rfb25seToKICAgICAgICAgICAgIF9sb2cuaW5m bygiUnVubmluZyBiaW5kaW5ncyBnZW5lcmF0aW9uIHRlc3RzIikKICAgICAgICAgICAgIGFyZ3Mg PSBiaW5kaW5nc190ZXN0c19jb21tYW5kCiAgICAgICAgICAgICB0cnk6CkBAIC04MSw3ICs4MSw3 IEBAIGNsYXNzIFJ1blRlc3RzKEFic3RyYWN0U3RlcCk6CiAgICAgICAgICAgICAgICAgX2xvZy5p bmZvKCJFcnJvciBydW5uaW5nIHJ1bi1iaW5kaW5ncy10ZXN0czogJXMiICUgZS5tZXNzYWdlX3dp dGhfb3V0cHV0KCkpCiAKICAgICAgICAgd2Via2l0X3VuaXRfdGVzdHNfY29tbWFuZCA9IHNlbGYu X3Rvb2wuZGVwcmVjYXRlZF9wb3J0KCkucnVuX3dlYmtpdF91bml0X3Rlc3RzX2NvbW1hbmQoKQot ICAgICAgICBpZiB3ZWJraXRfdW5pdF90ZXN0c19jb21tYW5kOgorICAgICAgICBpZiB3ZWJraXRf dW5pdF90ZXN0c19jb21tYW5kIGFuZCBub3Qgc2VsZi5fb3B0aW9ucy5sYXlvdXRfdGVzdGxpc3Rf b25seToKICAgICAgICAgICAgIF9sb2cuaW5mbygiUnVubmluZyBXZWJLaXQgdW5pdCB0ZXN0cyIp CiAgICAgICAgICAgICBhcmdzID0gd2Via2l0X3VuaXRfdGVzdHNfY29tbWFuZAogICAgICAgICAg ICAgdHJ5OgpAQCAtMTA5LDYgKzEwOSw4IEBAIGNsYXNzIFJ1blRlc3RzKEFic3RyYWN0U3RlcCk6 CiAKICAgICAgICAgaWYgc2VsZi5fb3B0aW9ucy5xdWlldDoKICAgICAgICAgICAgIGFyZ3MuYXBw ZW5kKCItLXF1aWV0IikKKyAgICAgICAgaWYgc3RhdGVbInRlc3RsaXN0Il06CisgICAgICAgICAg ICBhcmdzLmV4dGVuZChzdGF0ZVsidGVzdGxpc3QiXSkKIAogICAgICAgICBzZWxmLl90b29sLmV4 ZWN1dGl2ZS5ydW5fYW5kX3Rocm93X2lmX2ZhaWwoYXJncywgY3dkPXNlbGYuX3Rvb2wuc2NtKCku Y2hlY2tvdXRfcm9vdCkKICAgICAgICAgCg==