51894 2011-01-04 14:03:54 -0800 [Qt] run-qtwebkit-tests needs timeout 2011-01-07 07:28:53 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) All All RESOLVED FIXED Qt, QtTriaged P1 Blocker --- 1 ossy jedrzej.nowacki commit-queue galpeter jedrzej.nowacki rgabor wsiegrist oldest_to_newest 329196 0 ossy 2011-01-04 14:03:54 -0800 Nowadays run-qtwebkit-tests hangs regularly on buildbots. It should have timeout and kill the bad testcases. INFO:Exec:Running... WebKitBuild/Release/WebKit/qt/tests/benchmarks/painting/tst_painting command timed out: 1200 seconds without output, killing pid .... I prefer run-qtwebkit-tests kills the tst_culprit_test in 1-2 minutes to buildbot kills run-qtwebkit-tests after 20 minutes. 329460 1 jedrzej.nowacki 2011-01-05 00:06:45 -0800 Sure, the script should be able to kill a test, but in the first place the test should not hang, it have to fail ASAP ;-). I'm going to add the new functionality. 329474 2 jedrzej.nowacki 2011-01-05 01:01:49 -0800 Just to be precise, you mean kill a bad testsuite, not a testcase, right? I would describe them as: Testsuite is something like tst_painting Testcase is something like tst_painting::paint() QTestLib doesn't provide timeout functionality. I may change run-qtwebkit-tests script so it can kill a subprocess (which is a testsuite) after specified timeout. But that solution is essentially similar to kill the script by buildbot. If you want to kill a testcase it would be much more complex, as the script have to run each testcase in a separate process. That may introduce an issue; results of running tests manually and by the script could be different, because of possible dependency between testcases. 329503 3 ossy 2011-01-05 04:06:06 -0800 (In reply to comment #2) > Just to be precise, you mean kill a bad testsuite, not a testcase, right? > > I would describe them as: > Testsuite is something like tst_painting > Testcase is something like tst_painting::paint() I was a little ambiguous, I mean killing the bad testsuite, of course. > QTestLib doesn't provide timeout functionality. I may change > run-qtwebkit-tests script so it can kill a subprocess (which > is a testsuite) after specified timeout. But that solution is > essentially similar to kill the script by buildbot. It would be absolutely correct solution. ;) I only would like to ensure that run-qtwebkit-tests won't block a buildslave for 20 minutes if something goes wrong in a testsuite. Now the runtime in parallel mode is ~14-15 seconds and ~38-40 seconds with single thread. So I think a 30 seconds timeout would be more than enough for each testsuite. Otherwise I supsect a network error/timeout might cause this problem, because the occurrences aren't random, but constant in a random (max. 1-2 hours long) time interval. 330141 4 78120 jedrzej.nowacki 2011-01-06 07:40:09 -0800 Created attachment 78120 patch I do not set cq to '?' as some configuration changes may by applied on the buildbot side. 330147 5 78120 ossy 2011-01-06 08:06:14 -0800 Comment on attachment 78120 patch View in context: https://bugs.webkit.org/attachment.cgi?id=78120&action=review I got many errors like this: Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner self.run() File "/usr/lib/python2.5/threading.py", line 663, in run self.function(*self.args, **self.kwargs) File "Tools/Scripts/run-qtwebkit-tests", line 133, in process_killer tst.terminate() AttributeError: 'Popen' object has no attribute 'terminate' > Tools/Scripts/run-qtwebkit-tests:72 > + opt.add_option("-t", "--timeout", action="store", type="int", > + dest="timeout", default=0, > + help="Timeout in seconds for each testsuite. Zero value means that there is not timeout. Default: %default.") I prefer 30 seconds as default value. 330149 6 galpeter 2011-01-06 08:08:25 -0800 Just some minor things: > Tools/Scripts/run-qtwebkit-tests:125 > + tst = Popen([test_suite.test_file_name(), ] + options.split(), stdout=PIPE, stderr=None, shell=False) The comma inside the brackets isn't required, it will be a list without it too. Also the shell=False is unnecessary, that is the default behavior for Popen. > Tools/Scripts/run-qtwebkit-tests:127 > + tst = Popen([test_suite.test_file_name(), ] + options.split(), stdout=None, stderr=STDOUT, shell=False) ditto. 330150 7 galpeter 2011-01-06 08:10:13 -0800 (In reply to comment #5) > (From update of attachment 78120 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=78120&action=review > > I got many errors like this: > > Exception in thread Thread-1: > Traceback (most recent call last): > File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner > self.run() > File "/usr/lib/python2.5/threading.py", line 663, in run > self.function(*self.args, **self.kwargs) > File "Tools/Scripts/run-qtwebkit-tests", line 133, in process_killer > tst.terminate() > AttributeError: 'Popen' object has no attribute 'terminate' Popen's terminate() method is only available from python 2.6 330152 8 galpeter 2011-01-06 08:15:31 -0800 > Tools/Scripts/run-qtwebkit-tests:133 > + log.debug("Setting timeout timer %i sec on %s (process %s)", timeout, test_suite.test_file_name(), tst.pid) > + tst.terminate() maybe 'os.kill(tst.pid, signal.SIGTERM)' could resolve the problem for python 2.5 330159 9 78120 ossy 2011-01-06 08:30:29 -0800 Comment on attachment 78120 patch Please fix issues mentioned in Comment #6 and Comment #8 and add 30 seconds default timeout and I'll give an r+. 330676 10 jedrzej.nowacki 2011-01-07 00:59:04 -0800 (In reply to comment #5) > I prefer 30 seconds as default value. I don't think that it is a good idea. 30 sec is good for buildbot, but is not good enough for others. For example I need 2 min to run tests, more in debug mode and It takes only a few "weeks" on my old laptop :-). Personally I prefer to be not bothered by timeouts. (In reply to comment #6) > The comma inside the brackets isn't required, it will be a list without it too. Also the shell=False is unnecessary, that is the default behavior for Popen. True. (In reply to comment #8) > maybe 'os.kill(tst.pid, signal.SIGTERM)' could resolve the problem for python 2.5 It won't work on windows until python 2.7 :/. Anyway I'm not aware of any Windows user who is using the script. Probably one of http://code.activestate.com/recipes/347462-terminating-a-subprocess-on-windows/ solutions should be used instead, but I don't have environment to check. 330678 11 jedrzej.nowacki 2011-01-07 01:08:00 -0800 (In reply to comment #10) > (In reply to comment #8) > > maybe 'os.kill(tst.pid, signal.SIGTERM)' could resolve the problem for python 2.5 > It won't work on windows until python 2.7 :/. Anyway I'm not aware of any Windows user who is using the script. Probably one of http://code.activestate.com/recipes/347462-terminating-a-subprocess-on-windows/ solutions should be used instead, but I don't have environment to check. I would write something like that: try: tst.terminate() except AttributeError: # python version < 2.6 os.kill(tst.pid, signal.SIGTERM) # FIXME: Windows support what do you think? Is it enough? 330693 12 galpeter 2011-01-07 01:57:35 -0800 (In reply to comment #11) > (In reply to comment #10) > > (In reply to comment #8) > > > maybe 'os.kill(tst.pid, signal.SIGTERM)' could resolve the problem for python 2.5 > > It won't work on windows until python 2.7 :/. Anyway I'm not aware of any Windows user who is using the script. Probably one of http://code.activestate.com/recipes/347462-terminating-a-subprocess-on-windows/ solutions should be used instead, but I don't have environment to check. > > I would write something like that: > > try: > tst.terminate() > except AttributeError: > # python version < 2.6 > os.kill(tst.pid, signal.SIGTERM) > # FIXME: Windows support > > what do you think? Is it enough? What if we check the python version? ( http://docs.python.org/library/platform.html#platform.python_version_tuple ) I personally don't like exceptions if I can found an other way for it. so maybe like this: ver = platform.python_version_tuple() if ver[0] == 2 and ver[1] < 6: if platform.system == "win32": # win .. else: os.kill(tst.pid, signal.SIGTERM) else: tst.terminate() 330721 13 ossy 2011-01-07 03:39:52 -0800 (In reply to comment #10) > I don't think that it is a good idea. 30 sec is good for buildbot, but is not good enough for others. For example I need 2 min to run tests, more in debug mode and It takes only a few "weeks" on my old laptop :-). Personally I prefer to be not bothered by timeouts. OK, you're right. Please add "--timeout 30" to Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg instead of the default timeout to make buildbots happy. (It must be enough, because the maximum runtime of a test suite is 10-12 seconds on the bot.) (In reply to comment #12) > What if we check the python version? ( http://docs.python.org/library/platform.html#platform.python_version_tuple ) > I personally don't like exceptions if I can found an other way for it. > > so maybe like this: > > ver = platform.python_version_tuple() > if ver[0] == 2 and ver[1] < 6: > if platform.system == "win32": > # win > .. > else: > os.kill(tst.pid, signal.SIGTERM) > else: > tst.terminate() It looks good to me with one of the windows implementation mentioned in comment #10 . 330724 14 jedrzej.nowacki 2011-01-07 04:07:13 -0800 (In reply to comment #12) > What if we check the python version? ( http://docs.python.org/library/platform.html#platform.python_version_tuple ) > I personally don't like exceptions if I can found an other way for it. You shouldn't avoid exceptions, it is nice feature that simplify your code ;-). Python encourages usage of exceptions. Please check http://docs.python.org/glossary.html#term-eafp 330726 15 78220 jedrzej.nowacki 2011-01-07 04:09:36 -0800 Created attachment 78220 Patch I haven't tested that on Win, but I checked docs and it should work(TM) :-) 330733 16 78220 commit-queue 2011-01-07 05:07:34 -0800 Comment on attachment 78220 Patch Clearing flags on attachment: 78220 Committed r75240: <http://trac.webkit.org/changeset/75240> 330734 17 commit-queue 2011-01-07 05:07:41 -0800 All reviewed patches have been landed. Closing bug. 330739 18 ossy 2011-01-07 05:39:20 -0800 Bill, could you update the master to make our bot happier, please? 330766 19 wsiegrist 2011-01-07 07:28:53 -0800 Buildbot updated. 78120 2011-01-06 07:40:09 -0800 2011-01-07 04:09:36 -0800 patch run.diff text/plain 4245 jedrzej.nowacki ZGlmZiAtLWdpdCBhL1Rvb2xzL0NoYW5nZUxvZyBiL1Rvb2xzL0NoYW5nZUxvZwppbmRleCAyZjY2 YTIzLi4yYjRmYmY5IDEwMDY0NAotLS0gYS9Ub29scy9DaGFuZ2VMb2cKKysrIGIvVG9vbHMvQ2hh bmdlTG9nCkBAIC0xLDMgKzEsMTggQEAKKzIwMTEtMDEtMDYgIEplZHJ6ZWogTm93YWNraSAgPGpl ZHJ6ZWoubm93YWNraUBub2tpYS5jb20+CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChP T1BTISkuCisKKyAgICAgICAgcnVuLXF0d2Via2l0LXRlc3RzIHNob3VsZCBiZSBhYmxlIHRvIGtp bGwgYSB0ZXN0c3VpdGUuCisKKyAgICAgICAgQWRkIG5ldyBvcHRpb24gaW4gdGhlIHNjcmlwdCB0 aGF0IGNhbiBzZXR1cCBhIHRpbWVvdXQgZm9yIGEgdGVzdC4KKyAgICAgICAgSWYgdGhlIHRlc3Qg ZXhlY3V0aW9uIHRha2VzIG1vcmUgdGhlbiBzcGVjaWZpZWQgdGltZSB0aGVuIHRoZSB0ZXN0Cisg ICAgICAgIHdvdWxkIGJlIHRlcm1pbmF0ZWQuCisKKyAgICAgICAgW1F0XSBydW4tcXR3ZWJraXQt dGVzdHMgbmVlZHMgdGltZW91dAorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93 X2J1Zy5jZ2k/aWQ9NTE4OTQKKworICAgICAgICAqIFNjcmlwdHMvcnVuLXF0d2Via2l0LXRlc3Rz OgorCiAyMDExLTAxLTA2ICBBZGFtIEJhcnRoICA8YWJhcnRoQHdlYmtpdC5vcmc+CiAKICAgICAg ICAgUmV2aWV3ZWQgYnkgRXJpYyBTZWlkZWwuCmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRzL3J1 bi1xdHdlYmtpdC10ZXN0cyBiL1Rvb2xzL1NjcmlwdHMvcnVuLXF0d2Via2l0LXRlc3RzCmluZGV4 IDM3M2RlMGEuLjYzNjUyOTYgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1NjcmlwdHMvcnVuLXF0d2Via2l0 LXRlc3RzCisrKyBiL1Rvb2xzL1NjcmlwdHMvcnVuLXF0d2Via2l0LXRlc3RzCkBAIC02Nyw2ICs2 Nyw5IEBAIGNsYXNzIE9wdGlvbnMoTG9nKToKICAgICAgICAgb3B0LmFkZF9vcHRpb24oIi1kIiwg Ii0tZGV2ZWxvcGVyLW1vZGUiLCBhY3Rpb249InN0b3JlX3RydWUiLAogICAgICAgICAgICAgICBk ZXN0PSJkZXZlbG9wZXIiLCBkZWZhdWx0PUZhbHNlLAogICAgICAgICAgICAgICBoZWxwPSJTcGVj aWFsIG1vZGUgZm9yIGRlYnVnZ2luZy4gSW4gZ2VuZXJhbCBpdCBzaW11bGF0ZXMgaHVtYW4gYmVo YXZpb3IsIHJ1bm5pbmcgYWxsIGF1dG90ZXN0cy4gSW4gdGhlIG1vZGUgZXZlcnl0aGluZyBpcyBl eGVjdXRlZCBzeW5jaHJvbm91c2x5LCBubyBodG1sIG91dHB1dCB3aWxsIGJlIGdlbmVyYXRlZCwg bm8gY2hhbmdlcyBvciB0cmFuc2Zvcm1hdGlvbiB3aWxsIGJlIGFwcGxpZWQgdG8gc3RkZXJyIG9y IHN0ZG91dC4gSW4gdGhpcyBtb2RlIG9wdGlvbnM7IHBhcmFsbGVsLWxldmVsLCBvdXRwdXQtZmls ZSwgYnJvd3NlciBhbmQgZG8tbm90LW9wZW4tcmVzdWx0cyB3aWxsIGJlIGlnbm9yZWQuIikKKyAg ICAgICAgb3B0LmFkZF9vcHRpb24oIi10IiwgIi0tdGltZW91dCIsIGFjdGlvbj0ic3RvcmUiLCB0 eXBlPSJpbnQiLAorICAgICAgICAgICAgICBkZXN0PSJ0aW1lb3V0IiwgZGVmYXVsdD0wLAorICAg ICAgICAgICAgICBoZWxwPSJUaW1lb3V0IGluIHNlY29uZHMgZm9yIGVhY2ggdGVzdHN1aXRlLiBa ZXJvIHZhbHVlIG1lYW5zIHRoYXQgdGhlcmUgaXMgbm90IHRpbWVvdXQuIERlZmF1bHQ6ICVkZWZh dWx0LiIpCiAKICAgICAgICAgc2VsZi5fbywgc2VsZi5fYSA9IG9wdC5wYXJzZV9hcmdzKGFyZ3Mp CiAgICAgICAgIHZlcmJvc2UgPSBzZWxmLl9vLnZlcmJvc2UKQEAgLTExMSwxNSArMTE0LDI5IEBA IGRlZiBydW5fdGVzdChhcmdzKToKICAgICAgIHN0ciB3aXRoIG9wdGlvbnMgdGhhdCBzaG91bGQg YmUgcGFzc2VkIHRvIHRoZSBhdXRvdGVzdCBleGVjdXRhYmxlCiAgICAgICBib29sIGlmIHRydWUg dGhlbiB0aGUgc3Rkb3V0IHdpbGwgYmUgYnVmZmVyZWQgYW5kIHNlcGFyYXRlZCBmcm9tIHRoZSBz dGRlcnIsIGlmIGl0IGlzIGZhbHNlCiAgICAgICAgIHRoZW4gdGhlIHN0ZG91dCBhbmQgdGhlIHN0 ZGVyciB3aWxsIGJlIG1lcmdlZCB0b2dldGhlciBhbmQgbGVmdCB1bmJ1ZmZlcmVkICh0aGUgVGVz dFN1aXRlUmVzdWx0IG91dHB1dCB3aWxsIGJlIE5vbmUpLgorICAgICAgaW50IHRpbWUgYWZ0ZXIg d2hpY2ggdGhlIGF1dG90ZXN0IGV4ZWN1dGFibGUgd291bGQgYmUga2lsbGVkCiAgICAgIiIiCiAg ICAgbG9nID0gbG9nZ2luZy5nZXRMb2dnZXIoIkV4ZWMiKQotICAgIHRlc3Rfc3VpdGUsIG9wdGlv bnMsIGJ1ZmZlcmVkID0gYXJncworICAgIHRlc3Rfc3VpdGUsIG9wdGlvbnMsIGJ1ZmZlcmVkLCB0 aW1lb3V0ID0gYXJncworICAgIHRpbWVyID0gTm9uZQogICAgIHRyeToKICAgICAgICAgbG9nLmlu Zm8oIlJ1bm5pbmcuLi4gJXMiLCB0ZXN0X3N1aXRlLnRlc3RfZmlsZV9uYW1lKCkpCiAgICAgICAg IGlmIGJ1ZmZlcmVkOgotICAgICAgICAgICAgdHN0ID0gUG9wZW4odGVzdF9zdWl0ZS50ZXN0X2Zp bGVfbmFtZSgpICsgb3B0aW9ucywgc3Rkb3V0PVBJUEUsIHN0ZGVycj1Ob25lLCBzaGVsbD1UcnVl KQorICAgICAgICAgICAgdHN0ID0gUG9wZW4oW3Rlc3Rfc3VpdGUudGVzdF9maWxlX25hbWUoKSwg XSArIG9wdGlvbnMuc3BsaXQoKSwgc3Rkb3V0PVBJUEUsIHN0ZGVycj1Ob25lLCBzaGVsbD1GYWxz ZSkKICAgICAgICAgZWxzZToKLSAgICAgICAgICAgIHRzdCA9IFBvcGVuKHRlc3Rfc3VpdGUudGVz dF9maWxlX25hbWUoKSArIG9wdGlvbnMsIHN0ZG91dD1Ob25lLCBzdGRlcnI9U1RET1VULCBzaGVs bD1UcnVlKQorICAgICAgICAgICAgdHN0ID0gUG9wZW4oW3Rlc3Rfc3VpdGUudGVzdF9maWxlX25h bWUoKSwgXSArIG9wdGlvbnMuc3BsaXQoKSwgc3Rkb3V0PU5vbmUsIHN0ZGVycj1TVERPVVQsIHNo ZWxsPUZhbHNlKQorICAgICAgICBpZiB0aW1lb3V0OgorICAgICAgICAgICAgZnJvbSB0aHJlYWRp bmcgaW1wb3J0IFRpbWVyCisgICAgICAgICAgICBsb2cuZGVidWcoIlNldHRpbmcgdGltZW91dCB0 aW1lciAlaSBzZWMgb24gJXMgKHByb2Nlc3MgJXMpIiwgdGltZW91dCwgdGVzdF9zdWl0ZS50ZXN0 X2ZpbGVfbmFtZSgpLCB0c3QucGlkKQorICAgICAgICAgICAgZGVmIHByb2Nlc3Nfa2lsbGVyKCk6 CisgICAgICAgICAgICAgICAgdHJ5OgorICAgICAgICAgICAgICAgICAgICB0c3QudGVybWluYXRl KCkKKyAgICAgICAgICAgICAgICAgICAgbG9nLmVycm9yKCJUaW1lb3V0LCBwcm9jZXNzICclcycg KCVpKSB3YXMgdGVybWluYXRlZCIsIHRlc3Rfc3VpdGUudGVzdF9maWxlX25hbWUoKSwgdHN0LnBp ZCkKKyAgICAgICAgICAgICAgICBleGNlcHQgT1NFcnJvciwgZToKKyAgICAgICAgICAgICAgICAg ICAgIyB0aGUgcHJvY2VzcyB3YXMgZmluaXNoZWQgYmVmb3JlIGdvdCBraWxsZWQKKyAgICAgICAg ICAgICAgICAgICAgcGFzcworICAgICAgICAgICAgdGltZXIgPSBUaW1lcih0aW1lb3V0LCBwcm9j ZXNzX2tpbGxlcikKKyAgICAgICAgICAgIHRpbWVyLnN0YXJ0KCkKICAgICBleGNlcHQgT1NFcnJv ciwgZToKICAgICAgICAgbG9nLmV4Y2VwdGlvbigiQ2FuJ3Qgb3BlbiBhbiBhdXRvdGVzdCBmaWxl OiAnJXMnLiBTa2lwcGluZyB0aGUgdGVzdC4uLiIsIGUuZmlsZW5hbWUpCiAgICAgZWxzZToKQEAg LTIwNyw3ICsyMjQsNyBAQCBjbGFzcyBNYWluKExvZyk6CiAgICAgICAgIHdvcmtlcnMgPSBzZWxm Ll9Qb29sKHByb2Nlc3Nlcz1zZWxmLl9vcHRpb25zLnBhcmFsbGVsX2xldmVsKQogICAgICAgICAj IHRvIGVhY2ggZmlsZSBhZGQgb3B0aW9ucy4KICAgICAgICAgc2VsZi5kZWJ1ZygiVXNpbmcgJXMg dGhlIHdvcmtlcnMgcG9vbCwgbnVtYmVyIG9mIHdvcmtlcnMgJWkiLCByZXByKHdvcmtlcnMpLCBz ZWxmLl9vcHRpb25zLnBhcmFsbGVsX2xldmVsKQotICAgICAgICBwYWNrYWdlID0gbWFwKGxhbWJk YSB3OiBbdywgc2VsZi5fb3B0aW9ucy50ZXN0c19vcHRpb25zLCBub3Qgc2VsZi5fb3B0aW9ucy5k ZXZlbG9wZXJdLCBmaWxlcykKKyAgICAgICAgcGFja2FnZSA9IG1hcChsYW1iZGEgdzogW3csIHNl bGYuX29wdGlvbnMudGVzdHNfb3B0aW9ucywgbm90IHNlbGYuX29wdGlvbnMuZGV2ZWxvcGVyLCBz ZWxmLl9vcHRpb25zLnRpbWVvdXRdLCBmaWxlcykKICAgICAgICAgc2VsZi5kZWJ1ZygiR2VuZXJh dGVkIHBhY2thZ2VzIGZvciB3b3JrZXJzOiAlcyIsIHJlcHIocGFja2FnZSkpCiAgICAgICAgIHJl c3VsdHMgPSB3b3JrZXJzLm1hcChydW5fdGVzdCwgcGFja2FnZSkgICMgQ29sbGVjdHMgcmVzdWx0 cy4KICAgICAgICAgcmV0dXJuIHJlc3VsdHMK 78220 2011-01-07 04:09:36 -0800 2011-01-07 05:07:34 -0800 Patch tmp.diff text/plain 5871 jedrzej.nowacki ZGlmZiAtLWdpdCBhL1Rvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29u ZmlnL21hc3Rlci5jZmcgYi9Ub29scy9CdWlsZFNsYXZlU3VwcG9ydC9idWlsZC53ZWJraXQub3Jn LWNvbmZpZy9tYXN0ZXIuY2ZnCmluZGV4IDFjMjlkMDkuLjFjZDU2YWQgMTAwNjQ0Ci0tLSBhL1Rv b2xzL0J1aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL21hc3Rlci5jZmcK KysrIGIvVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvbWFz dGVyLmNmZwpAQCAtMzEyLDcgKzMxMiw3IEBAIGNsYXNzIFJ1blF0QVBJVGVzdHMoc2hlbGwuVGVz dCk6CiAgICAgZGVzY3JpcHRpb24gPSBbIkFQSSB0ZXN0cyBydW5uaW5nIl0KICAgICBkZXNjcmlw dGlvbkRvbmUgPSBbIkFQSSB0ZXN0cyJdCiAgICAgY29tbWFuZCA9IFsicHl0aG9uIiwgIi4vVG9v bHMvU2NyaXB0cy9ydW4tcXR3ZWJraXQtdGVzdHMiLAotICAgICAgICAgICAgICAgIi0tb3V0cHV0 LWZpbGU9cXQtdW5pdC10ZXN0cy5odG1sIiwgIi0tZG8tbm90LW9wZW4tcmVzdWx0cyIsCisgICAg ICAgICAgICAgICAiLS1vdXRwdXQtZmlsZT1xdC11bml0LXRlc3RzLmh0bWwiLCAiLS1kby1ub3Qt b3Blbi1yZXN1bHRzIiwgIi0tdGltZW91dD0zMCIsCiAgICAgICAgICAgICAgICBXaXRoUHJvcGVy dGllcygiV2ViS2l0QnVpbGQvJShjb25maWd1cmF0aW9uX3ByZXR0eSlzL1dlYktpdC9xdC90ZXN0 cy8iKV0KIAogICAgIGRlZiBzdGFydChzZWxmKToKZGlmZiAtLWdpdCBhL1Rvb2xzL0NoYW5nZUxv ZyBiL1Rvb2xzL0NoYW5nZUxvZwppbmRleCAyZjY2YTIzLi4wMjM2Zjk0IDEwMDY0NAotLS0gYS9U b29scy9DaGFuZ2VMb2cKKysrIGIvVG9vbHMvQ2hhbmdlTG9nCkBAIC0xLDMgKzEsMTkgQEAKKzIw MTEtMDEtMDcgIEplZHJ6ZWogTm93YWNraSAgPGplZHJ6ZWoubm93YWNraUBub2tpYS5jb20+CisK KyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgcnVuLXF0d2Vi a2l0LXRlc3RzIHNob3VsZCBiZSBhYmxlIHRvIGtpbGwgYSB0ZXN0c3VpdGUuCisKKyAgICAgICAg QWRkIG5ldyBvcHRpb24gaW4gdGhlIHNjcmlwdCB0aGF0IGNhbiBzZXR1cCBhIHRpbWVvdXQgZm9y IGEgdGVzdC4KKyAgICAgICAgSWYgdGhlIHRlc3QgZXhlY3V0aW9uIHRha2VzIG1vcmUgdGhlbiBz cGVjaWZpZWQgdGltZSB0aGVuIHRoZSB0ZXN0CisgICAgICAgIHdvdWxkIGJlIHRlcm1pbmF0ZWQu CisKKyAgICAgICAgW1F0XSBydW4tcXR3ZWJraXQtdGVzdHMgbmVlZHMgdGltZW91dAorICAgICAg ICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9NTE4OTQKKworICAgICAg ICAqIEJ1aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL21hc3Rlci5jZmc6 CisgICAgICAgICogU2NyaXB0cy9ydW4tcXR3ZWJraXQtdGVzdHM6CisKIDIwMTEtMDEtMDYgIEFk YW0gQmFydGggIDxhYmFydGhAd2Via2l0Lm9yZz4KIAogICAgICAgICBSZXZpZXdlZCBieSBFcmlj IFNlaWRlbC4KZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvcnVuLXF0d2Via2l0LXRlc3RzIGIv VG9vbHMvU2NyaXB0cy9ydW4tcXR3ZWJraXQtdGVzdHMKaW5kZXggMzczZGUwYS4uNzdiZGQwNCAx MDA2NDQKLS0tIGEvVG9vbHMvU2NyaXB0cy9ydW4tcXR3ZWJraXQtdGVzdHMKKysrIGIvVG9vbHMv U2NyaXB0cy9ydW4tcXR3ZWJraXQtdGVzdHMKQEAgLTY3LDYgKzY3LDkgQEAgY2xhc3MgT3B0aW9u cyhMb2cpOgogICAgICAgICBvcHQuYWRkX29wdGlvbigiLWQiLCAiLS1kZXZlbG9wZXItbW9kZSIs IGFjdGlvbj0ic3RvcmVfdHJ1ZSIsCiAgICAgICAgICAgICAgIGRlc3Q9ImRldmVsb3BlciIsIGRl ZmF1bHQ9RmFsc2UsCiAgICAgICAgICAgICAgIGhlbHA9IlNwZWNpYWwgbW9kZSBmb3IgZGVidWdn aW5nLiBJbiBnZW5lcmFsIGl0IHNpbXVsYXRlcyBodW1hbiBiZWhhdmlvciwgcnVubmluZyBhbGwg YXV0b3Rlc3RzLiBJbiB0aGUgbW9kZSBldmVyeXRoaW5nIGlzIGV4ZWN1dGVkIHN5bmNocm9ub3Vz bHksIG5vIGh0bWwgb3V0cHV0IHdpbGwgYmUgZ2VuZXJhdGVkLCBubyBjaGFuZ2VzIG9yIHRyYW5z Zm9ybWF0aW9uIHdpbGwgYmUgYXBwbGllZCB0byBzdGRlcnIgb3Igc3Rkb3V0LiBJbiB0aGlzIG1v ZGUgb3B0aW9uczsgcGFyYWxsZWwtbGV2ZWwsIG91dHB1dC1maWxlLCBicm93c2VyIGFuZCBkby1u b3Qtb3Blbi1yZXN1bHRzIHdpbGwgYmUgaWdub3JlZC4iKQorICAgICAgICBvcHQuYWRkX29wdGlv bigiLXQiLCAiLS10aW1lb3V0IiwgYWN0aW9uPSJzdG9yZSIsIHR5cGU9ImludCIsCisgICAgICAg ICAgICAgIGRlc3Q9InRpbWVvdXQiLCBkZWZhdWx0PTAsCisgICAgICAgICAgICAgIGhlbHA9IlRp bWVvdXQgaW4gc2Vjb25kcyBmb3IgZWFjaCB0ZXN0c3VpdGUuIFplcm8gdmFsdWUgbWVhbnMgdGhh dCB0aGVyZSBpcyBub3QgdGltZW91dC4gRGVmYXVsdDogJWRlZmF1bHQuIikKIAogICAgICAgICBz ZWxmLl9vLCBzZWxmLl9hID0gb3B0LnBhcnNlX2FyZ3MoYXJncykKICAgICAgICAgdmVyYm9zZSA9 IHNlbGYuX28udmVyYm9zZQpAQCAtMTExLDE1ICsxMTQsNDIgQEAgZGVmIHJ1bl90ZXN0KGFyZ3Mp OgogICAgICAgc3RyIHdpdGggb3B0aW9ucyB0aGF0IHNob3VsZCBiZSBwYXNzZWQgdG8gdGhlIGF1 dG90ZXN0IGV4ZWN1dGFibGUKICAgICAgIGJvb2wgaWYgdHJ1ZSB0aGVuIHRoZSBzdGRvdXQgd2ls bCBiZSBidWZmZXJlZCBhbmQgc2VwYXJhdGVkIGZyb20gdGhlIHN0ZGVyciwgaWYgaXQgaXMgZmFs c2UKICAgICAgICAgdGhlbiB0aGUgc3Rkb3V0IGFuZCB0aGUgc3RkZXJyIHdpbGwgYmUgbWVyZ2Vk IHRvZ2V0aGVyIGFuZCBsZWZ0IHVuYnVmZmVyZWQgKHRoZSBUZXN0U3VpdGVSZXN1bHQgb3V0cHV0 IHdpbGwgYmUgTm9uZSkuCisgICAgICBpbnQgdGltZSBhZnRlciB3aGljaCB0aGUgYXV0b3Rlc3Qg ZXhlY3V0YWJsZSB3b3VsZCBiZSBraWxsZWQKICAgICAiIiIKICAgICBsb2cgPSBsb2dnaW5nLmdl dExvZ2dlcigiRXhlYyIpCi0gICAgdGVzdF9zdWl0ZSwgb3B0aW9ucywgYnVmZmVyZWQgPSBhcmdz CisgICAgdGVzdF9zdWl0ZSwgb3B0aW9ucywgYnVmZmVyZWQsIHRpbWVvdXQgPSBhcmdzCisgICAg dGltZXIgPSBOb25lCiAgICAgdHJ5OgogICAgICAgICBsb2cuaW5mbygiUnVubmluZy4uLiAlcyIs IHRlc3Rfc3VpdGUudGVzdF9maWxlX25hbWUoKSkKICAgICAgICAgaWYgYnVmZmVyZWQ6Ci0gICAg ICAgICAgICB0c3QgPSBQb3Blbih0ZXN0X3N1aXRlLnRlc3RfZmlsZV9uYW1lKCkgKyBvcHRpb25z LCBzdGRvdXQ9UElQRSwgc3RkZXJyPU5vbmUsIHNoZWxsPVRydWUpCisgICAgICAgICAgICB0c3Qg PSBQb3BlbihbdGVzdF9zdWl0ZS50ZXN0X2ZpbGVfbmFtZSgpXSArIG9wdGlvbnMuc3BsaXQoKSwg c3Rkb3V0PVBJUEUsIHN0ZGVycj1Ob25lKQogICAgICAgICBlbHNlOgotICAgICAgICAgICAgdHN0 ID0gUG9wZW4odGVzdF9zdWl0ZS50ZXN0X2ZpbGVfbmFtZSgpICsgb3B0aW9ucywgc3Rkb3V0PU5v bmUsIHN0ZGVycj1TVERPVVQsIHNoZWxsPVRydWUpCisgICAgICAgICAgICB0c3QgPSBQb3Blbihb dGVzdF9zdWl0ZS50ZXN0X2ZpbGVfbmFtZSgpXSArIG9wdGlvbnMuc3BsaXQoKSwgc3Rkb3V0PU5v bmUsIHN0ZGVycj1TVERPVVQpCisgICAgICAgIGlmIHRpbWVvdXQ6CisgICAgICAgICAgICBmcm9t IHRocmVhZGluZyBpbXBvcnQgVGltZXIKKyAgICAgICAgICAgIGxvZy5kZWJ1ZygiU2V0dGluZyB0 aW1lb3V0IHRpbWVyICVpIHNlYyBvbiAlcyAocHJvY2VzcyAlcykiLCB0aW1lb3V0LCB0ZXN0X3N1 aXRlLnRlc3RfZmlsZV9uYW1lKCksIHRzdC5waWQpCisgICAgICAgICAgICBkZWYgcHJvY2Vzc19r aWxsZXIoKToKKyAgICAgICAgICAgICAgICB0cnk6CisgICAgICAgICAgICAgICAgICAgIHRyeToK KyAgICAgICAgICAgICAgICAgICAgICAgIHRzdC50ZXJtaW5hdGUoKQorICAgICAgICAgICAgICAg ICAgICBleGNlcHQgQXR0cmlidXRlRXJyb3I6CisgICAgICAgICAgICAgICAgICAgICAgICAjIFdv cmthcm91bmQgZm9yIHB5dGhvbiB2ZXJzaW9uIDwgMi42IGl0IGNhbiBiZSByZW1vdmVkIGFzIHNv b24gYXMgd2UgZHJvcCBzdXBwb3J0IGZvciBweXRob24yLjUKKyAgICAgICAgICAgICAgICAgICAg ICAgIHRyeToKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICBpbXBvcnQgY3R5cGVzCisgICAg ICAgICAgICAgICAgICAgICAgICAgICAgUFJPQ0VTU19URVJNSU5BVEUgPSAxCisgICAgICAgICAg ICAgICAgICAgICAgICAgICAgaGFuZGxlID0gY3R5cGVzLndpbmRsbC5rZXJuZWwzMi5PcGVuUHJv Y2VzcyhQUk9DRVNTX1RFUk1JTkFURSwgRmFsc2UsIHRzdC5waWQpCisgICAgICAgICAgICAgICAg ICAgICAgICAgICAgY3R5cGVzLndpbmRsbC5rZXJuZWwzMi5UZXJtaW5hdGVQcm9jZXNzKGhhbmRs ZSwgLTEpCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgY3R5cGVzLndpbmRsbC5rZXJuZWwz Mi5DbG9zZUhhbmRsZShoYW5kbGUpCisgICAgICAgICAgICAgICAgICAgICAgICBleGNlcHQgQXR0 cmlidXRlRXJyb3I6CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgIyB3aW5kbGwgaXMgbm90 IGFjY2Vzc2libGUgc28gd2UgYXJlIG9uICpuaXggbGlrZSBzeXN0ZW0KKyAgICAgICAgICAgICAg ICAgICAgICAgICAgICBpbXBvcnQgc2lnbmFsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAg b3Mua2lsbCh0c3QucGlkLCBzaWduYWwuU0lHVEVSTSkKKyAgICAgICAgICAgICAgICAgICAgbG9n LmVycm9yKCJUaW1lb3V0LCBwcm9jZXNzICclcycgKCVpKSB3YXMgdGVybWluYXRlZCIsIHRlc3Rf c3VpdGUudGVzdF9maWxlX25hbWUoKSwgdHN0LnBpZCkKKyAgICAgICAgICAgICAgICBleGNlcHQg T1NFcnJvciwgZToKKyAgICAgICAgICAgICAgICAgICAgIyB0aGUgcHJvY2VzcyB3YXMgZmluaXNo ZWQgYmVmb3JlIGdvdCBraWxsZWQKKyAgICAgICAgICAgICAgICAgICAgcGFzcworICAgICAgICAg ICAgdGltZXIgPSBUaW1lcih0aW1lb3V0LCBwcm9jZXNzX2tpbGxlcikKKyAgICAgICAgICAgIHRp bWVyLnN0YXJ0KCkKICAgICBleGNlcHQgT1NFcnJvciwgZToKICAgICAgICAgbG9nLmV4Y2VwdGlv bigiQ2FuJ3Qgb3BlbiBhbiBhdXRvdGVzdCBmaWxlOiAnJXMnLiBTa2lwcGluZyB0aGUgdGVzdC4u LiIsIGUuZmlsZW5hbWUpCiAgICAgZWxzZToKQEAgLTIwNyw3ICsyMzcsNyBAQCBjbGFzcyBNYWlu KExvZyk6CiAgICAgICAgIHdvcmtlcnMgPSBzZWxmLl9Qb29sKHByb2Nlc3Nlcz1zZWxmLl9vcHRp b25zLnBhcmFsbGVsX2xldmVsKQogICAgICAgICAjIHRvIGVhY2ggZmlsZSBhZGQgb3B0aW9ucy4K ICAgICAgICAgc2VsZi5kZWJ1ZygiVXNpbmcgJXMgdGhlIHdvcmtlcnMgcG9vbCwgbnVtYmVyIG9m IHdvcmtlcnMgJWkiLCByZXByKHdvcmtlcnMpLCBzZWxmLl9vcHRpb25zLnBhcmFsbGVsX2xldmVs KQotICAgICAgICBwYWNrYWdlID0gbWFwKGxhbWJkYSB3OiBbdywgc2VsZi5fb3B0aW9ucy50ZXN0 c19vcHRpb25zLCBub3Qgc2VsZi5fb3B0aW9ucy5kZXZlbG9wZXJdLCBmaWxlcykKKyAgICAgICAg cGFja2FnZSA9IG1hcChsYW1iZGEgdzogW3csIHNlbGYuX29wdGlvbnMudGVzdHNfb3B0aW9ucywg bm90IHNlbGYuX29wdGlvbnMuZGV2ZWxvcGVyLCBzZWxmLl9vcHRpb25zLnRpbWVvdXRdLCBmaWxl cykKICAgICAgICAgc2VsZi5kZWJ1ZygiR2VuZXJhdGVkIHBhY2thZ2VzIGZvciB3b3JrZXJzOiAl cyIsIHJlcHIocGFja2FnZSkpCiAgICAgICAgIHJlc3VsdHMgPSB3b3JrZXJzLm1hcChydW5fdGVz dCwgcGFja2FnZSkgICMgQ29sbGVjdHMgcmVzdWx0cy4KICAgICAgICAgcmV0dXJuIHJlc3VsdHMK