55116 2011-02-23 23:46:25 -0800 generate a times.json file when running layout tests 2011-02-24 23:45:10 -0800 1 1 1 Unclassified WebKit New Bugs 528+ (Nightly build) Other OS X 10.5 RESOLVED DUPLICATE 52267 P2 Normal --- 1 ojan ojan dpranke eric evan kinuko mihaip tony victorw oldest_to_newest 356712 0 ojan 2011-02-23 23:46:25 -0800 generate a times.json file when running layout tests 356715 1 83613 ojan 2011-02-24 00:08:16 -0800 Created attachment 83613 Patch 356730 2 83613 kinuko 2011-02-24 00:40:28 -0800 Comment on attachment 83613 Patch LGTM though I'm not a reviewer. Making it easier to find the bottlenecks in running tests sounds great! View in context: https://bugs.webkit.org/attachment.cgi?id=83613&action=review > Tools/Scripts/webkitpy/layout_tests/layout_package/json_results_generator.py:84 > + TIMES_PREFIX = "ADD_TIMES(" nit: RESULTS_JSON_PREFIX, TIMES_JSON_PREFIX might be easier to read? (to make it match with 'JSON_SUFFIX') 357176 3 83613 tony 2011-02-24 11:04:46 -0800 Comment on attachment 83613 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=83613&action=review > Tools/Scripts/webkitpy/layout_tests/layout_package/json_results_generator.py:184 > + # Convert floats from seconds into milliseconds. > + float_format = lambda o, allow_nan=True: format(o * 1000, '.0f') Why don't we just do this conversion when adding values to self._test_timings (or in get_times())? This avoids having to add an optional param to _generate_json_file and the hackery of setting encoder.floatstr. Also, the comment and the changelog disagree about the format of the map (is it seconds on ms?). 357540 4 ojan 2011-02-24 16:17:38 -0800 (In reply to comment #3) > (From update of attachment 83613 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=83613&action=review > > > Tools/Scripts/webkitpy/layout_tests/layout_package/json_results_generator.py:184 > > + # Convert floats from seconds into milliseconds. > > + float_format = lambda o, allow_nan=True: format(o * 1000, '.0f') > > Why don't we just do this conversion when adding values to self._test_timings (or in get_times())? This avoids having to add an optional param to _generate_json_file and the hackery of setting encoder.floatstr. > > Also, the comment and the changelog disagree about the format of the map (is it seconds on ms?). We use the map values to create the incremental_results.json file as well. So, in one case, we want truncated to seconds and in another we want it converted to milliseconds. So, I don't see how we could set it when adding values to self._test_timings. I guess we could do the conversion in get_times, but we'd still need and argument to tell it whether to return seconds or milliseconds. 357558 5 83613 dpranke 2011-02-24 16:40:10 -0800 Comment on attachment 83613 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=83613&action=review > Tools/Scripts/webkitpy/layout_tests/layout_package/json_layout_results_generator.py:88 > + here you have "get_times" returning "_test_timings", which appears to be the whole dictionary; further down you have "get_test_timing" which refers to a single lookup. Maybe this should be "get_timings" to be more consistent? Or, if you don't like two functions that are different by only one character, "get_all_test_times" and "_get_test_time", and then make the member variables match the function names? On a more general note, a couple of comments ... 1) Are the file formats documented anywhere other than buried in the source code? 2) I feel like this is kind of an overly specialized file. It may work to generate a treemap, but I'm not sure if it is useful for much else. In particular, it would be nice to be able to see the time a test took next to the expectation for the test and the actual result. It's not clear to me if I can get that info from results.json (due to the arcane format of that file), and I don't know that it would make expectations.json much larger to add in the actual result and the time it took. 357564 6 tony 2011-02-24 16:43:59 -0800 (In reply to comment #4) > > We use the map values to create the incremental_results.json file as well. So, in one case, we want truncated to seconds and in another we want it converted to milliseconds. So, I don't see how we could set it when adding values to self._test_timings. I guess we could do the conversion in get_times, but we'd still need and argument to tell it whether to return seconds or milliseconds. Alternately, you could just make a copy of the dict and convert the values before calling _generate_json_file. That seems better than reaching into simplejson and changing the formatter. 357587 7 mihaip 2011-02-24 17:02:58 -0800 (In reply to comment #5) > 2) I feel like this is kind of an overly specialized file. It may work to generate a treemap, but I'm not sure if it is useful for much else. In particular, it would be nice to be able to see the time a test took next to the expectation for the test and the actual result. It's not clear to me if I can get that info from results.json (due to the arcane format of that file), and I don't know that it would make expectations.json much larger to add in the actual result and the time it took. +1 to this (this would effectively fix bug 52267, where I was hoping for a more sane results.json format). 357733 8 ojan 2011-02-24 23:45:10 -0800 (In reply to comment #7) > (In reply to comment #5) > > 2) I feel like this is kind of an overly specialized file. It may work to generate a treemap, but I'm not sure if it is useful for much else. In particular, it would be nice to be able to see the time a test took next to the expectation for the test and the actual result. It's not clear to me if I can get that info from results.json (due to the arcane format of that file), and I don't know that it would make expectations.json much larger to add in the actual result and the time it took. > > +1 to this (this would effectively fix bug 52267, where I was hoping for a more sane results.json format). I'll upload a patch that does this to bug 52267. I'm a bit torn. On the one hand, it's better to have less code to maintain. On the other, the file size going grows from ~750KB to ~2000KB. I'm a bit worried about this affecting test cycle time since these files are uploaded to the test-results-server on each run and need to be downloaded for the treemaps. We can try it and see. *** This bug has been marked as a duplicate of bug 52267 *** 83613 2011-02-24 00:08:16 -0800 2011-02-24 16:40:10 -0800 Patch bug-55116-20110224190813.patch text/plain 9891 ojan U3VidmVyc2lvbiBSZXZpc2lvbjogNzk1MTUKZGlmZiAtLWdpdCBhL1Rvb2xzL0NoYW5nZUxvZyBi L1Rvb2xzL0NoYW5nZUxvZwppbmRleCBhOWViZmJmODE1ZDMyNWU1YmI4MjQ0YjMzMDFhNTc1ZWNj MTMyZDMxLi5iNjJmNjE0NjY1Y2I0NWJiZjI3YzZiYTE5MTEwN2YwZDNkZTJlNjgwIDEwMDY0NAot LS0gYS9Ub29scy9DaGFuZ2VMb2cKKysrIGIvVG9vbHMvQ2hhbmdlTG9nCkBAIC0xLDMgKzEsMjMg QEAKKzIwMTEtMDItMjMgIE9qYW4gVmFmYWkgIDxvamFuQGNocm9taXVtLm9yZz4KKworICAgICAg ICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICBnZW5lcmF0ZSBhIHRpbWVz Lmpzb24gZmlsZSB3aGVuIHJ1bm5pbmcgbGF5b3V0IHRlc3RzCisgICAgICAgIGh0dHBzOi8vYnVn cy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD01NTExNgorCisgICAgICAgIEl0J3MgYSBzaW1w bGUgbWFwIGZyb20gdGVzdC1uYW1lIHRvIHRlc3QgcnVudGltZSBpbiBtaWxsaXNlY29uZHMuCisg ICAgICAgIFRoaXMgd2lsbCBiZSB1c2VkIHRvIGdlbmVyYXRlIGEgdHJlZW1hcCBzbyB3ZSBjYW4g c2VlIHdoZXJlIHRoZQorICAgICAgICBib3R0bGVuZWNrcyBhcmUgaW4gcnVubmluZyB0ZXN0cy4K KworICAgICAgICBXZSB3YW50IGEgc2VwYXJhdGUgSlNPTiBmaWxlIGJlY2F1c2UgdGhpcyBpcyB3 YXkgdG9vIG11Y2ggaW5mb3JtYXRpb24KKyAgICAgICAgdG8gcHV0IGluIHRoZSByZWd1bGFyIHJl c3VsdHMuanNvbiBmaWxlIHdoaWNoIG5lZWRzIHRvIGJlIGtlcHQgc21hbGwKKyAgICAgICAgdG8g YXZvaWQgZnVydGhlciBzbG93aW5nIGRvd24gbG9hZGluZyB0aGUgdGVzdCBkYXNoYm9hcmQuCisK KyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9q c29uX2xheW91dF9yZXN1bHRzX2dlbmVyYXRvci5weToKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtp dHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9qc29uX3Jlc3VsdHNfZ2VuZXJhdG9yLnB5 OgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2xheW91dF9wYWNrYWdl L2pzb25fcmVzdWx0c19nZW5lcmF0b3JfdW5pdHRlc3QucHk6CisgICAgICAgICogU2NyaXB0cy93 ZWJraXRweS9sYXlvdXRfdGVzdHMvbGF5b3V0X3BhY2thZ2UvdGVzdF9ydW5uZXIucHk6CisKIDIw MTEtMDItMjMgIEtlbm5ldGggUnVzc2VsbCAgPGtickBnb29nbGUuY29tPgogCiAgICAgICAgIFVu cmV2aWV3ZWQsIHJvbGxpbmcgb3V0IHI3OTM4Ny4KZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMv d2Via2l0cHkvbGF5b3V0X3Rlc3RzL2xheW91dF9wYWNrYWdlL2pzb25fbGF5b3V0X3Jlc3VsdHNf Z2VuZXJhdG9yLnB5IGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvbGF5b3V0 X3BhY2thZ2UvanNvbl9sYXlvdXRfcmVzdWx0c19nZW5lcmF0b3IucHkKaW5kZXggODIyNmVkMDM5 N2E2NmVhYjI1NWQxMDcxOGYzYjU2NjFmOGM0YWE0Ny4uOTk0OTA5ZWNhMTQwMjk3OTkxNDk1ODc1 MzM4ZmI1YThjOTYyZWYyNiAxMDA2NDQKLS0tIGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlv dXRfdGVzdHMvbGF5b3V0X3BhY2thZ2UvanNvbl9sYXlvdXRfcmVzdWx0c19nZW5lcmF0b3IucHkK KysrIGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvbGF5b3V0X3BhY2thZ2Uv anNvbl9sYXlvdXRfcmVzdWx0c19nZW5lcmF0b3IucHkKQEAgLTgzLDYgKzgzLDkgQEAgY2xhc3Mg SlNPTkxheW91dFJlc3VsdHNHZW5lcmF0b3IoanNvbl9yZXN1bHRzX2dlbmVyYXRvci5KU09OUmVz dWx0c0dlbmVyYXRvckJhc2UKIAogICAgICAgICBzZWxmLmdlbmVyYXRlX2pzb25fb3V0cHV0KCkK IAorICAgIGRlZiBnZXRfdGltZXMoc2VsZik6CisgICAgICAgIHJldHVybiBzZWxmLl90ZXN0X3Rp bWluZ3MKKwogICAgIGRlZiBfZ2V0X3BhdGhfcmVsYXRpdmVfdG9fbGF5b3V0X3Rlc3Rfcm9vdChz ZWxmLCB0ZXN0KToKICAgICAgICAgIiIiUmV0dXJucyB0aGUgcGF0aCBvZiB0aGUgdGVzdCByZWxh dGl2ZSB0byB0aGUgbGF5b3V0IHRlc3Qgcm9vdC4KICAgICAgICAgRm9yIGV4YW1wbGUsIGZvcjoK QEAgLTEwNCwxMyArMTA3LDYgQEAgY2xhc3MgSlNPTkxheW91dFJlc3VsdHNHZW5lcmF0b3IoanNv bl9yZXN1bHRzX2dlbmVyYXRvci5KU09OUmVzdWx0c0dlbmVyYXRvckJhc2UKICAgICAgICAgcmV0 dXJuIHJlbGF0aXZlUGF0aC5yZXBsYWNlKCdcXCcsICcvJykKIAogICAgICMgb3ZlcnJpZGUKLSAg ICBkZWYgX2dldF90ZXN0X3RpbWluZyhzZWxmLCB0ZXN0X25hbWUpOgotICAgICAgICBpZiB0ZXN0 X25hbWUgaW4gc2VsZi5fdGVzdF90aW1pbmdzOgotICAgICAgICAgICAgIyBGbG9vciBmb3Igbm93 IHRvIGdldCB0aW1lIGluIHNlY29uZHMuCi0gICAgICAgICAgICByZXR1cm4gaW50KHNlbGYuX3Rl c3RfdGltaW5nc1t0ZXN0X25hbWVdKQotICAgICAgICByZXR1cm4gMAotCi0gICAgIyBvdmVycmlk ZQogICAgIGRlZiBfZ2V0X2ZhaWxlZF90ZXN0X25hbWVzKHNlbGYpOgogICAgICAgICByZXR1cm4g c2V0KHNlbGYuX2ZhaWx1cmVzLmtleXMoKSkKIApkaWZmIC0tZ2l0IGEvVG9vbHMvU2NyaXB0cy93 ZWJraXRweS9sYXlvdXRfdGVzdHMvbGF5b3V0X3BhY2thZ2UvanNvbl9yZXN1bHRzX2dlbmVyYXRv ci5weSBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2xheW91dF9wYWNrYWdl L2pzb25fcmVzdWx0c19nZW5lcmF0b3IucHkKaW5kZXggMDU2NjJjMjFiZTJiMzRhNjc2NTExZDRl NDllZDAwMWJjNDJhNzRhNy4uZWUxM2UzOGM0ZGJhYjBmNmM2NTcwNDZlNmNmNzBkYWMxOGExOWRk ZCAxMDA2NDQKLS0tIGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvbGF5b3V0 X3BhY2thZ2UvanNvbl9yZXN1bHRzX2dlbmVyYXRvci5weQorKysgYi9Ub29scy9TY3JpcHRzL3dl YmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9qc29uX3Jlc3VsdHNfZ2VuZXJhdG9y LnB5CkBAIC04MCw3ICs4MCw4IEBAIGNsYXNzIEpTT05SZXN1bHRzR2VuZXJhdG9yQmFzZShvYmpl Y3QpOgogICAgIE1BWF9OVU1CRVJfT0ZfQlVJTERfUkVTVUxUU19UT19MT0cgPSA3NTAKICAgICAj IE1pbiB0aW1lIChzZWNvbmRzKSB0aGF0IHdpbGwgYmUgYWRkZWQgdG8gdGhlIEpTT04uCiAgICAg TUlOX1RJTUUgPSAxCi0gICAgSlNPTl9QUkVGSVggPSAiQUREX1JFU1VMVFMoIgorICAgIFJFU1VM VFNfUFJFRklYID0gIkFERF9SRVNVTFRTKCIKKyAgICBUSU1FU19QUkVGSVggPSAiQUREX1RJTUVT KCIKICAgICBKU09OX1NVRkZJWCA9ICIpOyIKIAogICAgICMgTm90ZSB0aGF0IGluIG5vbi1jaHJv bWl1bSB0ZXN0cyB0aG9zZSBjaGFycyBhcmUgdXNlZCB0byBpbmRpY2F0ZQpAQCAtMTEwLDYgKzEx MSw3IEBAIGNsYXNzIEpTT05SZXN1bHRzR2VuZXJhdG9yQmFzZShvYmplY3QpOgogCiAgICAgUkVT VUxUU19GSUxFTkFNRSA9ICJyZXN1bHRzLmpzb24iCiAgICAgSU5DUkVNRU5UQUxfUkVTVUxUU19G SUxFTkFNRSA9ICJpbmNyZW1lbnRhbF9yZXN1bHRzLmpzb24iCisgICAgVElNRVNfRklMRU5BTUUg PSAidGltZXMuanNvbiIKIAogICAgIFVSTF9GT1JfVEVTVF9MSVNUX0pTT04gPSBcCiAgICAgICAg ICJodHRwOi8vJXMvdGVzdGZpbGU/YnVpbGRlcj0lcyZuYW1lPSVzJnRlc3RsaXN0anNvbj0xJnRl c3R0eXBlPSVzIgpAQCAtMTUxLDEyICsxNTMsMTMgQEAgY2xhc3MgSlNPTlJlc3VsdHNHZW5lcmF0 b3JCYXNlKG9iamVjdCk6CiAgICAgICAgIHNlbGYuX2J1aWxkX251bWJlciA9IGJ1aWxkX251bWJl cgogICAgICAgICBzZWxmLl9idWlsZGVyX2Jhc2VfdXJsID0gYnVpbGRlcl9iYXNlX3VybAogICAg ICAgICBzZWxmLl9yZXN1bHRzX2RpcmVjdG9yeSA9IHJlc3VsdHNfZmlsZV9iYXNlX3BhdGgKLSAg ICAgICAgc2VsZi5fcmVzdWx0c19maWxlX3BhdGggPSBzZWxmLl9mcy5qb2luKHJlc3VsdHNfZmls ZV9iYXNlX3BhdGgsCi0gICAgICAgICAgICBzZWxmLlJFU1VMVFNfRklMRU5BTUUpCi0gICAgICAg IHNlbGYuX2luY3JlbWVudGFsX3Jlc3VsdHNfZmlsZV9wYXRoID0gc2VsZi5fZnMuam9pbigKLSAg ICAgICAgICAgIHJlc3VsdHNfZmlsZV9iYXNlX3BhdGgsIHNlbGYuSU5DUkVNRU5UQUxfUkVTVUxU U19GSUxFTkFNRSkKIAogICAgICAgICBzZWxmLl90ZXN0X3Jlc3VsdHNfbWFwID0gdGVzdF9yZXN1 bHRzX21hcAorCisgICAgICAgIHNlbGYuX3Rlc3RfdGltaW5ncyA9IHt9CisgICAgICAgIGZvciB0 ZXN0IGluIHNlbGYuX3Rlc3RfcmVzdWx0c19tYXA6CisgICAgICAgICAgICBzZWxmLl90ZXN0X3Rp bWluZ3NbdGVzdF0gPSBzZWxmLl90ZXN0X3Jlc3VsdHNfbWFwW3Rlc3RdLnRpbWUKKwogICAgICAg ICBzZWxmLl90ZXN0X3Jlc3VsdHMgPSB0ZXN0X3Jlc3VsdHNfbWFwLnZhbHVlcygpCiAKICAgICAg ICAgc2VsZi5fc3ZuX3JlcG9zaXRvcmllcyA9IHN2bl9yZXBvc2l0b3JpZXMKQEAgLTE3Miw4ICsx NzUsMTQgQEAgY2xhc3MgSlNPTlJlc3VsdHNHZW5lcmF0b3JCYXNlKG9iamVjdCk6CiAgICAgZGVm IGdlbmVyYXRlX2pzb25fb3V0cHV0KHNlbGYpOgogICAgICAgICBqc29uID0gc2VsZi5nZXRfanNv bigpCiAgICAgICAgIGlmIGpzb246Ci0gICAgICAgICAgICBzZWxmLl9nZW5lcmF0ZV9qc29uX2Zp bGUoCi0gICAgICAgICAgICAgICAganNvbiwgc2VsZi5faW5jcmVtZW50YWxfcmVzdWx0c19maWxl X3BhdGgpCisgICAgICAgICAgICAjIENvbnZlcnQgZmxvYXRzIHRvIGludGVnZXIgdmFsdWVzIHRv IGtlZXAgZmlsZXNpemUgZG93bi4KKyAgICAgICAgICAgIGZsb2F0X2Zvcm1hdCA9IGxhbWJkYSBv LCBhbGxvd19uYW49VHJ1ZTogZm9ybWF0KG8sICcuMGYnKQorICAgICAgICAgICAgc2VsZi5fZ2Vu ZXJhdGVfanNvbl9maWxlKGpzb24sIHNlbGYuSU5DUkVNRU5UQUxfUkVTVUxUU19GSUxFTkFNRSwg c2VsZi5SRVNVTFRTX1BSRUZJWCwgc2VsZi5KU09OX1NVRkZJWCwgZmxvYXRfZm9ybWF0KQorCisg ICAgICAgIHRpbWVzID0gc2VsZi5nZXRfdGltZXMoKQorICAgICAgICAjIENvbnZlcnQgZmxvYXRz IGZyb20gc2Vjb25kcyBpbnRvIG1pbGxpc2Vjb25kcy4KKyAgICAgICAgZmxvYXRfZm9ybWF0ID0g bGFtYmRhIG8sIGFsbG93X25hbj1UcnVlOiBmb3JtYXQobyAqIDEwMDAsICcuMGYnKQorICAgICAg ICBzZWxmLl9nZW5lcmF0ZV9qc29uX2ZpbGUodGltZXMsIHNlbGYuVElNRVNfRklMRU5BTUUsIHNl bGYuVElNRVNfUFJFRklYLCBzZWxmLkpTT05fU1VGRklYLCBmbG9hdF9mb3JtYXQpCiAKICAgICBk ZWYgZ2V0X2pzb24oc2VsZik6CiAgICAgICAgICIiIkdldHMgdGhlIHJlc3VsdHMgZm9yIHRoZSBy ZXN1bHRzLmpzb24gZmlsZS4iIiIKQEAgLTIxNiw2ICsyMjUsOSBAQCBjbGFzcyBKU09OUmVzdWx0 c0dlbmVyYXRvckJhc2Uob2JqZWN0KToKIAogICAgICAgICByZXR1cm4gcmVzdWx0c19qc29uCiAK KyAgICBkZWYgZ2V0X3RpbWVzKHNlbGYpOgorICAgICAgICByZXR1cm4gc2VsZi5fdGVzdF90aW1p bmdzCisKICAgICBkZWYgc2V0X2FyY2hpdmVkX3Jlc3VsdHMoc2VsZiwgYXJjaGl2ZWRfcmVzdWx0 cyk6CiAgICAgICAgIHNlbGYuX2FyY2hpdmVkX3Jlc3VsdHMgPSBhcmNoaXZlZF9yZXN1bHRzCiAK QEAgLTIzNyw4ICsyNDksNyBAQCBjbGFzcyBKU09OUmVzdWx0c0dlbmVyYXRvckJhc2Uob2JqZWN0 KToKICAgICAgICAgZmlsZXMgPSBbKGZpbGUsIHNlbGYuX2ZzLmpvaW4oc2VsZi5fcmVzdWx0c19k aXJlY3RvcnksIGZpbGUpKQogICAgICAgICAgICAgZm9yIGZpbGUgaW4ganNvbl9maWxlc10KIAot ICAgICAgICB1cGxvYWRlciA9IHRlc3RfcmVzdWx0c191cGxvYWRlci5UZXN0UmVzdWx0c1VwbG9h ZGVyKAotICAgICAgICAgICAgc2VsZi5fdGVzdF9yZXN1bHRzX3NlcnZlcikKKyAgICAgICAgdXBs b2FkZXIgPSB0ZXN0X3Jlc3VsdHNfdXBsb2FkZXIuVGVzdFJlc3VsdHNVcGxvYWRlcihzZWxmLl90 ZXN0X3Jlc3VsdHNfc2VydmVyKQogICAgICAgICB0cnk6CiAgICAgICAgICAgICAjIFNldCB1cGxv YWRpbmcgdGltZW91dCBpbiBjYXNlIGFwcGVuZ2luZSBzZXJ2ZXIgaXMgaGF2aW5nIHByb2JsZW0u CiAgICAgICAgICAgICAjIDEyMCBzZWNvbmRzIGFyZSBtb3JlIHRoYW4gZW5vdWdoIHRvIHVwbG9h ZCB0ZXN0IHJlc3VsdHMuCkBAIC0yNDksMTggKzI2MCwyNiBAQCBjbGFzcyBKU09OUmVzdWx0c0dl bmVyYXRvckJhc2Uob2JqZWN0KToKIAogICAgICAgICBfbG9nLmluZm8oIkpTT04gZmlsZXMgdXBs b2FkZWQuIikKIAotICAgIGRlZiBfZ2VuZXJhdGVfanNvbl9maWxlKHNlbGYsIGpzb24sIGZpbGVf cGF0aCk6CisgICAgZGVmIF9nZW5lcmF0ZV9qc29uX2ZpbGUoc2VsZiwganNvbiwgZmlsZW5hbWUs IHByZWZpeCwgc3VmZml4LCBmbG9hdF9mb3JtYXQ9Tm9uZSk6CisgICAgICAgICMgRklYTUU6IE9u Y2Ugd2UgbW92ZSB0byBweXRob24ncyBidWlsdGluIGpzb24gbW9kdWxlLCBzZXQgc2ltcGxlanNv bi5lbmNvZGVyLkZMT0FUX1JFUFIgaW5zdGVhZC4KKyAgICAgICAgaWYgZmxvYXRfZm9ybWF0Ogor ICAgICAgICAgICAgb2xkZmxvYXRzdHIgPSBzaW1wbGVqc29uLmVuY29kZXIuZmxvYXRzdHIKKyAg ICAgICAgICAgIHNpbXBsZWpzb24uZW5jb2Rlci5mbG9hdHN0ciA9IGZsb2F0X2Zvcm1hdAorCiAg ICAgICAgICMgU3BlY2lmeSBzZXBhcmF0b3JzIGluIG9yZGVyIHRvIGdldCBjb21wYWN0IGVuY29k aW5nLgogICAgICAgICBqc29uX2RhdGEgPSBzaW1wbGVqc29uLmR1bXBzKGpzb24sIHNlcGFyYXRv cnM9KCcsJywgJzonKSkKLSAgICAgICAganNvbl9zdHJpbmcgPSBzZWxmLkpTT05fUFJFRklYICsg anNvbl9kYXRhICsgc2VsZi5KU09OX1NVRkZJWAorICAgICAgICBqc29uX3N0cmluZyA9IHByZWZp eCArIGpzb25fZGF0YSArIHN1ZmZpeAorICAgICAgICBmaWxlX3BhdGggPSBzZWxmLl9mcy5qb2lu KHNlbGYuX3Jlc3VsdHNfZGlyZWN0b3J5LCBmaWxlbmFtZSkKICAgICAgICAgc2VsZi5fZnMud3Jp dGVfdGV4dF9maWxlKGZpbGVfcGF0aCwganNvbl9zdHJpbmcpCiAKKyAgICAgICAgaWYgZmxvYXRf Zm9ybWF0OgorICAgICAgICAgICAgc2ltcGxlanNvbi5lbmNvZGVyLmZsb2F0c3RyID0gb2xkZmxv YXRzdHIKKwogICAgIGRlZiBfZ2V0X3Rlc3RfdGltaW5nKHNlbGYsIHRlc3RfbmFtZSk6CiAgICAg ICAgICIiIlJldHVybnMgdGVzdCB0aW1pbmcgZGF0YSAoZWxhcHNlZCB0aW1lKSBpbiBzZWNvbmQK ICAgICAgICAgZm9yIHRoZSBnaXZlbiB0ZXN0X25hbWUuIiIiCi0gICAgICAgIGlmIHRlc3RfbmFt ZSBpbiBzZWxmLl90ZXN0X3Jlc3VsdHNfbWFwOgotICAgICAgICAgICAgIyBGbG9vciBmb3Igbm93 IHRvIGdldCB0aW1lIGluIHNlY29uZHMuCi0gICAgICAgICAgICByZXR1cm4gaW50KHNlbGYuX3Rl c3RfcmVzdWx0c19tYXBbdGVzdF9uYW1lXS50aW1lKQorICAgICAgICBpZiB0ZXN0X25hbWUgaW4g c2VsZi5fdGVzdF90aW1pbmdzOgorICAgICAgICAgICAgcmV0dXJuIHNlbGYuX3Rlc3RfdGltaW5n c1t0ZXN0X25hbWVdCiAgICAgICAgIHJldHVybiAwCiAKICAgICBkZWYgX2dldF9mYWlsZWRfdGVz dF9uYW1lcyhzZWxmKToKQEAgLTM1Nyw4ICszNzYsNyBAQCBjbGFzcyBKU09OUmVzdWx0c0dlbmVy YXRvckJhc2Uob2JqZWN0KToKIAogICAgICAgICBpZiBvbGRfcmVzdWx0czoKICAgICAgICAgICAg ICMgU3RyaXAgdGhlIHByZWZpeCBhbmQgc3VmZml4IHNvIHdlIGNhbiBnZXQgdGhlIGFjdHVhbCBK U09OIG9iamVjdC4KLSAgICAgICAgICAgIG9sZF9yZXN1bHRzID0gb2xkX3Jlc3VsdHNbbGVuKHNl bGYuSlNPTl9QUkVGSVgpOgotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBs ZW4ob2xkX3Jlc3VsdHMpIC0gbGVuKHNlbGYuSlNPTl9TVUZGSVgpXQorICAgICAgICAgICAgb2xk X3Jlc3VsdHMgPSBvbGRfcmVzdWx0c1tsZW4oc2VsZi5SRVNVTFRTX1BSRUZJWCk6bGVuKG9sZF9y ZXN1bHRzKSAtIGxlbihzZWxmLkpTT05fU1VGRklYKV0KIAogICAgICAgICAgICAgdHJ5OgogICAg ICAgICAgICAgICAgIHJlc3VsdHNfanNvbiA9IHNpbXBsZWpzb24ubG9hZHMob2xkX3Jlc3VsdHMp CmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRf cGFja2FnZS9qc29uX3Jlc3VsdHNfZ2VuZXJhdG9yX3VuaXR0ZXN0LnB5IGIvVG9vbHMvU2NyaXB0 cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvbGF5b3V0X3BhY2thZ2UvanNvbl9yZXN1bHRzX2dlbmVy YXRvcl91bml0dGVzdC5weQppbmRleCA5NWRhOGZiNDYzM2Y4MGFhMWNkYzg4YzZmNmI5ODE2NzEw ZGNhZjIzLi44MTAzNmUxZDZmMjk4ZGQxNGRlMmJiZjljOGJjNzg3ZjliMmM1Y2RhIDEwMDY0NAot LS0gYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9q c29uX3Jlc3VsdHNfZ2VuZXJhdG9yX3VuaXR0ZXN0LnB5CisrKyBiL1Rvb2xzL1NjcmlwdHMvd2Vi a2l0cHkvbGF5b3V0X3Rlc3RzL2xheW91dF9wYWNrYWdlL2pzb25fcmVzdWx0c19nZW5lcmF0b3Jf dW5pdHRlc3QucHkKQEAgLTEwNiw2ICsxMDYsMTUgQEAgY2xhc3MgSlNPTkdlbmVyYXRvclRlc3Qo dW5pdHRlc3QuVGVzdENhc2UpOgogICAgICAgICAgICAgaW5jcmVtZW50YWxfanNvbiwKICAgICAg ICAgICAgIDEpCiAKKyAgICAgICAgIyBKdXN0IHJ1biB0aGlzIGNvZGUuIFdlIGRvbid0IHZlcmlm eSB0aGUgb3V0cHV0LCBidXQgYXQgbGVhc3QgaXQgcnVucyB3aXRob3V0IGVycm9ycy4KKyAgICAg ICAgZ2VuZXJhdG9yLmdlbmVyYXRlX2pzb25fb3V0cHV0KCkKKworICAgICAgICB0aW1pbmdzID0g Z2VuZXJhdG9yLmdldF90aW1lcygpCisgICAgICAgIGkgPSAwCisgICAgICAgIGZvciB0ZXN0IGlu IHRlc3RzX3NldDoKKyAgICAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwodGltaW5nc1t0ZXN0XSwg ZmxvYXQoc2VsZi5fbnVtX3J1bnMgKiAxMDAgKyBpKSkKKyAgICAgICAgICAgIGkgKz0gMQorCiAg ICAgZGVmIF92ZXJpZnlfanNvbl9yZXN1bHRzKHNlbGYsIHRlc3RzX3NldCwgdGVzdF90aW1pbmdz LCBmYWlsZWRfY291bnRfbWFwLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBQQVNTX2Nv dW50LCBESVNBQkxFRF9jb3VudCwgRkxBS1lfY291bnQsCiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIGZpeGFibGVfY291bnQsCmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5 L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS90ZXN0X3J1bm5lci5weSBiL1Rvb2xzL1Njcmlw dHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2xheW91dF9wYWNrYWdlL3Rlc3RfcnVubmVyLnB5Cmlu ZGV4IGUzYmQ0YWQyNmFmMDhhMGE2ODExZTRmMDZhNzhhNWI2NTI4MjIyNGUuLjNkZTE0ZTZmMzBl NDg4YWUxNmYyN2U3ZmI5YTY0YjAyZjU4MGIwMmIgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1NjcmlwdHMv d2Via2l0cHkvbGF5b3V0X3Rlc3RzL2xheW91dF9wYWNrYWdlL3Rlc3RfcnVubmVyLnB5CisrKyBi L1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2xheW91dF9wYWNrYWdlL3Rlc3Rf cnVubmVyLnB5CkBAIC04MzIsNyArODMyLDcgQEAgY2xhc3MgVGVzdFJ1bm5lcjoKIAogICAgICAg ICBfbG9nLmRlYnVnKCJGaW5pc2hlZCB3cml0aW5nIEpTT04gZmlsZXMuIikKIAotICAgICAgICBq c29uX2ZpbGVzID0gWyJleHBlY3RhdGlvbnMuanNvbiIsICJpbmNyZW1lbnRhbF9yZXN1bHRzLmpz b24iXQorICAgICAgICBqc29uX2ZpbGVzID0gWyJleHBlY3RhdGlvbnMuanNvbiIsICJpbmNyZW1l bnRhbF9yZXN1bHRzLmpzb24iLCAidGltZXMuanNvbiJdCiAKICAgICAgICAgZ2VuZXJhdG9yLnVw bG9hZF9qc29uX2ZpbGVzKGpzb25fZmlsZXMpCiAK