104184 2012-12-05 16:31:20 -0800 nrwt: record the order and run times of all of the tests in a single file 2013-05-21 16:07:36 -0700 1 1 1 Unclassified WebKit New Bugs 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 103826 1 dpranke dpranke abarth eric kadam kbalazs ojan ossy rniwa tony webkit.review.bot zarvai oldest_to_newest 784240 0 dpranke 2012-12-05 16:31:20 -0800 nrwt: record the order and run times of all of the tests in a single file 784245 1 177858 dpranke 2012-12-05 16:33:36 -0800 Created attachment 177858 Patch 784267 2 177858 ojan 2012-12-05 16:45:07 -0800 Comment on attachment 177858 Patch Are python dicts ordered and will json.dumps preserve the ordering? 784275 3 dpranke 2012-12-05 16:49:05 -0800 No, and no. That's why the results tuples include the worker and the order in each worker. I actually wrote a separate script (not included in this patch) that verified that you could reconstruct the tests_run*.txt files from the json and they matched. I believe you can recreate the total ordering of tests across the whole run to within a millisecond with this data; in the unit test data all of the tests complete in less than a millisecond, so the values are all zeros and I'm not doing more fancy testing to verify that we can-do cross-shard ordering here (that, and the unit test only runs with one worker). 787938 4 177858 webkit.review.bot 2012-12-10 18:47:26 -0800 Comment on attachment 177858 Patch Rejecting attachment 177858 from commit-queue. New failing tests: inspector-protocol/debugger-terminate-dedicated-worker-while-paused.html Full output: http://queues.webkit.org/results/15231765 788749 5 dpranke 2012-12-11 13:46:07 -0800 Committed r137356: <http://trac.webkit.org/changeset/137356> 891656 6 ossy 2013-05-21 02:12:13 -0700 (In reply to comment #3) > No, and no. That's why the results tuples include the worker and the order in each worker. I actually wrote a separate script (not included in this patch) that verified that you could reconstruct the tests_run*.txt files from the json and they matched. Can we find this script somewhere? tests_run*.txt files were really useful to debug flakinesses related to the order. 891702 7 kbalazs 2013-05-21 04:23:24 -0700 (In reply to comment #6) > (In reply to comment #3) > > No, and no. That's why the results tuples include the worker and the order in each worker. I actually wrote a separate script (not included in this patch) that verified that you could reconstruct the tests_run*.txt files from the json and they matched. > > Can we find this script somewhere? tests_run*.txt files were > really useful to debug flakinesses related to the order. +1 IMHO it was better to generate them by default. 891854 8 dpranke 2013-05-21 09:57:30 -0700 I have no idea what happened to that script but I'd be happy to write another one. I will post it in a separate bug. 891973 9 dpranke 2013-05-21 13:18:42 -0700 script posted in bug 116571. Feel free to play with it / review it / land it at your convenience. 892032 10 kbalazs 2013-05-21 15:33:47 -0700 (In reply to comment #9) > script posted in bug 116571. Feel free to play with it / review it / land it at your convenience. Thanks!. I vote for merging the code into rwt and generate the files by default. Ossy? Others? 892043 11 dpranke 2013-05-21 16:07:36 -0700 (In reply to comment #10) > (In reply to comment #9) > > script posted in bug 116571. Feel free to play with it / review it / land it at your convenience. > > Thanks!. I vote for merging the code into rwt and generate the files by default. Ossy? Others? I don't particularly want to merge this code back into rwt (since this patch took it out in favor of the new format). It takes up space in the results files, complicates the core run-webkit-tests code, and is unused 99% of the time. That said, I won't stop you if you want to do the work :). 177858 2012-12-05 16:33:36 -0800 2012-12-10 18:47:26 -0800 Patch bug-104184-20121205163100.patch text/plain 12678 dpranke U3VidmVyc2lvbiBSZXZpc2lvbjogMTM2NzcxCmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggYzk1ZTMwOGU3N2ViNTk4NTM5MWVkYWE5MmVjYThlZmY0 ZjBiMmFhNS4uYzFjYjBiNzllNzA4YTg5Nzc0ZTQ0ZWFhMmI3ZGQ4NmRiNTUyMGM0ZiAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSw1ICsxLDQy IEBACiAyMDEyLTEyLTA1ICBEaXJrIFByYW5rZSAgPGRwcmFua2VAY2hyb21pdW0ub3JnPgogCisg ICAgICAgIG5yd3Q6IHJlY29yZCB0aGUgb3JkZXIgYW5kIHJ1biB0aW1lcyBvZiBhbGwgb2YgdGhl IHRlc3RzIGluIGEgc2luZ2xlIGZpbGUKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcv c2hvd19idWcuY2dpP2lkPTEwNDE4NAorCisgICAgICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09Q UyEpLgorCisgICAgICAgIFJlcGxhY2UgdGhlIE4gdGVzdHNfcnVuKi50eHQgZmlsZXMgdGhhdCB3 ZSBvdXRwdXQgaW50bworICAgICAgICBsYXlvdXQtdGVzdC1yZXN1bHRzIHdpdGggYSBzaW5nbGUg cGFja2VkIEpTT04gdHJpZSB0aGF0IGNvbnRhaW5zCisgICAgICAgIHRoZSB0ZXN0IG9yZGVyaW5n IGFuZCBydW4gc3RhdGlzdGljcy4gVGhpcyBjYW4gZXZlbnR1YWxseSBiZSB1cGxvYWRlZAorICAg ICAgICB0byB0aGUgVGVzdFJlc3VsdHNTZXJ2ZXIgdG8gcmVwbGFjZSB0aGUgdGltZXMuanNvbiBm aWxlIHdlIGN1cnJlbnRseQorICAgICAgICB1cGxvYWQgYXMgd2VsbCwgYW5kIHdpbGwgbGV0IHVz IHJlcGxheSB0aGUgcmVzdWx0cyBvZiBhIHByZXZpb3VzCisgICAgICAgIHJ1biBmYWlybHkgZGV0 ZXJtaW5pc3RpY2FsbHkuCisKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0 cy9jb250cm9sbGVycy9sYXlvdXRfdGVzdF9ydW5uZXIucHk6CisgICAgICAgIChXb3JrZXIuX19p bml0X18pOgorICAgICAgICAoV29ya2VyLnN0YXJ0KToKKyAgICAgICAgKFdvcmtlci5fcnVuX3Rl c3QpOgorICAgICAgICAoV29ya2VyLnN0b3ApOgorICAgICAgICAoV29ya2VyLl9jbGVhbl91cF9h ZnRlcl90ZXN0KToKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9jb250 cm9sbGVycy9tYW5hZ2VyLnB5OgorICAgICAgICAoTWFuYWdlci5fdXBsb2FkX2pzb25fZmlsZXMp OgorICAgICAgICAoTWFuYWdlci5fY29weV9yZXN1bHRzX2h0bWxfZmlsZSk6CisgICAgICAgIChN YW5hZ2VyKToKKyAgICAgICAgKE1hbmFnZXIuX3N0YXRzX3RyaWUpOgorICAgICAgICAoTWFuYWdl ci5fc3RhdHNfdHJpZS5fd29ya2VyX251bWJlcik6CisgICAgICAgICogU2NyaXB0cy93ZWJraXRw eS9sYXlvdXRfdGVzdHMvbW9kZWxzL3Rlc3RfcmVzdWx0cy5weToKKyAgICAgICAgKFRlc3RSZXN1 bHQuX19pbml0X18pOgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3J1 bl93ZWJraXRfdGVzdHNfaW50ZWdyYXRpb250ZXN0LnB5OgorICAgICAgICAoTWFpblRlc3QudGVz dF9taXNzaW5nX2FuZF91bmV4cGVjdGVkX3Jlc3VsdHMpOgorICAgICAgICAoTWFpblRlc3QudGVz dF9yZXRyeWluZ19hbmRfZmxha3lfdGVzdHMpOgorICAgICAgICAoRW5kVG9FbmRUZXN0LnRlc3Rf ZW5kX3RvX2VuZCk6CisgICAgICAgIChFbmRUb0VuZFRlc3QudGVzdF9yZWZ0ZXN0X3dpdGhfdHdv X25vdHJlZnMpOgorICAgICAgICAoUmViYXNlbGluZVRlc3QudGVzdF9yZXNldF9yZXN1bHRzKToK KyAgICAgICAgKFJlYmFzZWxpbmVUZXN0LnRlc3RfbWlzc2luZ19yZXN1bHRzKToKKyAgICAgICAg KFJlYmFzZWxpbmVUZXN0LnRlc3RfbmV3X2Jhc2VsaW5lKToKKworMjAxMi0xMi0wNSAgRGlyayBQ cmFua2UgIDxkcHJhbmtlQGNocm9taXVtLm9yZz4KKwogICAgICAgICBucnd0OiBydW5fd2Via2l0 X3Rlc3RzX2ludGVncmF0aW9udGVzdC5weSBkZXBlbmRzIG9uIGJlaW5nIGFibGUgdG8gc2hhcmUg YSBwb3J0IHcvIGJldHdlZW4gbWFzdGVyIGFuZCB3b3JrZXIKICAgICAgICAgaHR0cHM6Ly9idWdz LndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTc4MTY4CiAKZGlmZiAtLWdpdCBhL1Rvb2xzL1Nj cmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2NvbnRyb2xsZXJzL2xheW91dF90ZXN0X3J1bm5l ci5weSBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2NvbnRyb2xsZXJzL2xh eW91dF90ZXN0X3J1bm5lci5weQppbmRleCAyYTljNmU2MTk0NjMwNWE1OGYwOGE3ODdjNzk1NmNh ZmMwM2U1M2E0Li5kMGVlMDkwNzU2MGE5ZWE3MzExNTk2ZDVmYmExYjQ4OTgwNDQwM2FlIDEwMDY0 NAotLS0gYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9jb250cm9sbGVycy9s YXlvdXRfdGVzdF9ydW5uZXIucHkKKysrIGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRf dGVzdHMvY29udHJvbGxlcnMvbGF5b3V0X3Rlc3RfcnVubmVyLnB5CkBAIC0yNjAsOCArMjYwLDcg QEAgY2xhc3MgV29ya2VyKG9iamVjdCk6CiAgICAgICAgIHNlbGYuX2JhdGNoX2NvdW50ID0gTm9u ZQogICAgICAgICBzZWxmLl9maWxlc3lzdGVtID0gTm9uZQogICAgICAgICBzZWxmLl9kcml2ZXIg PSBOb25lCi0gICAgICAgIHNlbGYuX3Rlc3RzX3J1bl9maWxlID0gTm9uZQotICAgICAgICBzZWxm Ll90ZXN0c19ydW5fZmlsZW5hbWUgPSBOb25lCisgICAgICAgIHNlbGYuX251bV90ZXN0cyA9IDAK IAogICAgIGRlZiBfX2RlbF9fKHNlbGYpOgogICAgICAgICBzZWxmLnN0b3AoKQpAQCAtMjc2LDgg KzI3NSw2IEBAIGNsYXNzIFdvcmtlcihvYmplY3QpOgogCiAgICAgICAgIHNlbGYuX2JhdGNoX2Nv dW50ID0gMAogICAgICAgICBzZWxmLl9iYXRjaF9zaXplID0gc2VsZi5fb3B0aW9ucy5iYXRjaF9z aXplIG9yIDAKLSAgICAgICAgdGVzdHNfcnVuX2ZpbGVuYW1lID0gc2VsZi5fZmlsZXN5c3RlbS5q b2luKHNlbGYuX3Jlc3VsdHNfZGlyZWN0b3J5LCAidGVzdHNfcnVuJWQudHh0IiAlIHNlbGYuX3dv cmtlcl9udW1iZXIpCi0gICAgICAgIHNlbGYuX3Rlc3RzX3J1bl9maWxlID0gc2VsZi5fZmlsZXN5 c3RlbS5vcGVuX3RleHRfZmlsZV9mb3Jfd3JpdGluZyh0ZXN0c19ydW5fZmlsZW5hbWUpCiAKICAg ICBkZWYgaGFuZGxlKHNlbGYsIG5hbWUsIHNvdXJjZSwgdGVzdF9saXN0X25hbWUsIHRlc3RfaW5w dXRzKToKICAgICAgICAgYXNzZXJ0IG5hbWUgPT0gJ3Rlc3RfbGlzdCcKQEAgLTMxMSw2ICszMDgs OCBAQCBjbGFzcyBXb3JrZXIob2JqZWN0KToKICAgICAgICAgcmVzdWx0LnNoYXJkX25hbWUgPSBz aGFyZF9uYW1lCiAgICAgICAgIHJlc3VsdC53b3JrZXJfbmFtZSA9IHNlbGYuX25hbWUKICAgICAg ICAgcmVzdWx0LnRvdGFsX3J1bl90aW1lID0gdGltZS50aW1lKCkgLSBzdGFydAorICAgICAgICBy ZXN1bHQudGVzdF9udW1iZXIgPSBzZWxmLl9udW1fdGVzdHMKKyAgICAgICAgc2VsZi5fbnVtX3Rl c3RzICs9IDEKIAogICAgICAgICBzZWxmLl9jYWxsZXIucG9zdCgnZmluaXNoZWRfdGVzdCcsIHJl c3VsdCkKIApAQCAtMzE5LDkgKzMxOCw2IEBAIGNsYXNzIFdvcmtlcihvYmplY3QpOgogICAgIGRl ZiBzdG9wKHNlbGYpOgogICAgICAgICBfbG9nLmRlYnVnKCIlcyBjbGVhbmluZyB1cCIgJSBzZWxm Ll9uYW1lKQogICAgICAgICBzZWxmLl9raWxsX2RyaXZlcigpCi0gICAgICAgIGlmIHNlbGYuX3Rl c3RzX3J1bl9maWxlOgotICAgICAgICAgICAgc2VsZi5fdGVzdHNfcnVuX2ZpbGUuY2xvc2UoKQot ICAgICAgICAgICAgc2VsZi5fdGVzdHNfcnVuX2ZpbGUgPSBOb25lCiAKICAgICBkZWYgX3RpbWVv dXQoc2VsZiwgdGVzdF9pbnB1dCk6CiAgICAgICAgICIiIkNvbXB1dGUgdGhlIGFwcHJvcHJpYXRl IHRpbWVvdXQgdmFsdWUgZm9yIGEgdGVzdC4iIiIKQEAgLTM1NSw3ICszNTEsNiBAQCBjbGFzcyBX b3JrZXIob2JqZWN0KToKIAogICAgIGRlZiBfY2xlYW5fdXBfYWZ0ZXJfdGVzdChzZWxmLCB0ZXN0 X2lucHV0LCByZXN1bHQpOgogICAgICAgICB0ZXN0X25hbWUgPSB0ZXN0X2lucHV0LnRlc3RfbmFt ZQotICAgICAgICBzZWxmLl90ZXN0c19ydW5fZmlsZS53cml0ZSh0ZXN0X25hbWUgKyAiXG4iKQog CiAgICAgICAgIGlmIHJlc3VsdC5mYWlsdXJlczoKICAgICAgICAgICAgICMgQ2hlY2sgYW5kIGtp bGwgRHVtcFJlbmRlclRyZWUgaWYgd2UgbmVlZCB0by4KZGlmZiAtLWdpdCBhL1Rvb2xzL1Njcmlw dHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2NvbnRyb2xsZXJzL21hbmFnZXIucHkgYi9Ub29scy9T Y3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9jb250cm9sbGVycy9tYW5hZ2VyLnB5CmluZGV4 IDU5Mjg5OWEwYjI4MWMyYTFlN2U0NjU1NmQ2MzEwNTBjZDk4MDQ0M2QuLjkzMGY1N2QzYTBhZDY3 NGY3ODIzOGRhZTM1ZDc3Njg2ZTlhNTY3NjkgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1NjcmlwdHMvd2Vi a2l0cHkvbGF5b3V0X3Rlc3RzL2NvbnRyb2xsZXJzL21hbmFnZXIucHkKKysrIGIvVG9vbHMvU2Ny aXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvY29udHJvbGxlcnMvbWFuYWdlci5weQpAQCAtMzQs NiArMzQsNyBAQCBvYmplY3RzIHRvIHRoZSBNYW5hZ2VyLiBUaGUgTWFuYWdlciB0aGVuIGFnZ3Jl Z2F0ZXMgdGhlIFRlc3RGYWlsdXJlcyB0bwogY3JlYXRlIGEgZmluYWwgcmVwb3J0LgogIiIiCiAK K2ltcG9ydCBqc29uCiBpbXBvcnQgbG9nZ2luZwogaW1wb3J0IHJhbmRvbQogaW1wb3J0IHN5cwpA QCAtNDc2LDEwICs0NzcsMTUgQEAgY2xhc3MgTWFuYWdlcihvYmplY3QpOgogICAgICAgICAiIiIK ICAgICAgICAgX2xvZy5kZWJ1ZygiV3JpdGluZyBKU09OIGZpbGVzIGluICVzLiIgJSBzZWxmLl9y ZXN1bHRzX2RpcmVjdG9yeSkKIAorICAgICAgICAjIEZJWE1FOiBVcGxvYWQgc3RhdHMuanNvbiB0 byB0aGUgc2VydmVyIGFuZCBkZWxldGUgdGltZXNfbXMuCiAgICAgICAgIHRpbWVzX3RyaWUgPSBq c29uX3Jlc3VsdHNfZ2VuZXJhdG9yLnRlc3RfdGltaW5nc190cmllKHNlbGYuX3BvcnQsIHJlc3Vs dF9zdW1tYXJ5LnJlc3VsdHMudmFsdWVzKCkpCiAgICAgICAgIHRpbWVzX2pzb25fcGF0aCA9IHNl bGYuX2ZpbGVzeXN0ZW0uam9pbihzZWxmLl9yZXN1bHRzX2RpcmVjdG9yeSwgInRpbWVzX21zLmpz b24iKQogICAgICAgICBqc29uX3Jlc3VsdHNfZ2VuZXJhdG9yLndyaXRlX2pzb24oc2VsZi5fZmls ZXN5c3RlbSwgdGltZXNfdHJpZSwgdGltZXNfanNvbl9wYXRoKQogCisgICAgICAgIHN0YXRzX3Ry aWUgPSBzZWxmLl9zdGF0c190cmllKHJlc3VsdF9zdW1tYXJ5KQorICAgICAgICBzdGF0c19wYXRo ID0gc2VsZi5fZmlsZXN5c3RlbS5qb2luKHNlbGYuX3Jlc3VsdHNfZGlyZWN0b3J5LCAic3RhdHMu anNvbiIpCisgICAgICAgIHNlbGYuX2ZpbGVzeXN0ZW0ud3JpdGVfdGV4dF9maWxlKHN0YXRzX3Bh dGgsIGpzb24uZHVtcHMoc3RhdHNfdHJpZSkpCisKICAgICAgICAgZnVsbF9yZXN1bHRzX3BhdGgg PSBzZWxmLl9maWxlc3lzdGVtLmpvaW4oc2VsZi5fcmVzdWx0c19kaXJlY3RvcnksICJmdWxsX3Jl c3VsdHMuanNvbiIpCiAgICAgICAgICMgV2Ugd3JpdGUgZnVsbF9yZXN1bHRzLmpzb24gb3V0IGFz IGpzb25wIGJlY2F1c2Ugd2UgbmVlZCB0byBsb2FkIGl0IGZyb20gYSBmaWxlIHVybCBhbmQgQ2hy b21pdW0gZG9lc24ndCBhbGxvdyB0aGF0LgogICAgICAgICBqc29uX3Jlc3VsdHNfZ2VuZXJhdG9y LndyaXRlX2pzb24oc2VsZi5fZmlsZXN5c3RlbSwgc3VtbWFyaXplZF9yZXN1bHRzLCBmdWxsX3Jl c3VsdHNfcGF0aCwgY2FsbGJhY2s9IkFERF9SRVNVTFRTIikKQEAgLTQ5NSw2ICs1MDEsNyBAQCBj bGFzcyBNYW5hZ2VyKG9iamVjdCk6CiAKICAgICAgICAgX2xvZy5kZWJ1ZygiRmluaXNoZWQgd3Jp dGluZyBKU09OIGZpbGVzLiIpCiAKKwogICAgICAgICBqc29uX2ZpbGVzID0gWyJpbmNyZW1lbnRh bF9yZXN1bHRzLmpzb24iLCAiZnVsbF9yZXN1bHRzLmpzb24iLCAidGltZXNfbXMuanNvbiJdCiAK ICAgICAgICAgZ2VuZXJhdG9yLnVwbG9hZF9qc29uX2ZpbGVzKGpzb25fZmlsZXMpCkBAIC01MTMs MyArNTIwLDE2IEBAIGNsYXNzIE1hbmFnZXIob2JqZWN0KToKICAgICAgICAgIyBzbyBtYWtlIHN1 cmUgaXQgZXhpc3RzIGJlZm9yZSB3ZSB0cnkgdG8gY29weSBpdC4KICAgICAgICAgaWYgc2VsZi5f ZmlsZXN5c3RlbS5leGlzdHMocmVzdWx0c19maWxlKToKICAgICAgICAgICAgIHNlbGYuX2ZpbGVz eXN0ZW0uY29weWZpbGUocmVzdWx0c19maWxlLCBkZXN0aW5hdGlvbl9wYXRoKQorCisgICAgZGVm IF9zdGF0c190cmllKHNlbGYsIHJlc3VsdF9zdW1tYXJ5KToKKyAgICAgICAgZGVmIF93b3JrZXJf bnVtYmVyKHdvcmtlcl9uYW1lKToKKyAgICAgICAgICAgIHJldHVybiBpbnQod29ya2VyX25hbWUu c3BsaXQoJy8nKVsxXSkgaWYgd29ya2VyX25hbWUgZWxzZSAtMQorCisgICAgICAgIHN0YXRzID0g e30KKyAgICAgICAgZm9yIHJlc3VsdCBpbiByZXN1bHRfc3VtbWFyeS5yZXN1bHRzLnZhbHVlcygp OgorICAgICAgICAgICAgaWYgcmVzdWx0LnR5cGUgIT0gdGVzdF9leHBlY3RhdGlvbnMuU0tJUDoK KyAgICAgICAgICAgICAgICBzdGF0c1tyZXN1bHQudGVzdF9uYW1lXSA9IHsncmVzdWx0cyc6IChf d29ya2VyX251bWJlcihyZXN1bHQud29ya2VyX25hbWUpLCByZXN1bHQudGVzdF9udW1iZXIsIHJl c3VsdC5waWQsIGludChyZXN1bHQudGVzdF9ydW5fdGltZSAqIDEwMDApLCBpbnQocmVzdWx0LnRv dGFsX3J1bl90aW1lICogMTAwMCkpfQorICAgICAgICBzdGF0c190cmllID0ge30KKyAgICAgICAg Zm9yIG5hbWUsIHZhbHVlIGluIHN0YXRzLml0ZXJpdGVtcygpOgorICAgICAgICAgICAganNvbl9y ZXN1bHRzX2dlbmVyYXRvci5hZGRfcGF0aF90b190cmllKG5hbWUsIHZhbHVlLCBzdGF0c190cmll KQorICAgICAgICByZXR1cm4gc3RhdHNfdHJpZQpkaWZmIC0tZ2l0IGEvVG9vbHMvU2NyaXB0cy93 ZWJraXRweS9sYXlvdXRfdGVzdHMvbW9kZWxzL3Rlc3RfcmVzdWx0cy5weSBiL1Rvb2xzL1Njcmlw dHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL21vZGVscy90ZXN0X3Jlc3VsdHMucHkKaW5kZXggMTMw MTVkNGU0OWY1OWU3Mjk5NWIwMDZmMzJiYjcwODdiM2VkZGZmYy4uZDZmZDEwYjE4MTg4ZGE4M2Y2 MjNlNjY4MzliOTU1NzdhZGY1NTk3MiAxMDA2NDQKLS0tIGEvVG9vbHMvU2NyaXB0cy93ZWJraXRw eS9sYXlvdXRfdGVzdHMvbW9kZWxzL3Rlc3RfcmVzdWx0cy5weQorKysgYi9Ub29scy9TY3JpcHRz L3dlYmtpdHB5L2xheW91dF90ZXN0cy9tb2RlbHMvdGVzdF9yZXN1bHRzLnB5CkBAIC01NCw2ICs1 NCw3IEBAIGNsYXNzIFRlc3RSZXN1bHQob2JqZWN0KToKICAgICAgICAgc2VsZi53b3JrZXJfbmFt ZSA9ICcnCiAgICAgICAgIHNlbGYuc2hhcmRfbmFtZSA9ICcnCiAgICAgICAgIHNlbGYudG90YWxf cnVuX3RpbWUgPSAwICAjIFRoZSB0aW1lIHRha2VuIHRvIHJ1biB0aGUgdGVzdCBwbHVzIGFueSBy ZWZlcmVuY2VzLCBjb21wdXRlIGRpZmZzLCBldGMuCisgICAgICAgIHNlbGYudGVzdF9udW1iZXIg PSBOb25lCiAKICAgICBkZWYgX19lcV9fKHNlbGYsIG90aGVyKToKICAgICAgICAgcmV0dXJuIChz ZWxmLnRlc3RfbmFtZSA9PSBvdGhlci50ZXN0X25hbWUgYW5kCmRpZmYgLS1naXQgYS9Ub29scy9T Y3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9ydW5fd2Via2l0X3Rlc3RzX2ludGVncmF0aW9u dGVzdC5weSBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3J1bl93ZWJraXRf dGVzdHNfaW50ZWdyYXRpb250ZXN0LnB5CmluZGV4IGNkMTZjMzI2ZjQxOGZlMWRkMTBlNTQ3YTcy MTk4OWEyOTA0OWRlNWMuLjYyNTNiYmUyMDg3NjM2YzQ4ZDUwMDgyOWI1Y2IyYmEwYjMzMTQzMmMg MTAwNjQ0Ci0tLSBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3J1bl93ZWJr aXRfdGVzdHNfaW50ZWdyYXRpb250ZXN0LnB5CisrKyBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkv bGF5b3V0X3Rlc3RzL3J1bl93ZWJraXRfdGVzdHNfaW50ZWdyYXRpb250ZXN0LnB5CkBAIC01ODEs NyArNTgxLDYgQEAgY2xhc3MgTWFpblRlc3QodW5pdHRlc3QuVGVzdENhc2UsIFN0cmVhbVRlc3Rp bmdNaXhpbik6CiAgICAgICAgICAgICAnZmFpbHVyZXMvdW5leHBlY3RlZC90ZXh0LWltYWdlLWNo ZWNrc3VtLmh0bWwnXSwKICAgICAgICAgICAgIHRlc3RzX2luY2x1ZGVkPVRydWUsIGhvc3Q9aG9z dCkKICAgICAgICAgZmlsZV9saXN0ID0gaG9zdC5maWxlc3lzdGVtLndyaXR0ZW5fZmlsZXMua2V5 cygpCi0gICAgICAgIGZpbGVfbGlzdC5yZW1vdmUoJy90bXAvbGF5b3V0LXRlc3QtcmVzdWx0cy90 ZXN0c19ydW4wLnR4dCcpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwocmVzLCAxKQogICAgICAg ICBleHBlY3RlZF90b2tlbiA9ICcidW5leHBlY3RlZCI6eyJ0ZXh0LWltYWdlLWNoZWNrc3VtLmh0 bWwiOnsiZXhwZWN0ZWQiOiJQQVNTIiwiYWN0dWFsIjoiSU1BR0UrVEVYVCIsImltYWdlX2RpZmZf cGVyY2VudCI6MX0sIm1pc3NpbmdfdGV4dC5odG1sIjp7ImV4cGVjdGVkIjoiUEFTUyIsImlzX21p c3NpbmdfdGV4dCI6dHJ1ZSwiYWN0dWFsIjoiTUlTU0lORyJ9JwogICAgICAgICBqc29uX3N0cmlu ZyA9IGhvc3QuZmlsZXN5c3RlbS5yZWFkX3RleHRfZmlsZSgnL3RtcC9sYXlvdXQtdGVzdC1yZXN1 bHRzL2Z1bGxfcmVzdWx0cy5qc29uJykKQEAgLTczMCw3ICs3MjksNiBAQCBjbGFzcyBNYWluVGVz dCh1bml0dGVzdC5UZXN0Q2FzZSwgU3RyZWFtVGVzdGluZ01peGluKToKICAgICAgICAgc2VsZi5h c3NlcnRFcXVhbChyZXMsIDApCiAgICAgICAgIHNlbGYuYXNzZXJ0VHJ1ZSgnUmV0cnlpbmcnIGlu IGVyci5nZXR2YWx1ZSgpKQogICAgICAgICBzZWxmLmFzc2VydFRydWUoaG9zdC5maWxlc3lzdGVt LmV4aXN0cygnL3RtcC9sYXlvdXQtdGVzdC1yZXN1bHRzL2ZhaWx1cmVzL2ZsYWt5L3RleHQtYWN0 dWFsLnR4dCcpKQotICAgICAgICBzZWxmLmFzc2VydFRydWUoaG9zdC5maWxlc3lzdGVtLmV4aXN0 cygnL3RtcC9sYXlvdXQtdGVzdC1yZXN1bHRzL3JldHJpZXMvdGVzdHNfcnVuMC50eHQnKSkKICAg ICAgICAgc2VsZi5hc3NlcnRGYWxzZShob3N0LmZpbGVzeXN0ZW0uZXhpc3RzKCcvdG1wL2xheW91 dC10ZXN0LXJlc3VsdHMvcmV0cmllcy9mYWlsdXJlcy9mbGFreS90ZXh0LWFjdHVhbC50eHQnKSkK IAogICAgICAgICAjIE5vdyB3ZSB0ZXN0IHRoYXQgLS1jbG9iYmVyLW9sZC1yZXN1bHRzIGRvZXMg cmVtb3ZlIHRoZSBvbGQgZW50cmllcyBhbmQgdGhlIG9sZCByZXRyaWVzLApAQCAtOTIwLDEzICs5 MTgsMTggQEAgY2xhc3MgRW5kVG9FbmRUZXN0KHVuaXR0ZXN0LlRlc3RDYXNlKToKICAgICAgICAg IyBDaGVjayB0aGF0IHdlIGF0dGVtcHRlZCB0byBkaXNwbGF5IHRoZSByZXN1bHRzIHBhZ2UgaW4g YSBicm93c2VyLgogICAgICAgICBzZWxmLmFzc2VydFRydWUodXNlci5vcGVuZWRfdXJscykKIAor ICAgICAgICAjIENoZWNrIHRoYXQgd2UgcmVjb3JkZWQgdGhlIHRlc3QgcnVuIHRpbWVzIGFuZCBv cmRlcmluZy4gTm90ZSB0aGF0CisgICAgICAgICMgcHJldHR5IG11Y2ggbm9uZSBvZiB0aGUgYWN0 dWFsIHZhbHVlcyBjYW4gYmUgZ3VhcmFudGVlZCwgYnV0IGF0IGxlYXN0CisgICAgICAgICMgd2Ug Y2FuIHRlc3QgdGhhdCB0aGV5J3JlIHRoZXJlLgorICAgICAgICBzdGF0cyA9IGpzb24ubG9hZHMo aG9zdC5maWxlc3lzdGVtLnJlYWRfdGV4dF9maWxlKCcvdG1wL2xheW91dC10ZXN0LXJlc3VsdHMv c3RhdHMuanNvbicpKQorICAgICAgICBzZWxmLmFzc2VydEVxdWFsKGxlbihzdGF0c1snaHR0cCdd Wyd0ZXN0cyddWydwYXNzZXMnXVsnaW1hZ2UuaHRtbCddWydyZXN1bHRzJ10pLCA1KQorCiAgICAg ZGVmIHRlc3RfcmVmdGVzdF93aXRoX3R3b19ub3RyZWZzKHNlbGYpOgogICAgICAgICAjIFRlc3Qg dGhhdCB3ZSB1cGRhdGUgZXhwZWN0YXRpb25zIGluIHBsYWNlLiBJZiB0aGUgZXhwZWN0YXRpb24K ICAgICAgICAgIyBpcyBtaXNzaW5nLCB1cGRhdGUgdGhlIGV4cGVjdGVkIGdlbmVyaWMgbG9jYXRp b24uCiAgICAgICAgIGhvc3QgPSBNb2NrSG9zdCgpCiAgICAgICAgIHJlcywgXywgXyA9IGxvZ2dp bmdfcnVuKFsnLS1uby1zaG93LXJlc3VsdHMnLCAncmVmdGVzdHMvZm9vLyddLCB0ZXN0c19pbmNs dWRlZD1UcnVlLCBob3N0PWhvc3QpCiAgICAgICAgIGZpbGVfbGlzdCA9IGhvc3QuZmlsZXN5c3Rl bS53cml0dGVuX2ZpbGVzLmtleXMoKQotICAgICAgICBmaWxlX2xpc3QucmVtb3ZlKCcvdG1wL2xh eW91dC10ZXN0LXJlc3VsdHMvdGVzdHNfcnVuMC50eHQnKQogICAgICAgICBqc29uX3N0cmluZyA9 IGhvc3QuZmlsZXN5c3RlbS5yZWFkX3RleHRfZmlsZSgnL3RtcC9sYXlvdXQtdGVzdC1yZXN1bHRz L2Z1bGxfcmVzdWx0cy5qc29uJykKICAgICAgICAganNvbiA9IHNlbGYucGFyc2VfZnVsbF9yZXN1 bHRzKGpzb25fc3RyaW5nKQogICAgICAgICBzZWxmLmFzc2VydFRydWUoIm11bHRpcGxlLW1hdGNo LXN1Y2Nlc3MuaHRtbCIgbm90IGluIGpzb25bInRlc3RzIl1bInJlZnRlc3RzIl1bImZvbyJdKQpA QCAtOTYwLDkgKzk2Myw4IEBAIGNsYXNzIFJlYmFzZWxpbmVUZXN0KHVuaXR0ZXN0LlRlc3RDYXNl LCBTdHJlYW1UZXN0aW5nTWl4aW4pOgogICAgICAgICAgICAgWyctLXBpeGVsLXRlc3RzJywgJy0t cmVzZXQtcmVzdWx0cycsICdwYXNzZXMvaW1hZ2UuaHRtbCcsICdmYWlsdXJlcy9leHBlY3RlZC9t aXNzaW5nX2ltYWdlLmh0bWwnXSwKICAgICAgICAgICAgIHRlc3RzX2luY2x1ZGVkPVRydWUsIGhv c3Q9aG9zdCwgbmV3X3Jlc3VsdHM9VHJ1ZSkKICAgICAgICAgZmlsZV9saXN0ID0gaG9zdC5maWxl c3lzdGVtLndyaXR0ZW5fZmlsZXMua2V5cygpCi0gICAgICAgIGZpbGVfbGlzdC5yZW1vdmUoJy90 bXAvbGF5b3V0LXRlc3QtcmVzdWx0cy90ZXN0c19ydW4wLnR4dCcpCiAgICAgICAgIHNlbGYuYXNz ZXJ0RXF1YWwocmVzLCAwKQotICAgICAgICBzZWxmLmFzc2VydEVxdWFsKGxlbihmaWxlX2xpc3Qp LCA3KQorICAgICAgICBzZWxmLmFzc2VydEVxdWFsKGxlbihmaWxlX2xpc3QpLCA4KQogICAgICAg ICBzZWxmLmFzc2VydEJhc2VsaW5lcyhmaWxlX2xpc3QsICJwYXNzZXMvaW1hZ2UiLCBbIi50eHQi LCAiLnBuZyJdLCBlcnIpCiAgICAgICAgIHNlbGYuYXNzZXJ0QmFzZWxpbmVzKGZpbGVfbGlzdCwg ImZhaWx1cmVzL2V4cGVjdGVkL21pc3NpbmdfaW1hZ2UiLCBbIi50eHQiLCAiLnBuZyJdLCBlcnIp CiAKQEAgLTk3Nyw5ICs5NzksOCBAQCBjbGFzcyBSZWJhc2VsaW5lVGVzdCh1bml0dGVzdC5UZXN0 Q2FzZSwgU3RyZWFtVGVzdGluZ01peGluKToKICAgICAgICAgICAgICdmYWlsdXJlcy91bmV4cGVj dGVkL21pc3NpbmdfcmVuZGVyX3RyZWVfZHVtcC5odG1sJ10sCiAgICAgICAgICAgICB0ZXN0c19p bmNsdWRlZD1UcnVlLCBob3N0PWhvc3QsIG5ld19yZXN1bHRzPVRydWUpCiAgICAgICAgIGZpbGVf bGlzdCA9IGhvc3QuZmlsZXN5c3RlbS53cml0dGVuX2ZpbGVzLmtleXMoKQotICAgICAgICBmaWxl X2xpc3QucmVtb3ZlKCcvdG1wL2xheW91dC10ZXN0LXJlc3VsdHMvdGVzdHNfcnVuMC50eHQnKQog ICAgICAgICBzZWxmLmFzc2VydEVxdWFsKHJlcywgMCkKLSAgICAgICAgc2VsZi5hc3NlcnRFcXVh bChsZW4oZmlsZV9saXN0KSwgOSkKKyAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChsZW4oZmlsZV9s aXN0KSwgMTApCiAgICAgICAgIHNlbGYuYXNzZXJ0QmFzZWxpbmVzKGZpbGVfbGlzdCwgImZhaWx1 cmVzL3VuZXhwZWN0ZWQvbWlzc2luZ190ZXh0IiwgWyIudHh0Il0sIGVycikKICAgICAgICAgc2Vs Zi5hc3NlcnRCYXNlbGluZXMoZmlsZV9saXN0LCAicGxhdGZvcm0vdGVzdC9mYWlsdXJlcy91bmV4 cGVjdGVkL21pc3NpbmdfaW1hZ2UiLCBbIi5wbmciXSwgZXJyKQogICAgICAgICBzZWxmLmFzc2Vy dEJhc2VsaW5lcyhmaWxlX2xpc3QsICJwbGF0Zm9ybS90ZXN0L2ZhaWx1cmVzL3VuZXhwZWN0ZWQv bWlzc2luZ19yZW5kZXJfdHJlZV9kdW1wIiwgWyIudHh0Il0sIGVycikKQEAgLTk5Miw5ICs5OTMs OCBAQCBjbGFzcyBSZWJhc2VsaW5lVGVzdCh1bml0dGVzdC5UZXN0Q2FzZSwgU3RyZWFtVGVzdGlu Z01peGluKToKICAgICAgICAgICAgIFsnLS1waXhlbC10ZXN0cycsICctLW5ldy1iYXNlbGluZScs ICdwYXNzZXMvaW1hZ2UuaHRtbCcsICdmYWlsdXJlcy9leHBlY3RlZC9taXNzaW5nX2ltYWdlLmh0 bWwnXSwKICAgICAgICAgICAgIHRlc3RzX2luY2x1ZGVkPVRydWUsIGhvc3Q9aG9zdCwgbmV3X3Jl c3VsdHM9VHJ1ZSkKICAgICAgICAgZmlsZV9saXN0ID0gaG9zdC5maWxlc3lzdGVtLndyaXR0ZW5f ZmlsZXMua2V5cygpCi0gICAgICAgIGZpbGVfbGlzdC5yZW1vdmUoJy90bXAvbGF5b3V0LXRlc3Qt cmVzdWx0cy90ZXN0c19ydW4wLnR4dCcpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwocmVzLCAw KQotICAgICAgICBzZWxmLmFzc2VydEVxdWFsKGxlbihmaWxlX2xpc3QpLCA3KQorICAgICAgICBz ZWxmLmFzc2VydEVxdWFsKGxlbihmaWxlX2xpc3QpLCA4KQogICAgICAgICBzZWxmLmFzc2VydEJh c2VsaW5lcyhmaWxlX2xpc3QsCiAgICAgICAgICAgICAicGxhdGZvcm0vdGVzdC1tYWMtbGVvcGFy ZC9wYXNzZXMvaW1hZ2UiLCBbIi50eHQiLCAiLnBuZyJdLCBlcnIpCiAgICAgICAgIHNlbGYuYXNz ZXJ0QmFzZWxpbmVzKGZpbGVfbGlzdCwK