111726 2013-03-07 07:04:02 -0800 run-pert-tests should have a --test-runner-count option 2013-03-07 20:14:03 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 77037 1 glenn glenn abarth benjamin dpranke eric rniwa webkit.review.bot oldest_to_newest 850051 0 glenn 2013-03-07 07:04:02 -0800 Given the recent addition of process_run_count argument to PerfTest(), which controls the number of samples taken when performing an individual test in a test set, the value of this argument should be exposed as a command line option, e.g., --sampleCount. Note that this is different from the --repeat argument proposed in bug 100030, which controls the number of times the entire test set is run. In particular, the number of samples (controlled by process_run_count) determine the number of data points used in statistical analysis; while different --repeat runs are not aggregated for statistical analysis in the shown results, i.e., are treated as distinct statistical result sets. 850241 1 rniwa 2013-03-07 10:27:35 -0800 Let's stick to the same terminology: process run count. 850487 2 glenn 2013-03-07 14:26:52 -0800 (In reply to comment #1) > Let's stick to the same terminology: process run count. thanks, we agreed on IRC to change this to test-runner-count (test_runner_count), for option and argument names, since this corresponds to the number of test runner (DRT/WTR) instances invoked (in serial execution - not parallel) 850755 3 192128 glenn 2013-03-07 19:09:59 -0800 Created attachment 192128 Patch 850810 4 192128 webkit.review.bot 2013-03-07 20:13:59 -0800 Comment on attachment 192128 Patch Clearing flags on attachment: 192128 Committed r145173: <http://trac.webkit.org/changeset/145173> 850811 5 webkit.review.bot 2013-03-07 20:14:03 -0800 All reviewed patches have been landed. Closing bug. 192128 2013-03-07 19:09:59 -0800 2013-03-07 20:13:59 -0800 Patch bug-111726-20130307200602.patch text/plain 13077 glenn U3VidmVyc2lvbiBSZXZpc2lvbjogMTQ1MTYyCmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggMjY2YzRmMWJlNjVmNTBiNjE4ZWYzOGMyNDQ1ZjlkNDgw NmE4NTZhYS4uODRhNjUyZGJhNzUxMTJlZWY4ZjI3NTdmNDdhZTQ1MDEyOTkyMGNjNiAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDM0 IEBACisyMDEzLTAzLTA3ICBHbGVubiBBZGFtcyAgPGdsZW5uQHNreW5hdi5jb20+CisKKyAgICAg ICAgcnVuLXBlcnQtdGVzdHMgc2hvdWxkIGhhdmUgYSAtLXRlc3QtcnVubmVyLWNvdW50IG9wdGlv bgorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MTExNzI2 CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgQWRkIC0t dGVzdC1ydW5uZXItY291bnQgb3B0aW9uIHRvIHJ1bi1wZXJmLXRlc3RzLCB3aXRoIGRlZmF1bHQg dmFsdWUgb2YgNC4KKyAgICAgICAgVGhpcyBvcHRpb24gYWxsb3dzIGNvbW1hbmQgbGluZSBjb25m aWd1cmF0aW9uIG9mIHRoZSBudW1iZXIgb2YgdGltZXMgdG8gaW52b2tlCisgICAgICAgIHRoZSB0 ZXN0IHJ1bm5lciAoc2VyaWFsbHksIG5vdCBpbiBwYXJhbGxlbCkgd2hlbiBydW5uaW5nIGEgc3Bl Y2lmaWMgdGVzdCBpbgorICAgICAgICBhIHRlc3Qgc2V0LgorCisgICAgICAgICogU2NyaXB0cy93 ZWJraXRweS9wZXJmb3JtYW5jZV90ZXN0cy9wZXJmdGVzdC5weToKKyAgICAgICAgKFBlcmZUZXN0 KTogQWRkIERFRkFVTFRfVEVTVF9SVU5ORVJfQ09VTlQgY29uc3RhbnQuCisgICAgICAgIChQZXJm VGVzdC5fX2luaXRfXyk6IFJlbmFtZSBwcm9jZXNzX3J1bl9jb3VudCB0byB0ZXN0X3J1bm5lcl9j b3VudCAodGhyb3VnaG91dCkuCisgICAgICAgIChQZXJmVGVzdC5ydW4pOiBSZW5hbWUgcHJvY2Vz c19ydW5fY291bnQgdG8gdGVzdF9ydW5uZXJfY291bnQuCisgICAgICAgIChTaW5nbGVQcm9jZXNz UGVyZlRlc3QuX19pbml0X18pOiBBZGQgdGVzdF9ydW5uZXJfY291bnQgb3B0aW9uYWwgYXJndW1l bnQuCisgICAgICAgIChDaHJvbWl1bVN0eWxlUGVyZlRlc3QuX19pbml0X18pOiBBZGQgdGVzdF9y dW5uZXJfY291bnQgb3B0aW9uYWwgYXJndW1lbnQuCisgICAgICAgIChSZXBsYXlQZXJmVGVzdC5f X2luaXRfXyk6IEFkZCB0ZXN0X3J1bm5lcl9jb3VudCBvcHRpb25hbCBhcmd1bWVudC4KKyAgICAg ICAgKFBlcmZUZXN0RmFjdG9yeS5jcmVhdGVfcGVyZl90ZXN0KTogQWRkIHRlc3RfcnVubmVyX2Nv dW50IG9wdGlvbmFsIGFyZ3VtZW50LgorICAgICAgICAqIFNjcmlwdHMvd2Via2l0cHkvcGVyZm9y bWFuY2VfdGVzdHMvcGVyZnRlc3RzcnVubmVyLnB5OgorICAgICAgICAoUGVyZlRlc3RzUnVubmVy Ll9wYXJzZV9hcmdzKTogQWRkICctLXRlc3QtcnVubmVyLWNvdW50JyBvcHRpb24uCisgICAgICAg IChQZXJmVGVzdHNSdW5uZXIuX2NvbGxlY3RfdGVzdHMpOiBVc2UgdGVzdF9ydW5uZXJfY291bnQg d2hlbiBjcmVhdGluZyBQZXJmVGVzdC4KKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L3BlcmZv cm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1bm5lcl9pbnRlZ3JhdGlvbnRlc3QucHk6CisgICAgICAg IChNYWluVGVzdC5fdGVzdF9ydW5fd2l0aF9qc29uX291dHB1dCk6IEFkZCBvcHRpb25hbCBjb21w YXJlX2xvZ3MgYXJndW1lbnQuCisgICAgICAgIChNYWluVGVzdC50ZXN0X3J1bl93aXRoX3JlcGVh dCk6CisgICAgICAgIChNYWluVGVzdC50ZXN0X3J1bl93aXRoX3Rlc3RfcnVubmVyX2NvdW50KTog QWRkIG5ldyB0ZXN0IGZvciB0ZXN0LXJ1bm5lci1jb3VudCBvcHRpb24gc2VtYW50aWNzLgorICAg ICAgICAqIFNjcmlwdHMvd2Via2l0cHkvcGVyZm9ybWFuY2VfdGVzdHMvcGVyZnRlc3RzcnVubmVy X3VuaXR0ZXN0LnB5OgorICAgICAgICAoTWFpblRlc3QudGVzdF9kZWZhdWx0X2FyZ3MpOiBBZGQg dGVzdCBmb3IgZGVmYXVsdCBvcHRpb24gYXJndW1lbnQgdmFsdWVzLgorICAgICAgICAoTWFpblRl c3QudGVzdF9wYXJzZV9hcmdzKTogQWRkIHRlc3QgZm9yIHRlc3QtcnVubmVyLWNvdW50IG9wdGlv biBwYXJzaW5nLgorCiAyMDEzLTAzLTA3ICBDaHJpc3RpYW4gQmllc2luZ2VyICA8Y2JpZXNpbmdl ckBjaHJvbWl1bS5vcmc+CiAKICAgICAgICAgQWRkIG15c2VsZiB0byByZXZpZXdlcnMucHkKZGlm ZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvcGVyZm9ybWFuY2VfdGVzdHMvcGVyZnRl c3QucHkgYi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3BlcmZvcm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0 LnB5CmluZGV4IGQ1M2U3ZjEzN2EyOTMwMTg0Y2EwNzYwMTAyOGZkY2VmZDI0ZDMxNWUuLjRiMmUx YzlmODljNTA3ZGQzOTg3MmFiZWMwMWE0NzcyNjAzZDViOTYgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1Nj cmlwdHMvd2Via2l0cHkvcGVyZm9ybWFuY2VfdGVzdHMvcGVyZnRlc3QucHkKKysrIGIvVG9vbHMv U2NyaXB0cy93ZWJraXRweS9wZXJmb3JtYW5jZV90ZXN0cy9wZXJmdGVzdC5weQpAQCAtNDgsNiAr NDgsNyBAQCBmcm9tIHdlYmtpdHB5LmxheW91dF90ZXN0cy5jb250cm9sbGVycy50ZXN0X3Jlc3Vs dF93cml0ZXIgaW1wb3J0IFRlc3RSZXN1bHRXcml0ZQogZnJvbSB3ZWJraXRweS5sYXlvdXRfdGVz dHMucG9ydC5kcml2ZXIgaW1wb3J0IERyaXZlcklucHV0CiBmcm9tIHdlYmtpdHB5LmxheW91dF90 ZXN0cy5wb3J0LmRyaXZlciBpbXBvcnQgRHJpdmVyT3V0cHV0CiAKK0RFRkFVTFRfVEVTVF9SVU5O RVJfQ09VTlQgPSA0CiAKIF9sb2cgPSBsb2dnaW5nLmdldExvZ2dlcihfX25hbWVfXykKIApAQCAt ODksMTQgKzkwLDE1IEBAIGNsYXNzIFBlcmZUZXN0TWV0cmljKG9iamVjdCk6CiAKIAogY2xhc3Mg UGVyZlRlc3Qob2JqZWN0KToKLSAgICBkZWYgX19pbml0X18oc2VsZiwgcG9ydCwgdGVzdF9uYW1l LCB0ZXN0X3BhdGgsIHByb2Nlc3NfcnVuX2NvdW50PTQpOgorCisgICAgZGVmIF9faW5pdF9fKHNl bGYsIHBvcnQsIHRlc3RfbmFtZSwgdGVzdF9wYXRoLCB0ZXN0X3J1bm5lcl9jb3VudD1ERUZBVUxU X1RFU1RfUlVOTkVSX0NPVU5UKToKICAgICAgICAgc2VsZi5fcG9ydCA9IHBvcnQKICAgICAgICAg c2VsZi5fdGVzdF9uYW1lID0gdGVzdF9uYW1lCiAgICAgICAgIHNlbGYuX3Rlc3RfcGF0aCA9IHRl c3RfcGF0aAogICAgICAgICBzZWxmLl9kZXNjcmlwdGlvbiA9IE5vbmUKICAgICAgICAgc2VsZi5f bWV0cmljcyA9IHt9CiAgICAgICAgIHNlbGYuX29yZGVyZWRfbWV0cmljc19uYW1lID0gW10KLSAg ICAgICAgc2VsZi5fcHJvY2Vzc19ydW5fY291bnQgPSBwcm9jZXNzX3J1bl9jb3VudAorICAgICAg ICBzZWxmLl90ZXN0X3J1bm5lcl9jb3VudCA9IHRlc3RfcnVubmVyX2NvdW50CiAKICAgICBkZWYg dGVzdF9uYW1lKHNlbGYpOgogICAgICAgICByZXR1cm4gc2VsZi5fdGVzdF9uYW1lCkBAIC0xMTcs NyArMTE5LDcgQEAgY2xhc3MgUGVyZlRlc3Qob2JqZWN0KToKICAgICAgICAgcmV0dXJuIHNlbGYu X3BvcnQuY3JlYXRlX2RyaXZlcih3b3JrZXJfbnVtYmVyPTAsIG5vX3RpbWVvdXQ9VHJ1ZSkKIAog ICAgIGRlZiBydW4oc2VsZiwgdGltZV9vdXRfbXMpOgotICAgICAgICBmb3IgXyBpbiB4cmFuZ2Uo c2VsZi5fcHJvY2Vzc19ydW5fY291bnQpOgorICAgICAgICBmb3IgXyBpbiB4cmFuZ2Uoc2VsZi5f dGVzdF9ydW5uZXJfY291bnQpOgogICAgICAgICAgICAgZHJpdmVyID0gc2VsZi5fY3JlYXRlX2Ry aXZlcigpCiAgICAgICAgICAgICB0cnk6CiAgICAgICAgICAgICAgICAgaWYgbm90IHNlbGYuX3J1 bl93aXRoX2RyaXZlcihkcml2ZXIsIHRpbWVfb3V0X21zKToKQEAgLTI1OSwxNSArMjYxLDE1IEBA IGNsYXNzIFBlcmZUZXN0KG9iamVjdCk6CiAKIAogY2xhc3MgU2luZ2xlUHJvY2Vzc1BlcmZUZXN0 KFBlcmZUZXN0KToKLSAgICBkZWYgX19pbml0X18oc2VsZiwgcG9ydCwgdGVzdF9uYW1lLCB0ZXN0 X3BhdGgpOgotICAgICAgICBzdXBlcihTaW5nbGVQcm9jZXNzUGVyZlRlc3QsIHNlbGYpLl9faW5p dF9fKHBvcnQsIHRlc3RfbmFtZSwgdGVzdF9wYXRoLCBwcm9jZXNzX3J1bl9jb3VudD0xKQorICAg IGRlZiBfX2luaXRfXyhzZWxmLCBwb3J0LCB0ZXN0X25hbWUsIHRlc3RfcGF0aCwgdGVzdF9ydW5u ZXJfY291bnQ9MSk6CisgICAgICAgIHN1cGVyKFNpbmdsZVByb2Nlc3NQZXJmVGVzdCwgc2VsZiku X19pbml0X18ocG9ydCwgdGVzdF9uYW1lLCB0ZXN0X3BhdGgsIHRlc3RfcnVubmVyX2NvdW50KQog CiAKIGNsYXNzIENocm9taXVtU3R5bGVQZXJmVGVzdChQZXJmVGVzdCk6CiAgICAgX2Nocm9taXVt X3N0eWxlX3Jlc3VsdF9yZWdleCA9IHJlLmNvbXBpbGUocideUkVTVUxUXHMrKD9QPG5hbWU+W149 XSspXHMqPVxzKyg/UDx2YWx1ZT5cZCsoXC5cZCspPylccyooP1A8dW5pdD5cdyspJCcpCiAKLSAg ICBkZWYgX19pbml0X18oc2VsZiwgcG9ydCwgdGVzdF9uYW1lLCB0ZXN0X3BhdGgpOgotICAgICAg ICBzdXBlcihDaHJvbWl1bVN0eWxlUGVyZlRlc3QsIHNlbGYpLl9faW5pdF9fKHBvcnQsIHRlc3Rf bmFtZSwgdGVzdF9wYXRoKQorICAgIGRlZiBfX2luaXRfXyhzZWxmLCBwb3J0LCB0ZXN0X25hbWUs IHRlc3RfcGF0aCwgdGVzdF9ydW5uZXJfY291bnQ9REVGQVVMVF9URVNUX1JVTk5FUl9DT1VOVCk6 CisgICAgICAgIHN1cGVyKENocm9taXVtU3R5bGVQZXJmVGVzdCwgc2VsZikuX19pbml0X18ocG9y dCwgdGVzdF9uYW1lLCB0ZXN0X3BhdGgsIHRlc3RfcnVubmVyX2NvdW50KQogCiAgICAgZGVmIHJ1 bihzZWxmLCB0aW1lX291dF9tcyk6CiAgICAgICAgIGRyaXZlciA9IHNlbGYuX2NyZWF0ZV9kcml2 ZXIoKQpAQCAtMzM1LDggKzMzNyw4IEBAIGNsYXNzIFJlcGxheVNlcnZlcihvYmplY3QpOgogY2xh c3MgUmVwbGF5UGVyZlRlc3QoUGVyZlRlc3QpOgogICAgIF9GT1JDRV9HQ19GSUxFID0gJ3Jlc291 cmNlcy9mb3JjZS1nYy5odG1sJwogCi0gICAgZGVmIF9faW5pdF9fKHNlbGYsIHBvcnQsIHRlc3Rf bmFtZSwgdGVzdF9wYXRoKToKLSAgICAgICAgc3VwZXIoUmVwbGF5UGVyZlRlc3QsIHNlbGYpLl9f aW5pdF9fKHBvcnQsIHRlc3RfbmFtZSwgdGVzdF9wYXRoKQorICAgIGRlZiBfX2luaXRfXyhzZWxm LCBwb3J0LCB0ZXN0X25hbWUsIHRlc3RfcGF0aCwgdGVzdF9ydW5uZXJfY291bnQ9REVGQVVMVF9U RVNUX1JVTk5FUl9DT1VOVCk6CisgICAgICAgIHN1cGVyKFJlcGxheVBlcmZUZXN0LCBzZWxmKS5f X2luaXRfXyhwb3J0LCB0ZXN0X25hbWUsIHRlc3RfcGF0aCwgdGVzdF9ydW5uZXJfY291bnQpCiAg ICAgICAgIHNlbGYuZm9yY2VfZ2NfdGVzdCA9IHNlbGYuX3BvcnQuaG9zdC5maWxlc3lzdGVtLmpv aW4oc2VsZi5fcG9ydC5wZXJmX3Rlc3RzX2RpcigpLCBzZWxmLl9GT1JDRV9HQ19GSUxFKQogCiAg ICAgZGVmIF9zdGFydF9yZXBsYXlfc2VydmVyKHNlbGYsIGFyY2hpdmUsIHJlY29yZCk6CkBAIC00 NTYsOCArNDU4LDggQEAgY2xhc3MgUGVyZlRlc3RGYWN0b3J5KG9iamVjdCk6CiAgICAgXQogCiAg ICAgQGNsYXNzbWV0aG9kCi0gICAgZGVmIGNyZWF0ZV9wZXJmX3Rlc3QoY2xzLCBwb3J0LCB0ZXN0 X25hbWUsIHBhdGgpOgorICAgIGRlZiBjcmVhdGVfcGVyZl90ZXN0KGNscywgcG9ydCwgdGVzdF9u YW1lLCBwYXRoLCB0ZXN0X3J1bm5lcl9jb3VudD1ERUZBVUxUX1RFU1RfUlVOTkVSX0NPVU5UKToK ICAgICAgICAgZm9yIChwYXR0ZXJuLCB0ZXN0X2NsYXNzKSBpbiBjbHMuX3BhdHRlcm5fbWFwOgog ICAgICAgICAgICAgaWYgcGF0dGVybi5tYXRjaCh0ZXN0X25hbWUpOgotICAgICAgICAgICAgICAg IHJldHVybiB0ZXN0X2NsYXNzKHBvcnQsIHRlc3RfbmFtZSwgcGF0aCkKLSAgICAgICAgcmV0dXJu IFBlcmZUZXN0KHBvcnQsIHRlc3RfbmFtZSwgcGF0aCkKKyAgICAgICAgICAgICAgICByZXR1cm4g dGVzdF9jbGFzcyhwb3J0LCB0ZXN0X25hbWUsIHBhdGgsIHRlc3RfcnVubmVyX2NvdW50KQorICAg ICAgICByZXR1cm4gUGVyZlRlc3QocG9ydCwgdGVzdF9uYW1lLCBwYXRoLCB0ZXN0X3J1bm5lcl9j b3VudCkKZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvcGVyZm9ybWFuY2VfdGVz dHMvcGVyZnRlc3RzcnVubmVyLnB5IGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wZXJmb3JtYW5j ZV90ZXN0cy9wZXJmdGVzdHNydW5uZXIucHkKaW5kZXggYTc4OWU3N2U3MGFjYTU3M2NlYzY3MDJm OWYzMWQ2MGMzODQ5NzExOC4uYzU4ODIxMjJkNzU2ZGE5NzU4MjU0ZWNmYmNmY2Q1MGZmZDAwZDNh ZCAxMDA2NDQKLS0tIGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wZXJmb3JtYW5jZV90ZXN0cy9w ZXJmdGVzdHNydW5uZXIucHkKKysrIGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wZXJmb3JtYW5j ZV90ZXN0cy9wZXJmdGVzdHNydW5uZXIucHkKQEAgLTQxLDYgKzQxLDcgQEAgZnJvbSB3ZWJraXRw eS5jb21tb24uY29uZmlnLnVybHMgaW1wb3J0IHZpZXdfc291cmNlX3VybAogZnJvbSB3ZWJraXRw eS5jb21tb24uaG9zdCBpbXBvcnQgSG9zdAogZnJvbSB3ZWJraXRweS5jb21tb24ubmV0LmZpbGVf dXBsb2FkZXIgaW1wb3J0IEZpbGVVcGxvYWRlcgogZnJvbSB3ZWJraXRweS5wZXJmb3JtYW5jZV90 ZXN0cy5wZXJmdGVzdCBpbXBvcnQgUGVyZlRlc3RGYWN0b3J5Citmcm9tIHdlYmtpdHB5LnBlcmZv cm1hbmNlX3Rlc3RzLnBlcmZ0ZXN0IGltcG9ydCBERUZBVUxUX1RFU1RfUlVOTkVSX0NPVU5UCiAK IAogX2xvZyA9IGxvZ2dpbmcuZ2V0TG9nZ2VyKF9fbmFtZV9fKQpAQCAtMTI4LDYgKzEyOSw4IEBA IGNsYXNzIFBlcmZUZXN0c1J1bm5lcihvYmplY3QpOgogICAgICAgICAgICAgICAgICAgICAgIlNw ZWNpZnkgbXVsdGlwbGUgdGltZXMgdG8gYWRkIG11bHRpcGxlIGZsYWdzLiIpLAogICAgICAgICAg ICAgb3B0cGFyc2UubWFrZV9vcHRpb24oIi0tcmVwZWF0IiwgZGVmYXVsdD0xLCB0eXBlPSJpbnQi LAogICAgICAgICAgICAgICAgIGhlbHA9IlNwZWNpZnkgbnVtYmVyIG9mIHRpbWVzIHRvIHJ1biB0 ZXN0IHNldCAoZGVmYXVsdDogMSkuIiksCisgICAgICAgICAgICBvcHRwYXJzZS5tYWtlX29wdGlv bigiLS10ZXN0LXJ1bm5lci1jb3VudCIsIGRlZmF1bHQ9REVGQVVMVF9URVNUX1JVTk5FUl9DT1VO VCwgdHlwZT0iaW50IiwKKyAgICAgICAgICAgICAgICBoZWxwPSJTcGVjaWZ5IG51bWJlciBvZiB0 aW1lcyB0byBpbnZva2UgdGVzdCBydW5uZXIgZm9yIGVhY2ggcGVyZm9ybWFuY2UgdGVzdC4iKSwK ICAgICAgICAgICAgIF0KICAgICAgICAgcmV0dXJuIG9wdHBhcnNlLk9wdGlvblBhcnNlcihvcHRp b25fbGlzdD0ocGVyZl9vcHRpb25fbGlzdCkpLnBhcnNlX2FyZ3MoYXJncykKIApAQCAtMTU5LDcg KzE2Miw3IEBAIGNsYXNzIFBlcmZUZXN0c1J1bm5lcihvYmplY3QpOgogICAgICAgICAgICAgcmVs YXRpdmVfcGF0aCA9IGZpbGVzeXN0ZW0ucmVscGF0aChwYXRoLCBzZWxmLl9iYXNlX3BhdGgpLnJl cGxhY2UoJ1xcJywgJy8nKQogICAgICAgICAgICAgaWYgc2VsZi5fb3B0aW9ucy51c2Vfc2tpcHBl ZF9saXN0IGFuZCBzZWxmLl9wb3J0LnNraXBzX3BlcmZfdGVzdChyZWxhdGl2ZV9wYXRoKSBhbmQg ZmlsZXN5c3RlbS5ub3JtcGF0aChyZWxhdGl2ZV9wYXRoKSBub3QgaW4gcGF0aHM6CiAgICAgICAg ICAgICAgICAgY29udGludWUKLSAgICAgICAgICAgIHRlc3QgPSBQZXJmVGVzdEZhY3RvcnkuY3Jl YXRlX3BlcmZfdGVzdChzZWxmLl9wb3J0LCByZWxhdGl2ZV9wYXRoLCBwYXRoKQorICAgICAgICAg ICAgdGVzdCA9IFBlcmZUZXN0RmFjdG9yeS5jcmVhdGVfcGVyZl90ZXN0KHNlbGYuX3BvcnQsIHJl bGF0aXZlX3BhdGgsIHBhdGgsIHRlc3RfcnVubmVyX2NvdW50PXNlbGYuX29wdGlvbnMudGVzdF9y dW5uZXJfY291bnQpCiAgICAgICAgICAgICB0ZXN0cy5hcHBlbmQodGVzdCkKIAogICAgICAgICBy ZXR1cm4gdGVzdHMKZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvcGVyZm9ybWFu Y2VfdGVzdHMvcGVyZnRlc3RzcnVubmVyX2ludGVncmF0aW9udGVzdC5weSBiL1Rvb2xzL1Njcmlw dHMvd2Via2l0cHkvcGVyZm9ybWFuY2VfdGVzdHMvcGVyZnRlc3RzcnVubmVyX2ludGVncmF0aW9u dGVzdC5weQppbmRleCA4YTkwZmQ1MDAzMWQ3YmQ2MDYwODY1MjJiYjY1MTQ1OTQ0N2Q1NzQxLi44 YmMyMTQ4ZTRjYzQ5YTE2ZjI0OWJmM2Y5MWMwYzBjZWNmNTRiZDM0IDEwMDY0NAotLS0gYS9Ub29s cy9TY3JpcHRzL3dlYmtpdHB5L3BlcmZvcm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1bm5lcl9pbnRl Z3JhdGlvbnRlc3QucHkKKysrIGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wZXJmb3JtYW5jZV90 ZXN0cy9wZXJmdGVzdHNydW5uZXJfaW50ZWdyYXRpb250ZXN0LnB5CkBAIC0zMDUsNyArMzA1LDcg QEAgY2xhc3MgTWFpblRlc3QodW5pdHRlc3QuVGVzdENhc2UpOgogICAgICAgICBzZWxmLmFzc2Vy dEVxdWFsKHBhcnNlcl90ZXN0c1snbWVtb3J5LXRlc3QnXVsnbWV0cmljcyddWydKU0hlYXAnXSwg TWVtb3J5VGVzdERhdGEuanNfaGVhcF9yZXN1bHRzKQogICAgICAgICBzZWxmLmFzc2VydEVxdWFs KHBhcnNlcl90ZXN0c1snbWVtb3J5LXRlc3QnXVsnbWV0cmljcyddWydNYWxsb2MnXSwgTWVtb3J5 VGVzdERhdGEubWFsbG9jX3Jlc3VsdHMpCiAKLSAgICBkZWYgX3Rlc3RfcnVuX3dpdGhfanNvbl9v dXRwdXQoc2VsZiwgcnVubmVyLCBmaWxlc3lzdGVtLCB1cGxvYWRfc3VjY2VlZHM9RmFsc2UsIHJl c3VsdHNfc2hvd249VHJ1ZSwgZXhwZWN0ZWRfZXhpdF9jb2RlPTAsIHJlcGVhdD0xKToKKyAgICBk ZWYgX3Rlc3RfcnVuX3dpdGhfanNvbl9vdXRwdXQoc2VsZiwgcnVubmVyLCBmaWxlc3lzdGVtLCB1 cGxvYWRfc3VjY2VlZHM9RmFsc2UsIHJlc3VsdHNfc2hvd249VHJ1ZSwgZXhwZWN0ZWRfZXhpdF9j b2RlPTAsIHJlcGVhdD0xLCBjb21wYXJlX2xvZ3M9VHJ1ZSk6CiAgICAgICAgIGZpbGVzeXN0ZW0u d3JpdGVfdGV4dF9maWxlKHJ1bm5lci5fYmFzZV9wYXRoICsgJy9pbnNwZWN0b3IvcGFzcy5odG1s JywgJ3NvbWUgY29udGVudCcpCiAgICAgICAgIGZpbGVzeXN0ZW0ud3JpdGVfdGV4dF9maWxlKHJ1 bm5lci5fYmFzZV9wYXRoICsgJy9CaW5kaW5ncy9ldmVudC10YXJnZXQtd3JhcHBlci5odG1sJywg J3NvbWUgY29udGVudCcpCiAKQEAgLTMyOSw3ICszMjksNyBAQCBjbGFzcyBNYWluVGVzdCh1bml0 dGVzdC5UZXN0Q2FzZSk6CiAgICAgICAgIGZpbmFsbHk6CiAgICAgICAgICAgICBzdGRvdXQsIHN0 ZGVyciwgbG9ncyA9IG91dHB1dF9jYXB0dXJlLnJlc3RvcmVfb3V0cHV0KCkKIAotICAgICAgICBp ZiBub3QgZXhwZWN0ZWRfZXhpdF9jb2RlOgorICAgICAgICBpZiBub3QgZXhwZWN0ZWRfZXhpdF9j b2RlIGFuZCBjb21wYXJlX2xvZ3M6CiAgICAgICAgICAgICBleHBlY3RlZF9sb2dzID0gJycKICAg ICAgICAgICAgIGZvciBpIGluIHhyYW5nZShyZXBlYXQpOgogICAgICAgICAgICAgICAgIHJ1bnMg PSAnIChSdW4gJWQgb2YgJWQpJyAlIChpICsgMSwgcmVwZWF0KSBpZiByZXBlYXQgPiAxIGVsc2Ug JycKQEAgLTU2NiwzICs1NjYsMTcgQEAgY2xhc3MgTWFpblRlc3QodW5pdHRlc3QuVGVzdENhc2Up OgogICAgICAgICAgICAgeyJidWlsZFRpbWUiOiAiMjAxMy0wMi0wOFQxNToxOTozNy40NjAwMDAi LAogICAgICAgICAgICAgInRlc3RzIjogc2VsZi5fZXZlbnRfdGFyZ2V0X3dyYXBwZXJfYW5kX2lu c3BlY3Rvcl9yZXN1bHRzLAogICAgICAgICAgICAgInJldmlzaW9ucyI6IHsiV2ViS2l0IjogeyJ0 aW1lc3RhbXAiOiAiMjAxMy0wMi0wMSAwODo0ODowNSArMDAwMCIsICJyZXZpc2lvbiI6ICI1Njc4 In19fV0pCisKKyAgICBkZWYgdGVzdF9ydW5fd2l0aF90ZXN0X3J1bm5lcl9jb3VudChzZWxmKToK KyAgICAgICAgcnVubmVyLCBwb3J0ID0gc2VsZi5jcmVhdGVfcnVubmVyX2FuZF9zZXR1cF9yZXN1 bHRzX3RlbXBsYXRlKGFyZ3M9WyctLW91dHB1dC1qc29uLXBhdGg9L21vY2stY2hlY2tvdXQvb3V0 cHV0Lmpzb24nLAorICAgICAgICAgICAgJy0tdGVzdC1ydW5uZXItY291bnQ9MyddKQorICAgICAg ICBzZWxmLl90ZXN0X3J1bl93aXRoX2pzb25fb3V0cHV0KHJ1bm5lciwgcG9ydC5ob3N0LmZpbGVz eXN0ZW0sIGNvbXBhcmVfbG9ncz1GYWxzZSkKKyAgICAgICAgZ2VuZXJhdGVkX2pzb24gPSBqc29u LmxvYWRzKHBvcnQuaG9zdC5maWxlc3lzdGVtLmZpbGVzWycvbW9jay1jaGVja291dC9vdXRwdXQu anNvbiddKQorICAgICAgICBzZWxmLmFzc2VydFRydWUoaXNpbnN0YW5jZShnZW5lcmF0ZWRfanNv biwgbGlzdCkpCisgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwobGVuKGdlbmVyYXRlZF9qc29uKSwg MSkKKworICAgICAgICBvdXRwdXQgPSBnZW5lcmF0ZWRfanNvblswXVsndGVzdHMnXVsnQmluZGlu Z3MnXVsndGVzdHMnXVsnZXZlbnQtdGFyZ2V0LXdyYXBwZXInXVsnbWV0cmljcyddWydUaW1lJ11b J2N1cnJlbnQnXQorICAgICAgICBzZWxmLmFzc2VydEVxdWFsKGxlbihvdXRwdXQpLCAzKQorICAg ICAgICBleHBlY3RlZE1ldHJpY3MgPSBFdmVudFRhcmdldFdyYXBwZXJUZXN0RGF0YS5yZXN1bHRz WydtZXRyaWNzJ11bJ1RpbWUnXVsnY3VycmVudCddWzBdCisgICAgICAgIGZvciBtZXRyaWNzIGlu IG91dHB1dDoKKyAgICAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwobWV0cmljcywgZXhwZWN0ZWRN ZXRyaWNzKQpkaWZmIC0tZ2l0IGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS9wZXJmb3JtYW5jZV90 ZXN0cy9wZXJmdGVzdHNydW5uZXJfdW5pdHRlc3QucHkgYi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5 L3BlcmZvcm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1bm5lcl91bml0dGVzdC5weQppbmRleCBmOWQ1 NTkxOWQzNmYzMWZjYzlmOWY1N2I4M2Q2MGViNDdhZTRkOTM5Li40ZTM4ZjUzMGQ0MTkxZmM5MDcw MTBlZThiOWRmYjE1NWExMGJjNDM0IDEwMDY0NAotLS0gYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5 L3BlcmZvcm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1bm5lcl91bml0dGVzdC5weQorKysgYi9Ub29s cy9TY3JpcHRzL3dlYmtpdHB5L3BlcmZvcm1hbmNlX3Rlc3RzL3BlcmZ0ZXN0c3J1bm5lcl91bml0 dGVzdC5weQpAQCAtMzYsNiArMzYsNyBAQCBpbXBvcnQgdW5pdHRlc3QyIGFzIHVuaXR0ZXN0CiBm cm9tIHdlYmtpdHB5LmNvbW1vbi5ob3N0X21vY2sgaW1wb3J0IE1vY2tIb3N0CiBmcm9tIHdlYmtp dHB5LmNvbW1vbi5zeXN0ZW0ub3V0cHV0Y2FwdHVyZSBpbXBvcnQgT3V0cHV0Q2FwdHVyZQogZnJv bSB3ZWJraXRweS5sYXlvdXRfdGVzdHMucG9ydC50ZXN0IGltcG9ydCBUZXN0UG9ydAorZnJvbSB3 ZWJraXRweS5wZXJmb3JtYW5jZV90ZXN0cy5wZXJmdGVzdCBpbXBvcnQgREVGQVVMVF9URVNUX1JV Tk5FUl9DT1VOVAogZnJvbSB3ZWJraXRweS5wZXJmb3JtYW5jZV90ZXN0cy5wZXJmdGVzdHNydW5u ZXIgaW1wb3J0IFBlcmZUZXN0c1J1bm5lcgogCiAKQEAgLTEyMiw2ICsxMjMsMTggQEAgY2xhc3Mg TWFpblRlc3QodW5pdHRlc3QuVGVzdENhc2UpOgogICAgICAgICBzZWxmLmFzc2VydEVxdWFsKGxl bih0ZXN0cyksIDEpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwodGVzdHNbMF0uX19jbGFzc19f Ll9fbmFtZV9fLCAnUmVwbGF5UGVyZlRlc3QnKQogCisgICAgZGVmIHRlc3RfZGVmYXVsdF9hcmdz KHNlbGYpOgorICAgICAgICBydW5uZXIsIHBvcnQgPSBzZWxmLmNyZWF0ZV9ydW5uZXIoKQorICAg ICAgICBvcHRpb25zLCBhcmdzID0gUGVyZlRlc3RzUnVubmVyLl9wYXJzZV9hcmdzKFtdKQorICAg ICAgICBzZWxmLmFzc2VydFRydWUob3B0aW9ucy5idWlsZCkKKyAgICAgICAgc2VsZi5hc3NlcnRF cXVhbChvcHRpb25zLnRpbWVfb3V0X21zLCA2MDAgKiAxMDAwKQorICAgICAgICBzZWxmLmFzc2Vy dFRydWUob3B0aW9ucy5nZW5lcmF0ZV9yZXN1bHRzKQorICAgICAgICBzZWxmLmFzc2VydFRydWUo b3B0aW9ucy5zaG93X3Jlc3VsdHMpCisgICAgICAgIHNlbGYuYXNzZXJ0RmFsc2Uob3B0aW9ucy5y ZXBsYXkpCisgICAgICAgIHNlbGYuYXNzZXJ0VHJ1ZShvcHRpb25zLnVzZV9za2lwcGVkX2xpc3Qp CisgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwob3B0aW9ucy5yZXBlYXQsIDEpCisgICAgICAgIHNl bGYuYXNzZXJ0RXF1YWwob3B0aW9ucy50ZXN0X3J1bm5lcl9jb3VudCwgREVGQVVMVF9URVNUX1JV Tk5FUl9DT1VOVCkKKwogICAgIGRlZiB0ZXN0X3BhcnNlX2FyZ3Moc2VsZik6CiAgICAgICAgIHJ1 bm5lciwgcG9ydCA9IHNlbGYuY3JlYXRlX3J1bm5lcigpCiAgICAgICAgIG9wdGlvbnMsIGFyZ3Mg PSBQZXJmVGVzdHNSdW5uZXIuX3BhcnNlX2FyZ3MoWwpAQCAtMTM4LDYgKzE1MSw3IEBAIGNsYXNz IE1haW5UZXN0KHVuaXR0ZXN0LlRlc3RDYXNlKToKICAgICAgICAgICAgICAgICAnLS1hZGRpdGlv bmFsLWRydC1mbGFnPS0tZW5hYmxlLXRocmVhZGVkLXBhcnNlcicsCiAgICAgICAgICAgICAgICAg Jy0tYWRkaXRpb25hbC1kcnQtZmxhZz0tLWF3ZXNvbWVzYXVjZScsCiAgICAgICAgICAgICAgICAg Jy0tcmVwZWF0PTUnLAorICAgICAgICAgICAgICAgICctLXRlc3QtcnVubmVyLWNvdW50PTUnLAog ICAgICAgICAgICAgICAgICctLWRlYnVnJ10pCiAgICAgICAgIHNlbGYuYXNzZXJ0VHJ1ZShvcHRp b25zLmJ1aWxkKQogICAgICAgICBzZWxmLmFzc2VydEVxdWFsKG9wdGlvbnMuYnVpbGRfZGlyZWN0 b3J5LCAnZm9sZGVyNDInKQpAQCAtMTUzLDYgKzE2Nyw3IEBAIGNsYXNzIE1haW5UZXN0KHVuaXR0 ZXN0LlRlc3RDYXNlKToKICAgICAgICAgc2VsZi5hc3NlcnRFcXVhbChvcHRpb25zLnRlc3RfcmVz dWx0c19zZXJ2ZXIsICdzb21laG9zdCcpCiAgICAgICAgIHNlbGYuYXNzZXJ0RXF1YWwob3B0aW9u cy5hZGRpdGlvbmFsX2RydF9mbGFnLCBbJy0tZW5hYmxlLXRocmVhZGVkLXBhcnNlcicsICctLWF3 ZXNvbWVzYXVjZSddKQogICAgICAgICBzZWxmLmFzc2VydEVxdWFsKG9wdGlvbnMucmVwZWF0LCA1 KQorICAgICAgICBzZWxmLmFzc2VydEVxdWFsKG9wdGlvbnMudGVzdF9ydW5uZXJfY291bnQsIDUp CiAKICAgICBkZWYgdGVzdF91cGxvYWRfanNvbihzZWxmKToKICAgICAgICAgcnVubmVyLCBwb3J0 ID0gc2VsZi5jcmVhdGVfcnVubmVyKCkK