180082 2017-11-28 08:15:19 -0800 WebDriver: add an option to dump test results to a json file 2017-11-28 23:27:30 -0800 1 1 1 Unclassified WebKit WebDriver WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 cgarcia webkit-unassigned bburg clopez ews-watchlist glenn webkit-bug-importer oldest_to_newest 1375344 0 cgarcia 2017-11-28 08:15:19 -0800 Add --json-output command line option to run-webdriver-tests to dump test results to a json file in a format compatible with the W3C report. See https://w3c.github.io/webdriver/results/2017-11/html/all.html 1375346 1 327752 cgarcia 2017-11-28 08:22:36 -0800 Created attachment 327752 Patch 1375472 2 327752 bburg 2017-11-28 12:45:48 -0800 Comment on attachment 327752 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=327752&action=review r=me > Tools/Scripts/run-webdriver-tests:51 > + help='Write results to JSON file at the given path') Please set the metavariable. > Tools/Scripts/run-webdriver-tests:72 > + runner.dump_results_to_json_file(options.json_output) Please validate that the provided path exists, or create it if needed. 1375736 3 cgarcia 2017-11-28 23:26:03 -0800 Committed r225262: <https://trac.webkit.org/changeset/225262> 1375742 4 webkit-bug-importer 2017-11-28 23:27:30 -0800 <rdar://problem/35742433> 327752 2017-11-28 08:22:36 -0800 2017-11-28 12:45:48 -0800 Patch wd-tests-results-json.diff text/plain 7536 cgarcia ZGlmZiAtLWdpdCBhL1Rvb2xzL0NoYW5nZUxvZyBiL1Rvb2xzL0NoYW5nZUxvZwppbmRleCAxZGQ4 MDIwN2I4OS4uNmVkM2I1ODM1ZDcgMTAwNjQ0Ci0tLSBhL1Rvb2xzL0NoYW5nZUxvZworKysgYi9U b29scy9DaGFuZ2VMb2cKQEAgLTEsMyArMSwyNyBAQAorMjAxNy0xMS0yOCAgQ2FybG9zIEdhcmNp YSBDYW1wb3MgIDxjZ2FyY2lhQGlnYWxpYS5jb20+CisKKyAgICAgICAgV2ViRHJpdmVyOiBhZGQg YW4gb3B0aW9uIHRvIGR1bXAgdGVzdCByZXN1bHRzIHRvIGEganNvbiBmaWxlCisgICAgICAgIGh0 dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0xODAwODIKKworICAgICAgICBS ZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICBBZGQgLS1qc29uLW91dHB1dCBj b21tYW5kIGxpbmUgb3B0aW9uIHRvIHJ1bi13ZWJkcml2ZXItdGVzdHMgdG8gZHVtcCB0ZXN0IHJl c3VsdHMgdG8gYSBqc29uIGZpbGUgaW4gYSBmb3JtYXQKKyAgICAgICAgY29tcGF0aWJsZSB3aXRo IHRoZSBXM0MgcmVwb3J0LiBXZWJEcml2ZXJUZXN0UmVzdWx0IG5vdyByZXByZXNlbnRzIGEgdGVz dCBmaWxlIGFuZCBjb250YWlucyBhIGxpc3Qgb2YKKyAgICAgICAgc3VidGVzdHMsIGluc3RlYWQg b2YgaGF2aW5nIG9uZSBXZWJEcml2ZXJUZXN0UmVzdWx0IHBlciBzdWJ0ZXN0LiBUaGlzIHdheSB3 ZSBjYW4gc3RvcmUgYWxzbyB0aGUgaGFybmVzcyByZXN1bHQKKyAgICAgICAgYW5kIGR1bXAgdGhl IHJlc3VsdHMgdG8gZGlmZmVyZW50IGZvcm1hdHMuCisKKyAgICAgICAgKiBTY3JpcHRzL3J1bi13 ZWJkcml2ZXItdGVzdHM6CisgICAgICAgICogU2NyaXB0cy93ZWJraXRweS93ZWJkcml2ZXJfdGVz dHMvd2ViZHJpdmVyX3Rlc3RfcmVzdWx0LnB5OgorICAgICAgICAoV2ViRHJpdmVyVGVzdFJlc3Vs dC5fX2luaXRfXyk6CisgICAgICAgIChXZWJEcml2ZXJUZXN0UmVzdWx0KToKKyAgICAgICAgKFdl YkRyaXZlclRlc3RSZXN1bHQuYWRkX3N1YnRlc3RfcmVzdWx0cyk6CisgICAgICAgICogU2NyaXB0 cy93ZWJraXRweS93ZWJkcml2ZXJfdGVzdHMvd2ViZHJpdmVyX3Rlc3RfcnVubmVyLnB5OgorICAg ICAgICAoV2ViRHJpdmVyVGVzdFJ1bm5lci5wcmludF9yZXN1bHRzKToKKyAgICAgICAgKFdlYkRy aXZlclRlc3RSdW5uZXIpOgorICAgICAgICAoV2ViRHJpdmVyVGVzdFJ1bm5lci5kdW1wX3Jlc3Vs dHNfdG9fanNvbl9maWxlKToKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L3dlYmRyaXZlcl90 ZXN0cy93ZWJkcml2ZXJfdGVzdF9ydW5uZXJfdzNjLnB5OgorICAgICAgICAoV2ViRHJpdmVyVGVz dFJ1bm5lclczQy5ydW4pOgorCiAyMDE3LTExLTI4ICBGdWppaSBIaXJvbm9yaSAgPEhpcm9ub3Jp LkZ1amlpQHNvbnkuY29tPgogCiAgICAgICAgIHdlYmtpdHB5OiBQbGF0Zm9ybUluZm8gcmFpc2Vz IEFzc2VydGlvbkVycm9yICJhc3NlcnQgc2VsZi5vc192ZXJzaW9uIGlzIG5vdCBOb25lIiBpbiBD eWd3aW4gc2luY2UgQnVnIDE3OTYyMQpkaWZmIC0tZ2l0IGEvVG9vbHMvU2NyaXB0cy9ydW4td2Vi ZHJpdmVyLXRlc3RzIGIvVG9vbHMvU2NyaXB0cy9ydW4td2ViZHJpdmVyLXRlc3RzCmluZGV4IDJl NjAxY2Y2MzNmLi45YWE0MTI2MTBmNCAxMDA3NTUKLS0tIGEvVG9vbHMvU2NyaXB0cy9ydW4td2Vi ZHJpdmVyLXRlc3RzCisrKyBiL1Rvb2xzL1NjcmlwdHMvcnVuLXdlYmRyaXZlci10ZXN0cwpAQCAt NDcsNiArNDcsOCBAQCBvcHRpb25fcGFyc2VyLmFkZF9vcHRpb24oJy0tZGVidWcnLCBhY3Rpb249 J3N0b3JlX2NvbnN0JywgY29uc3Q9J0RlYnVnJywgZGVzdD0iYwogICAgICAgICAgICAgICAgICAg ICAgICAgIGhlbHA9J1NldCB0aGUgY29uZmlndXJhdGlvbiB0byBEZWJ1ZycpCiBvcHRpb25fcGFy c2VyLmFkZF9vcHRpb24oJy0tdGltZW91dCcsIGFjdGlvbj0nc3RvcmUnLCB0eXBlPSdpbnQnLCBk ZXN0PSd0aW1lb3V0JywgZGVmYXVsdD0xMCwKICAgICAgICAgICAgICAgICAgICAgICAgICBoZWxw PSdUaW1lIGluIHNlY29uZHMgdW50aWwgYSB0ZXN0IHRpbWVzIG91dCAodXNlIDAgdG8gZGlzYWJs ZSknKQorb3B0aW9uX3BhcnNlci5hZGRfb3B0aW9uKCctLWpzb24tb3V0cHV0JywgYWN0aW9uPSdz dG9yZScsCisgICAgICAgICAgICAgICAgICAgICAgICAgaGVscD0nV3JpdGUgcmVzdWx0cyB0byBK U09OIGZpbGUgYXQgdGhlIGdpdmVuIHBhdGgnKQogb3B0aW9uX3BhcnNlci5hZGRfb3B0aW9uKCct LWRpc3BsYXktc2VydmVyJywgY2hvaWNlcz1bJ3h2ZmInLCAneG9yZycsICd3ZXN0b24nLCAnd2F5 bGFuZCddLCBkZWZhdWx0PSd4dmZiJywKICAgICAgICAgICAgICAgICAgICAgICAgICBoZWxwPSci eHZmYiI6IFVzZSBhIHZpcnR1YWxpemVkIFgxMSBzZXJ2ZXIuICJ4b3JnIjogVXNlIHRoZSBjdXJy ZW50IFgxMSBzZXNzaW9uLiAnCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAnIndlc3Rv biI6IFVzZSBhIHZpcnR1YWxpemVkIFdlc3RvbiBzZXJ2ZXIuICJ3YXlsYW5kIjogVXNlIHRoZSBj dXJyZW50IHdheWxhbmQgc2Vzc2lvbi4nKQpAQCAtNjYsNSArNjgsOCBAQCBydW5uZXIgPSBXZWJE cml2ZXJUZXN0UnVubmVyKHBvcnQpCiByZXR2YWwgPSBydW5uZXIucnVuKGFyZ3MpCiBydW5uZXIu cHJpbnRfcmVzdWx0cygpCiAKK2lmIG9wdGlvbnMuanNvbl9vdXRwdXQgaXMgbm90IE5vbmU6Cisg ICAgcnVubmVyLmR1bXBfcmVzdWx0c190b19qc29uX2ZpbGUob3B0aW9ucy5qc29uX291dHB1dCkK Kwogc3lzLmV4aXQocmV0dmFsKQogCmRpZmYgLS1naXQgYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5 L3dlYmRyaXZlcl90ZXN0cy93ZWJkcml2ZXJfdGVzdF9yZXN1bHQucHkgYi9Ub29scy9TY3JpcHRz L3dlYmtpdHB5L3dlYmRyaXZlcl90ZXN0cy93ZWJkcml2ZXJfdGVzdF9yZXN1bHQucHkKaW5kZXgg NWU1MDdjZmJmZWIuLjY2YTE3ZWNmY2EzIDEwMDY0NAotLS0gYS9Ub29scy9TY3JpcHRzL3dlYmtp dHB5L3dlYmRyaXZlcl90ZXN0cy93ZWJkcml2ZXJfdGVzdF9yZXN1bHQucHkKKysrIGIvVG9vbHMv U2NyaXB0cy93ZWJraXRweS93ZWJkcml2ZXJfdGVzdHMvd2ViZHJpdmVyX3Rlc3RfcmVzdWx0LnB5 CkBAIC0yNSwxMSArMjUsMTQgQEAgaW1wb3J0IG9zCiAKIGNsYXNzIFdlYkRyaXZlclRlc3RSZXN1 bHQob2JqZWN0KToKIAotICAgIGRlZiBfX2luaXRfXyhzZWxmLCB0ZXN0X3ByZWZpeCwgdGVzdCwg c3RhdHVzLCBtZXNzYWdlLCBiYWNrdHJhY2U9Tm9uZSk6Ci0gICAgICAgIHNlbGYudGVzdCA9IG9z LnBhdGguam9pbih0ZXN0X3ByZWZpeCwgdGVzdCkKKyAgICBkZWYgX19pbml0X18oc2VsZiwgdGVz dCwgc3RhdHVzLCBtZXNzYWdlPU5vbmUpOgorICAgICAgICBzZWxmLnRlc3QgPSB0ZXN0CiAgICAg ICAgIHNlbGYuc3RhdHVzID0gc3RhdHVzCiAgICAgICAgIHNlbGYubWVzc2FnZSA9IG1lc3NhZ2UK LSAgICAgICAgc2VsZi5iYWNrdHJhY2UgPSBiYWNrdHJhY2UKKyAgICAgICAgc2VsZi5zdWJ0ZXN0 X3Jlc3VsdHMgPSBbXQorCisgICAgZGVmIGFkZF9zdWJ0ZXN0X3Jlc3VsdHMoc2VsZiwgc3VidGVz dCwgc3RhdHVzLCBtZXNzYWdlLCBiYWNrdHJhY2UpOgorICAgICAgICBzZWxmLnN1YnRlc3RfcmVz dWx0cy5hcHBlbmQoKHN1YnRlc3QsIHN0YXR1cywgbWVzc2FnZSwgYmFja3RyYWNlKSkKIAogICAg IGRlZiBfX3JlcHJfXyhzZWxmKToKICAgICAgICAgcmV0dXJuICI8JXMuJXMgJXMgJXM+IiAlIChz ZWxmLl9fbW9kdWxlX18sIHNlbGYuX19jbGFzc19fLl9fbmFtZV9fLCBzZWxmLnRlc3QsIHNlbGYu c3RhdHVzKQpkaWZmIC0tZ2l0IGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS93ZWJkcml2ZXJfdGVz dHMvd2ViZHJpdmVyX3Rlc3RfcnVubmVyLnB5IGIvVG9vbHMvU2NyaXB0cy93ZWJraXRweS93ZWJk cml2ZXJfdGVzdHMvd2ViZHJpdmVyX3Rlc3RfcnVubmVyLnB5CmluZGV4IDY0NjYxZWE5MmJhLi45 NDBiNTcwYWRlZCAxMDA2NDQKLS0tIGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS93ZWJkcml2ZXJf dGVzdHMvd2ViZHJpdmVyX3Rlc3RfcnVubmVyLnB5CisrKyBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0 cHkvd2ViZHJpdmVyX3Rlc3RzL3dlYmRyaXZlcl90ZXN0X3J1bm5lci5weQpAQCAtMjAsNyArMjAs OSBAQAogIyAoSU5DTFVESU5HIE5FR0xJR0VOQ0UgT1IgT1RIRVJXSVNFKSBBUklTSU5HIElOIEFO WSBXQVkgT1VUIE9GIFRIRSBVU0UgT0YgVEhJUwogIyBTT0ZUV0FSRSwgRVZFTiBJRiBBRFZJU0VE IE9GIFRIRSBQT1NTSUJJTElUWSBPRiBTVUNIIERBTUFHRS4KIAoraW1wb3J0IGpzb24KIGltcG9y dCBsb2dnaW5nCitpbXBvcnQgb3MKIAogZnJvbSB3ZWJraXRweS53ZWJkcml2ZXJfdGVzdHMud2Vi ZHJpdmVyX3Rlc3RfcnVubmVyX3czYyBpbXBvcnQgV2ViRHJpdmVyVGVzdFJ1bm5lclczQwogCkBA IC01NCwxMSArNTYsMTYgQEAgY2xhc3MgV2ViRHJpdmVyVGVzdFJ1bm5lcihvYmplY3QpOgogICAg ICAgICBwYXNzZWRfY291bnQgPSAwCiAgICAgICAgIGZhaWx1cmVzX2NvdW50ID0gMAogICAgICAg ICBmb3IgcmVzdWx0IGluIHNlbGYuX3J1bm5lci5yZXN1bHRzKCk6Ci0gICAgICAgICAgICBpZiBy ZXN1bHQuc3RhdHVzID09ICdQQVNTJzoKLSAgICAgICAgICAgICAgICBwYXNzZWRfY291bnQgKz0g MQotICAgICAgICAgICAgZWxpZiByZXN1bHQuc3RhdHVzID09ICdGQUlMJzoKLSAgICAgICAgICAg ICAgICByZXN1bHRzLnNldGRlZmF1bHQocmVzdWx0LnN0YXR1cywgW10pLmFwcGVuZChyZXN1bHQu dGVzdCkKLSAgICAgICAgICAgICAgICBmYWlsdXJlc19jb3VudCArPSAxCisgICAgICAgICAgICBp ZiByZXN1bHQuc3RhdHVzID09ICdPSyc6CisgICAgICAgICAgICAgICAgZm9yIHN1YnRlc3QsIHN0 YXR1cywgXywgXyBpbiByZXN1bHQuc3VidGVzdF9yZXN1bHRzOgorICAgICAgICAgICAgICAgICAg ICBpZiBzdGF0dXMgPT0gJ1BBU1MnOgorICAgICAgICAgICAgICAgICAgICAgICAgcGFzc2VkX2Nv dW50ICs9IDEKKyAgICAgICAgICAgICAgICAgICAgZWxpZiBzdGF0dXMgPT0gJ0ZBSUwnOgorICAg ICAgICAgICAgICAgICAgICAgICAgcmVzdWx0cy5zZXRkZWZhdWx0KHN0YXR1cywgW10pLmFwcGVu ZChvcy5wYXRoLmpvaW4ob3MucGF0aC5kaXJuYW1lKHJlc3VsdC50ZXN0KSwgc3VidGVzdCkpCisg ICAgICAgICAgICAgICAgICAgICAgICBmYWlsdXJlc19jb3VudCArPSAxCisgICAgICAgICAgICBl bHNlOgorICAgICAgICAgICAgICAgICMgRklYTUU6IGhhbmRsZSBvdGhlciByZXN1bHRzLgorICAg ICAgICAgICAgICAgIHBhc3MKIAogICAgICAgICBfbG9nLmluZm8oJycpCiAKQEAgLTczLDMgKzgw LDIzIEBAIGNsYXNzIFdlYkRyaXZlclRlc3RSdW5uZXIob2JqZWN0KToKICAgICAgICAgICAgIF9s b2cuaW5mbygnVW5leHBlY3RlZCBmYWlsdXJlcyAoJWQpJyAlIGxlbihmYWlsZWQpKQogICAgICAg ICAgICAgZm9yIHRlc3QgaW4gZmFpbGVkOgogICAgICAgICAgICAgICAgIF9sb2cuaW5mbygnICAl cycgJSB0ZXN0KQorCisgICAgZGVmIGR1bXBfcmVzdWx0c190b19qc29uX2ZpbGUoc2VsZiwgb3V0 cHV0X3BhdGgpOgorICAgICAgICBqc29uX3Jlc3VsdHMgPSB7fQorICAgICAgICBqc29uX3Jlc3Vs dHNbJ3Jlc3VsdHMnXSA9IFtdCisgICAgICAgIGZvciByZXN1bHQgaW4gc2VsZi5fcnVubmVyLnJl c3VsdHMoKToKKyAgICAgICAgICAgIHJlc3VsdHMgPSB7fQorICAgICAgICAgICAgcmVzdWx0c1sn dGVzdCddID0gcmVzdWx0LnRlc3QKKyAgICAgICAgICAgIHJlc3VsdHNbJ3N0YXR1cyddID0gcmVz dWx0LnN0YXR1cworICAgICAgICAgICAgcmVzdWx0c1snbWVzc2FnZSddID0gcmVzdWx0Lm1lc3Nh Z2UKKyAgICAgICAgICAgIHJlc3VsdHNbJ3N1YnRlc3RzJ10gPSBbXQorICAgICAgICAgICAgZm9y IG5hbWUsIHN0YXR1cywgbWVzc2FnZSwgXyBpbiByZXN1bHQuc3VidGVzdF9yZXN1bHRzOgorICAg ICAgICAgICAgICAgIHN1YnRlc3QgPSB7fQorICAgICAgICAgICAgICAgIHN1YnRlc3RbJ25hbWUn XSA9IG5hbWUKKyAgICAgICAgICAgICAgICBzdWJ0ZXN0WydzdGF0dXMnXSA9IHN0YXR1cworICAg ICAgICAgICAgICAgIHN1YnRlc3RbJ21lc3NhZ2UnXSA9IG1lc3NhZ2UKKyAgICAgICAgICAgICAg ICByZXN1bHRzWydzdWJ0ZXN0cyddLmFwcGVuZChzdWJ0ZXN0KQorICAgICAgICAgICAganNvbl9y ZXN1bHRzWydyZXN1bHRzJ10uYXBwZW5kKHJlc3VsdHMpCisKKyAgICAgICAgd2l0aCBvcGVuKG91 dHB1dF9wYXRoLCAnd2InKSBhcyBmcDoKKyAgICAgICAgICAgIGpzb24uZHVtcChqc29uX3Jlc3Vs dHMsIGZwKQpkaWZmIC0tZ2l0IGEvVG9vbHMvU2NyaXB0cy93ZWJraXRweS93ZWJkcml2ZXJfdGVz dHMvd2ViZHJpdmVyX3Rlc3RfcnVubmVyX3czYy5weSBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkv d2ViZHJpdmVyX3Rlc3RzL3dlYmRyaXZlcl90ZXN0X3J1bm5lcl93M2MucHkKaW5kZXggMGJkZDkz ZjBkNDYuLmVjNWQyZTQ4OTMxIDEwMDY0NAotLS0gYS9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L3dl YmRyaXZlcl90ZXN0cy93ZWJkcml2ZXJfdGVzdF9ydW5uZXJfdzNjLnB5CisrKyBiL1Rvb2xzL1Nj cmlwdHMvd2Via2l0cHkvd2ViZHJpdmVyX3Rlc3RzL3dlYmRyaXZlcl90ZXN0X3J1bm5lcl93M2Mu cHkKQEAgLTk2LDEwICs5NiwxNCBAQCBjbGFzcyBXZWJEcml2ZXJUZXN0UnVubmVyVzNDKG9iamVj dCk6CiAgICAgICAgICAgICBmb3IgdGVzdCBpbiB0ZXN0czoKICAgICAgICAgICAgICAgICB0ZXN0 X25hbWUgPSBvcy5wYXRoLnJlbHBhdGgodGVzdCwgc2VsZi5fdGVzdHNfZGlyKCkpCiAgICAgICAg ICAgICAgICAgaGFybmVzc19yZXN1bHQsIHRlc3RfcmVzdWx0cyA9IGV4ZWN1dG9yLnJ1bih0ZXN0 KQotICAgICAgICAgICAgICAgIGlmIGhhcm5lc3NfcmVzdWx0WzBdICE9ICdPSyc6Ci0gICAgICAg ICAgICAgICAgICAgIF9sb2cuZXJyb3IoIkZhaWxlZCB0byBydW4gdGVzdCAlczogJXMiICUgKHRl c3RfbmFtZSwgaGFybmVzc19yZXN1bHRbMV0pKQorICAgICAgICAgICAgICAgIHJlc3VsdCA9IFdl YkRyaXZlclRlc3RSZXN1bHQodGVzdF9uYW1lLCAqaGFybmVzc19yZXN1bHQpCisgICAgICAgICAg ICAgICAgaWYgaGFybmVzc19yZXN1bHRbMF0gPT0gJ09LJzoKKyAgICAgICAgICAgICAgICAgICAg Zm9yIHRlc3RfcmVzdWx0IGluIHRlc3RfcmVzdWx0czoKKyAgICAgICAgICAgICAgICAgICAgICAg IHJlc3VsdC5hZGRfc3VidGVzdF9yZXN1bHRzKCp0ZXN0X3Jlc3VsdCkKICAgICAgICAgICAgICAg ICBlbHNlOgotICAgICAgICAgICAgICAgICAgICBzZWxmLl9hZGRfcmVzdWx0cyhvcy5wYXRoLmRp cm5hbWUodGVzdF9uYW1lKSwgdGVzdF9yZXN1bHRzKQorICAgICAgICAgICAgICAgICAgICAjIEZJ WE1FOiBoYW5kbGUgb3RoZXIgcmVzdWx0cy4KKyAgICAgICAgICAgICAgICAgICAgcGFzcworICAg ICAgICAgICAgICAgIHNlbGYuX3Jlc3VsdHMuYXBwZW5kKHJlc3VsdCkKICAgICAgICAgZmluYWxs eToKICAgICAgICAgICAgIGV4ZWN1dG9yLnRlYXJkb3duKCkKICAgICAgICAgICAgIHNlbGYuX3Nl cnZlci5zdG9wKCkK