59256 2011-04-22 16:38:16 -0700 only include failures in full_results.json 2011-04-27 15:39:04 -0700 1 1 1 Unclassified WebKit New Bugs 528+ (Nightly build) Other OS X 10.5 RESOLVED FIXED P2 Normal --- 1 ojan ojan abarth dpranke eric webkit.review.bot oldest_to_newest 391224 0 ojan 2011-04-22 16:38:16 -0700 only include failures in full_results.json 391226 1 90801 ojan 2011-04-22 16:39:05 -0700 Created attachment 90801 Patch 391227 2 ojan 2011-04-22 16:39:35 -0700 This patch depends on bug 59239. 391331 3 90801 tony 2011-04-22 19:17:24 -0700 Comment on attachment 90801 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=90801&action=review > Tools/Scripts/webkitpy/layout_tests/layout_package/manager.py:128 > + test_dict = {} It would be nice to just make these structs at some point. > Tools/Scripts/webkitpy/layout_tests/layout_package/manager.py:130 > + if False: > + test_dict['has_stderr'] = True Debugging code? 392941 4 90801 dpranke 2011-04-26 16:11:05 -0700 Comment on attachment 90801 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=90801&action=review > Tools/Scripts/webkitpy/layout_tests/layout_package/manager.py:74 > + """Summarize failing results as a dict. Unless I'm misunderstanding this, it looks like this change removes unexpected passes. That's bad - it'll break the bot output reporting unexpected passes and the log output for the same. It seems to me that if you want to exclude the results from the JSON file you should do it where you build the JSON file, not by changing this function. 393680 5 ojan 2011-04-27 11:20:56 -0700 (In reply to comment #4) > (From update of attachment 90801 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=90801&action=review > > > Tools/Scripts/webkitpy/layout_tests/layout_package/manager.py:74 > > + """Summarize failing results as a dict. > > Unless I'm misunderstanding this, it looks like this change removes unexpected passes. That's bad - it'll break the bot output reporting unexpected passes and the log output for the same. Good catch. Fixed. > It seems to me that if you want to exclude the results from the JSON file you should do it where you build the JSON file, not by changing this function. I don't see why. The only places the summarized results are used is to write out the json and print out unexpected failures. I don't really see that changing in the future, do you? 393831 6 ojan 2011-04-27 14:27:23 -0700 Committed r85095: <http://trac.webkit.org/changeset/85095> 393857 7 dpranke 2011-04-27 14:57:19 -0700 (In reply to comment #5) > (In reply to comment #4) > > (From update of attachment 90801 [details] [details]) > > View in context: https://bugs.webkit.org/attachment.cgi?id=90801&action=review > > > > > Tools/Scripts/webkitpy/layout_tests/layout_package/manager.py:74 > > > + """Summarize failing results as a dict. > > > > Unless I'm misunderstanding this, it looks like this change removes unexpected passes. That's bad - it'll break the bot output reporting unexpected passes and the log output for the same. > > Good catch. Fixed. > > > It seems to me that if you want to exclude the results from the JSON file you should do it where you build the JSON file, not by changing this function. > > I don't see why. The only places the summarized results are used is to write out the json and print out unexpected failures. I don't really see that changing in the future, do you? At some point I'd like to change the Manager.run() routine to return something like the summarized results so that we can write better integration tests, but it's a pretty low-priority thing. 393858 8 dpranke 2011-04-27 14:58:35 -0700 Nit, since the summarized results kept the unexpected passes, the original docstring was more accurate. I think you forgot to change it back. 393894 9 webkit.review.bot 2011-04-27 15:39:04 -0700 http://trac.webkit.org/changeset/85095 might have broken Windows 7 Release (Tests) 90801 2011-04-22 16:39:05 -0700 2011-04-26 16:11:05 -0700 Patch bug-59256-20110422163904.patch text/plain 4014 ojan U3VidmVyc2lvbiBSZXZpc2lvbjogODQ2NDAKZGlmZiAtLWdpdCBhL1Rvb2xzL0NoYW5nZUxvZyBi L1Rvb2xzL0NoYW5nZUxvZwppbmRleCBhYTgxNTJmYTg2NGFiZDA0OThhNjRkMzAyM2ZmMGFhMzcz NWVhMzEzLi41MzRkMWQ2OGQyNjY1ZGVhZmJmNjVhYzRjN2IxYTI5NDMxZmU5NzlhIDEwMDY0NAot LS0gYS9Ub29scy9DaGFuZ2VMb2cKKysrIGIvVG9vbHMvQ2hhbmdlTG9nCkBAIC0yLDYgKzIsMTUg QEAKIAogICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KIAorICAgICAgICBvbmx5 IGluY2x1ZGUgZmFpbHVyZXMgaW4gZnVsbF9yZXN1bHRzLmpzb24KKyAgICAgICAgaHR0cHM6Ly9i dWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTU5MjU2CisKKyAgICAgICAgKiBTY3JpcHRz L3dlYmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9tYW5hZ2VyLnB5OgorCisyMDEx LTA0LTIyICBPamFuIFZhZmFpICA8b2phbkBjaHJvbWl1bS5vcmc+CisKKyAgICAgICAgUmV2aWV3 ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKICAgICAgICAgbW92ZSB0aW1lcyBpbnRvIHRoZWlyIG93 biBqc29uIGZpbGUKICAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dp P2lkPTU5MjM5CiAKZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rl c3RzL2xheW91dF9wYWNrYWdlL21hbmFnZXIucHkgYi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2xh eW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9tYW5hZ2VyLnB5CmluZGV4IGY2ZjgxMjU4YmRiYjU3 NzY1YzViZDUyZTgwODEwYTJiNGY3YWIyODIuLjNkNTRiYTIwODVlMWNiZjExNzBkNjM2Mzc1ZWM2 N2Y4NDYyMmU2Y2IgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rl c3RzL2xheW91dF9wYWNrYWdlL21hbmFnZXIucHkKKysrIGIvVG9vbHMvU2NyaXB0cy93ZWJraXRw eS9sYXlvdXRfdGVzdHMvbGF5b3V0X3BhY2thZ2UvbWFuYWdlci5weQpAQCAtNzEsNyArNzEsNyBA QCBUZXN0RXhwZWN0YXRpb25zRmlsZSA9IHRlc3RfZXhwZWN0YXRpb25zLlRlc3RFeHBlY3RhdGlv bnNGaWxlCiAKIAogZGVmIHN1bW1hcml6ZV9yZXN1bHRzKHBvcnRfb2JqLCBleHBlY3RhdGlvbnMs IHJlc3VsdF9zdW1tYXJ5LCByZXRyeV9zdW1tYXJ5LCB0ZXN0X3RpbWluZ3MsIG9ubHlfdW5leHBl Y3RlZCk6Ci0gICAgIiIiU3VtbWFyaXplIGFueSB1bmV4cGVjdGVkIHJlc3VsdHMgYXMgYSBkaWN0 LgorICAgICIiIlN1bW1hcml6ZSBmYWlsaW5nIHJlc3VsdHMgYXMgYSBkaWN0LgogCiAgICAgRklY TUU6IHNwbGl0IHRoaXMgZGF0YSBzdHJ1Y3R1cmUgaW50byBhIHNlcGFyYXRlIGNsYXNzPwogCkBA IC0xMjUsOSArMTI1LDE0IEBAIGRlZiBzdW1tYXJpemVfcmVzdWx0cyhwb3J0X29iaiwgZXhwZWN0 YXRpb25zLCByZXN1bHRfc3VtbWFyeSwgcmV0cnlfc3VtbWFyeSwgdGVzCiAgICAgICAgIHJlc3Vs dF90eXBlID0gcmVzdWx0LnR5cGUKICAgICAgICAgYWN0dWFsID0gW2tleXdvcmRzW3Jlc3VsdF90 eXBlXV0KIAotICAgICAgICAjIEZJWE1FOiBvbmx5IGluY2x1ZGUgcGFzc2luZyB0ZXN0cyB0aGF0 IGhhdmUgc3RkZXJyIG91dHB1dC4KKyAgICAgICAgdGVzdF9kaWN0ID0ge30KKyAgICAgICAgaWYg RmFsc2U6CisgICAgICAgICAgICB0ZXN0X2RpY3RbJ2hhc19zdGRlcnInXSA9IFRydWUKKwogICAg ICAgICBpZiByZXN1bHRfdHlwZSA9PSB0ZXN0X2V4cGVjdGF0aW9ucy5QQVNTOgogICAgICAgICAg ICAgbnVtX3Bhc3NlcyArPSAxCisgICAgICAgICAgICAjIEZJWE1FOiBpbmNsdWRlIHBhc3Npbmcg dGVzdHMgdGhhdCBoYXZlIHN0ZGVyciBvdXRwdXQuCisgICAgICAgICAgICBjb250aW51ZQogICAg ICAgICBlbGlmIHJlc3VsdF90eXBlID09IHRlc3RfZXhwZWN0YXRpb25zLkNSQVNIOgogICAgICAg ICAgICAgbnVtX3JlZ3Jlc3Npb25zICs9IDEKICAgICAgICAgZWxpZiBmaWxlbmFtZSBpbiByZXN1 bHRfc3VtbWFyeS51bmV4cGVjdGVkX3Jlc3VsdHM6CkBAIC0xNDIsMzMgKzE0NywzNCBAQCBkZWYg c3VtbWFyaXplX3Jlc3VsdHMocG9ydF9vYmosIGV4cGVjdGF0aW9ucywgcmVzdWx0X3N1bW1hcnks IHJldHJ5X3N1bW1hcnksIHRlcwogICAgICAgICAgICAgICAgIGVsc2U6CiAgICAgICAgICAgICAg ICAgICAgIG51bV9yZWdyZXNzaW9ucyArPSAxCiAKLSAgICAgICAgdGVzdHNbdGVzdF0gPSB7fQot ICAgICAgICB0ZXN0c1t0ZXN0XVsnZXhwZWN0ZWQnXSA9IGV4cGVjdGVkCi0gICAgICAgIHRlc3Rz W3Rlc3RdWydhY3R1YWwnXSA9ICIgIi5qb2luKGFjdHVhbCkKKyAgICAgICAgdGVzdF9kaWN0Wydl eHBlY3RlZCddID0gZXhwZWN0ZWQKKyAgICAgICAgdGVzdF9kaWN0WydhY3R1YWwnXSA9ICIgIi5q b2luKGFjdHVhbCkKICAgICAgICAgIyBGSVhNRTogU2V0IHRoaXMgY29ycmVjdGx5IG9uY2UgaHR0 cHM6Ly93ZWJraXQub3JnL2IvMzc3MzkgaXMgZml4ZWQKICAgICAgICAgIyBhbmQgb25seSBzZXQg aXQgaWYgdGhlcmUgYWN0dWFsbHkgaXMgc3RkZXJyIGRhdGEuCi0gICAgICAgIGlmIEZhbHNlOgot ICAgICAgICAgICAgdGVzdHNbdGVzdF1bJ2hhc19zdGRlcnInXSA9IFRydWUKIAogICAgICAgICBm YWlsdXJlX3R5cGVzID0gW3R5cGUoZikgZm9yIGYgaW4gcmVzdWx0LmZhaWx1cmVzXQorICAgICAg ICAjIEZJWE1FOiBnZXQgcmlkIG9mIGFsbCB0aGlzIGlzXyogdmFsdWVzIG9uY2UgdGhlcmUgaXMg YSAxOjEgbWFwIGJldHdlZW4KKyAgICAgICAgIyBUZXN0RmFpbHVyZSB0eXBlIGFuZCB0ZXN0X2V4 cGVjdGF0aW9ucy5FWFBFQ1RBVElPTi4KICAgICAgICAgaWYgdGVzdF9mYWlsdXJlcy5GYWlsdXJl TWlzc2luZ0F1ZGlvIGluIGZhaWx1cmVfdHlwZXM6Ci0gICAgICAgICAgICB0ZXN0c1t0ZXN0XVsn aXNfbWlzc2luZ19hdWRpbyddID0gVHJ1ZQorICAgICAgICAgICAgdGVzdF9kaWN0Wydpc19taXNz aW5nX2F1ZGlvJ10gPSBUcnVlCiAKICAgICAgICAgaWYgdGVzdF9mYWlsdXJlcy5GYWlsdXJlUmVm dGVzdE1pc21hdGNoIGluIGZhaWx1cmVfdHlwZXM6Ci0gICAgICAgICAgICB0ZXN0c1t0ZXN0XVsn aXNfcmVmdGVzdCddID0gVHJ1ZQorICAgICAgICAgICAgdGVzdF9kaWN0Wydpc19yZWZ0ZXN0J10g PSBUcnVlCiAKICAgICAgICAgZm9yIGYgaW4gcmVzdWx0LmZhaWx1cmVzOgogICAgICAgICAgICAg aWYgJ2lzX3JlZnRlc3QnIGluIHJlc3VsdC5mYWlsdXJlczoKLSAgICAgICAgICAgICAgICB0ZXN0 c1t0ZXN0XVsnaXNfcmVmdGVzdCddID0gVHJ1ZQorICAgICAgICAgICAgICAgIHRlc3RfZGljdFsn aXNfcmVmdGVzdCddID0gVHJ1ZQogCiAgICAgICAgIGlmIHRlc3RfZmFpbHVyZXMuRmFpbHVyZVJl ZnRlc3RNaXNtYXRjaERpZE5vdE9jY3VyIGluIGZhaWx1cmVfdHlwZXM6Ci0gICAgICAgICAgICB0 ZXN0c1t0ZXN0XVsnaXNfbWlzbWF0Y2hfcmVmdGVzdCddID0gVHJ1ZQorICAgICAgICAgICAgdGVz dF9kaWN0Wydpc19taXNtYXRjaF9yZWZ0ZXN0J10gPSBUcnVlCiAKICAgICAgICAgaWYgdGVzdF9m YWlsdXJlcy5GYWlsdXJlTWlzc2luZ1Jlc3VsdCBpbiBmYWlsdXJlX3R5cGVzOgotICAgICAgICAg ICAgdGVzdHNbdGVzdF1bJ2lzX21pc3NpbmdfdGV4dCddID0gVHJ1ZQorICAgICAgICAgICAgdGVz dF9kaWN0Wydpc19taXNzaW5nX3RleHQnXSA9IFRydWUKIAogICAgICAgICBpZiB0ZXN0X2ZhaWx1 cmVzLkZhaWx1cmVNaXNzaW5nSW1hZ2UgaW4gZmFpbHVyZV90eXBlcyBvciB0ZXN0X2ZhaWx1cmVz LkZhaWx1cmVNaXNzaW5nSW1hZ2VIYXNoIGluIGZhaWx1cmVfdHlwZXM6Ci0gICAgICAgICAgICB0 ZXN0c1t0ZXN0XVsnaXNfbWlzc2luZ19pbWFnZSddID0gVHJ1ZQorICAgICAgICAgICAgdGVzdF9k aWN0Wydpc19taXNzaW5nX2ltYWdlJ10gPSBUcnVlCisKKyAgICAgICAgdGVzdHNbdGVzdF0gPSB0 ZXN0X2RpY3QKIAogICAgIHJlc3VsdHNbJ3Rlc3RzJ10gPSB0ZXN0cwogICAgIHJlc3VsdHNbJ251 bV9wYXNzZXMnXSA9IG51bV9wYXNzZXMK