49702 2010-11-17 17:33:17 -0800 [Chromium] REGRESSION: Record actual test results in the JSON result summary for non-layout tests 2010-11-19 15:08:49 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) PC OS X 10.5 RESOLVED FIXED P2 Normal --- 49784 1 kinuko webkit-unassigned eric levin ojan oldest_to_newest 310638 0 kinuko 2010-11-17 17:33:17 -0800 REGRESSION: Record actual test results in the JSON result summary for non-layout tests Since r71952 we started recording test modifiers (FLAKY/FAILS) for failed tests in the failure summaries (fixableCounts), but the patch wrongly changed the way we record the failure details for each test too. We should record 1. actual failure/pass info for each test (this has gotten broken in r71952), and 2. FLAKY/FAILS modifier stats for failure summaries (this was introduced in r71952). 310654 1 74188 kinuko 2010-11-17 18:28:34 -0800 Created attachment 74188 Patch 311219 2 kinuko 2010-11-18 18:06:15 -0800 Committed r72357: <http://trac.webkit.org/changeset/72357> 311220 3 levin 2010-11-18 18:08:19 -0800 There are no tests for this code? 311221 4 kinuko 2010-11-18 18:12:34 -0800 (In reply to comment #3) > There are no tests for this code? Oh, it does. I forgot to update the test expectations in the unittest. Am going to upload a follow-up patch. 311254 5 74347 kinuko 2010-11-18 19:41:57 -0800 Created attachment 74347 Patch 311264 6 eric 2010-11-18 20:22:13 -0800 This appears to have broken the test-webkitpy tests. 311273 7 kinuko 2010-11-18 20:46:24 -0800 (In reply to comment #6) > This appears to have broken the test-webkitpy tests. I'm rolling out this change for now. 311443 8 74347 ojan 2010-11-19 07:15:19 -0800 Comment on attachment 74347 Patch Looks like _get_test_modifier isn't used anymore. Can we delete it? 311656 9 kinuko 2010-11-19 13:24:33 -0800 (In reply to comment #8) > (From update of attachment 74347 [details]) > Looks like _get_test_modifier isn't used anymore. Can we delete it? It's used to record the summary info (fixable counts). _get_result_char is used to record each test result. For layout tests they're same but for non-layout tests for now we're recording different stats. 311657 10 kinuko 2010-11-19 13:27:02 -0800 (In reply to comment #9) > (In reply to comment #8) > > (From update of attachment 74347 [details] [details]) > > Looks like _get_test_modifier isn't used anymore. Can we delete it? > > It's used to record the summary info (fixable counts). _get_result_char is used to record each test result. For layout tests they're same but for non-layout tests for now we're recording different stats. Oops, of course you meant the one in the unittest. Yes it can be deleted. 311709 11 kinuko 2010-11-19 15:08:49 -0800 Committed r72445: <http://trac.webkit.org/changeset/72445> 74188 2010-11-17 18:28:34 -0800 2010-11-18 19:41:54 -0800 Patch bug-49702-20101117182833.patch text/plain 3390 kinuko ZGlmZiAtLWdpdCBhL1dlYktpdFRvb2xzL0NoYW5nZUxvZyBiL1dlYktpdFRvb2xzL0NoYW5nZUxv ZwppbmRleCBkMzE2NzE3ZGFmOWU5NDkzM2RmZDVhNzBiNDllNzlmZTI5ZGUyY2Y5Li41NTBmYzk1 OTM5NDQ0N2E3ZWNlYTAyMWEyMzczOTdmOTVmOGQzODYwIDEwMDY0NAotLS0gYS9XZWJLaXRUb29s cy9DaGFuZ2VMb2cKKysrIGIvV2ViS2l0VG9vbHMvQ2hhbmdlTG9nCkBAIC0xLDMgKzEsMTMgQEAK KzIwMTAtMTEtMTcgIEtpbnVrbyBZYXN1ZGEgIDxraW51a29AY2hyb21pdW0ub3JnPgorCisgICAg ICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIFtDaHJvbWl1bV0gUkVH UkVTU0lPTjogUmVjb3JkIGFjdHVhbCB0ZXN0IHJlc3VsdHMgaW4gdGhlIEpTT04gcmVzdWx0IHN1 bW1hcnkgZm9yIG5vbi1sYXlvdXQgdGVzdHMKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5v cmcvc2hvd19idWcuY2dpP2lkPTQ5NzAyCisKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2xh eW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9qc29uX2xheW91dF9yZXN1bHRzX2dlbmVyYXRvci5w eToKKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2Fn ZS9qc29uX3Jlc3VsdHNfZ2VuZXJhdG9yLnB5OgorCiAyMDEwLTExLTE1ICBBbmRyYXMgQmVjc2kg IDxhYmVjc2lAd2Via2l0Lm9yZz4KIAogICAgICAgICBSZXZpZXdlZCBieSBBbmRyZWFzIEtsaW5n LgpkaWZmIC0tZ2l0IGEvV2ViS2l0VG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMv bGF5b3V0X3BhY2thZ2UvanNvbl9sYXlvdXRfcmVzdWx0c19nZW5lcmF0b3IucHkgYi9XZWJLaXRU b29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9qc29uX2xh eW91dF9yZXN1bHRzX2dlbmVyYXRvci5weQppbmRleCAxMDFkMzBiMTZiMTljY2M0NDgwYzYzZTZh NDYyYzY4OWU4YWM1ZmM4Li5iMDU0YzViMjMwNTRmZjlmMGM4ZWZjOTRhZDE0YmFjY2IxZGI0Y2Jm IDEwMDY0NAotLS0gYS9XZWJLaXRUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9s YXlvdXRfcGFja2FnZS9qc29uX2xheW91dF9yZXN1bHRzX2dlbmVyYXRvci5weQorKysgYi9XZWJL aXRUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9qc29u X2xheW91dF9yZXN1bHRzX2dlbmVyYXRvci5weQpAQCAtMTI5LDYgKzEyOSwxMCBAQCBjbGFzcyBK U09OTGF5b3V0UmVzdWx0c0dlbmVyYXRvcihqc29uX3Jlc3VsdHNfZ2VuZXJhdG9yLkpTT05SZXN1 bHRzR2VuZXJhdG9yQmFzZQogICAgICAgICByZXR1cm4gc2VsZi5QQVNTX1JFU1VMVAogCiAgICAg IyBvdmVycmlkZQorICAgIGRlZiBfZ2V0X3Jlc3VsdF9jaGFyKHNlbGYsIHRlc3RfbmFtZSk6Cisg ICAgICAgIHJldHVybiBzZWxmLl9nZXRfbW9kaWZpZXJfY2hhcih0ZXN0X25hbWUpCisKKyAgICAj IG92ZXJyaWRlCiAgICAgZGVmIF9jb252ZXJ0X2pzb25fdG9fY3VycmVudF92ZXJzaW9uKHNlbGYs IHJlc3VsdHNfanNvbik6CiAgICAgICAgIGFyY2hpdmVfdmVyc2lvbiA9IE5vbmUKICAgICAgICAg aWYgc2VsZi5WRVJTSU9OX0tFWSBpbiByZXN1bHRzX2pzb246CmRpZmYgLS1naXQgYS9XZWJLaXRU b29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9qc29uX3Jl c3VsdHNfZ2VuZXJhdG9yLnB5IGIvV2ViS2l0VG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRf dGVzdHMvbGF5b3V0X3BhY2thZ2UvanNvbl9yZXN1bHRzX2dlbmVyYXRvci5weQppbmRleCAzMjY3 NzE4MjI4ZWE2YTk0N2QwOTNjODk1YWRiYzI0M2U1YjFiOGYwLi4zNDg0ZmNkNGM1YzY4NWI3OTEx YjRiMzg2YzM3MGI3ZmMyZTYwNDIxIDEwMDY0NAotLS0gYS9XZWJLaXRUb29scy9TY3JpcHRzL3dl YmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2FnZS9qc29uX3Jlc3VsdHNfZ2VuZXJhdG9y LnB5CisrKyBiL1dlYktpdFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2xheW91 dF9wYWNrYWdlL2pzb25fcmVzdWx0c19nZW5lcmF0b3IucHkKQEAgLTMwMyw2ICszMDMsMjMgQEAg Y2xhc3MgSlNPTlJlc3VsdHNHZW5lcmF0b3JCYXNlKG9iamVjdCk6CiAKICAgICAgICAgcmV0dXJu IEpTT05SZXN1bHRzR2VuZXJhdG9yLlBBU1NfUkVTVUxUCiAKKyAgICBkZWYgX2dldF9yZXN1bHRf Y2hhcihzZWxmLCB0ZXN0X25hbWUpOgorICAgICAgICAiIiJSZXR1cm5zIGEgc2luZ2xlIGNoYXIg KGUuZy4gU0tJUF9SRVNVTFQsIEZBSUxfUkVTVUxULAorICAgICAgICBQQVNTX1JFU1VMVCwgTk9f REFUQV9SRVNVTFQsIGV0YykgdGhhdCBpbmRpY2F0ZXMgdGhlIHRlc3QgcmVzdWx0CisgICAgICAg IGZvciB0aGUgZ2l2ZW4gdGVzdF9uYW1lLgorICAgICAgICAiIiIKKyAgICAgICAgaWYgdGVzdF9u YW1lIG5vdCBpbiBzZWxmLl90ZXN0X3Jlc3VsdHNfbWFwOgorICAgICAgICAgICAgcmV0dXJuIEpT T05SZXN1bHRzR2VuZXJhdG9yLk5PX0RBVEFfUkVTVUxUCisKKyAgICAgICAgdGVzdF9yZXN1bHQg PSBzZWxmLl90ZXN0X3Jlc3VsdHNfbWFwW3Rlc3RfbmFtZV0KKyAgICAgICAgaWYgdGVzdF9yZXN1 bHQubW9kaWZpZXIgPT0gVGVzdFJlc3VsdC5ESVNBQkxFRDoKKyAgICAgICAgICAgIHJldHVybiBK U09OUmVzdWx0c0dlbmVyYXRvci5TS0lQX1JFU1VMVAorCisgICAgICAgIGlmIHRlc3RfcmVzdWx0 LmZhaWxlZDoKKyAgICAgICAgICAgIHJldHVybiBKU09OUmVzdWx0c0dlbmVyYXRvci5GQUlMX1JF U1VMVAorCisgICAgICAgIHJldHVybiBKU09OUmVzdWx0c0dlbmVyYXRvci5QQVNTX1JFU1VMVAor CiAgICAgIyBGSVhNRTogQ2FsbGVycyBzaG91bGQgdXNlIHNjbS5weSBpbnN0ZWFkLgogICAgICMg RklYTUU6IElkZW50aWZ5IGFuZCBmaXggdGhlIHJ1bi10aW1lIGVycm9ycyB0aGF0IHdlcmUgb2Jz ZXJ2ZWQgb24gV2luZG93cwogICAgICMgY2hyb21pdW0gYnVpbGRib3Qgd2hlbiB3ZSBoYWQgdXBk YXRlZCB0aGlzIGNvZGUgdG8gdXNlIHNjbS5weSBvbmNlIGJlZm9yZS4KQEAgLTQ4NCw3ICs1MDEs NyBAQCBjbGFzcyBKU09OUmVzdWx0c0dlbmVyYXRvckJhc2Uob2JqZWN0KToKICAgICAgICAgICB0 ZXN0czogRGljdGlvbmFyeSBjb250YWluaW5nIHRlc3QgcmVzdWx0IGVudHJpZXMuCiAgICAgICAg ICIiIgogCi0gICAgICAgIHJlc3VsdCA9IHNlbGYuX2dldF9tb2RpZmllcl9jaGFyKHRlc3RfbmFt ZSkKKyAgICAgICAgcmVzdWx0ID0gc2VsZi5fZ2V0X3Jlc3VsdF9jaGFyKHRlc3RfbmFtZSkKICAg ICAgICAgdGltZSA9IHNlbGYuX2dldF90ZXN0X3RpbWluZyh0ZXN0X25hbWUpCiAKICAgICAgICAg aWYgdGVzdF9uYW1lIG5vdCBpbiB0ZXN0czoK 74347 2010-11-18 19:41:57 -0800 2010-11-19 07:15:19 -0800 Patch bug-49702-20101118194156.patch text/plain 1712 kinuko ZGlmZiAtLWdpdCBhL1dlYktpdFRvb2xzL0NoYW5nZUxvZyBiL1dlYktpdFRvb2xzL0NoYW5nZUxv ZwppbmRleCBlMjc4ZDM2ZTJmNTRkNWYzZjhjNjdiMDVmNjU1MzcyYTZiYTg2NWI3Li40OTBmNDgz YzY4MDRhOTVkMWMzYTlmZGM3YjU4ODliMDg2MmMyOTAzIDEwMDY0NAotLS0gYS9XZWJLaXRUb29s cy9DaGFuZ2VMb2cKKysrIGIvV2ViS2l0VG9vbHMvQ2hhbmdlTG9nCkBAIC0xLDMgKzEsMTQgQEAK KzIwMTAtMTEtMTggIEtpbnVrbyBZYXN1ZGEgIDxraW51a29AY2hyb21pdW0ub3JnPgorCisgICAg ICAgIFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIFtDaHJvbWl1bV0gUkVH UkVTU0lPTjogUmVjb3JkIGFjdHVhbCB0ZXN0IHJlc3VsdHMgaW4gdGhlIEpTT04gcmVzdWx0IHN1 bW1hcnkgZm9yIG5vbi1sYXlvdXQgdGVzdHMKKyAgICAgICAgaHR0cHM6Ly9idWdzLndlYmtpdC5v cmcvc2hvd19idWcuY2dpP2lkPTQ5NzAyCisKKyAgICAgICAgVXBkYXRpbmcgdGVzdCBjb2RlLCB3 aGljaCBzaG91bGQgaGF2ZSBiZWVuIGluY2x1ZGVkIGluIHI3MjM1Ny4KKworICAgICAgICAqIFNj cmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2xheW91dF9wYWNrYWdlL2pzb25fcmVzdWx0c19n ZW5lcmF0b3JfdW5pdHRlc3QucHk6CisKIDIwMTAtMTEtMTggIEdhdmluIEJhcnJhY2xvdWdoICA8 YmFycmFjbG91Z2hAYXBwbGUuY29tPgogCiAgICAgICAgIFJldmlld2VkIGJ5IE9saXZlciBIdW50 LgpkaWZmIC0tZ2l0IGEvV2ViS2l0VG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMv bGF5b3V0X3BhY2thZ2UvanNvbl9yZXN1bHRzX2dlbmVyYXRvcl91bml0dGVzdC5weSBiL1dlYktp dFRvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL2xheW91dF9wYWNrYWdlL2pzb25f cmVzdWx0c19nZW5lcmF0b3JfdW5pdHRlc3QucHkKaW5kZXggNjA2YTYxM2NmOGQzZTBhY2RkMzFj ZmMzOWFmNmI5ODczYmZlMGY1Ni4uZWYzNmZjOTQwNmJlMTU1NDY5MjJkMWRhMDA0YTk4OTgyNjBi NGE5NSAxMDA2NDQKLS0tIGEvV2ViS2l0VG9vbHMvU2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVz dHMvbGF5b3V0X3BhY2thZ2UvanNvbl9yZXN1bHRzX2dlbmVyYXRvcl91bml0dGVzdC5weQorKysg Yi9XZWJLaXRUb29scy9TY3JpcHRzL3dlYmtpdHB5L2xheW91dF90ZXN0cy9sYXlvdXRfcGFja2Fn ZS9qc29uX3Jlc3VsdHNfZ2VuZXJhdG9yX3VuaXR0ZXN0LnB5CkBAIC0xODAsMTAgKzE4MCwxMCBA QCBjbGFzcyBKU09OR2VuZXJhdG9yVGVzdCh1bml0dGVzdC5UZXN0Q2FzZSk6CiAgICAgICAgICAg ICAgICAgdGVzdCA9IHRlc3RzW3Rlc3RfbmFtZV0KIAogICAgICAgICAgICAgICAgIGZhaWxlZCA9 IDAKLSAgICAgICAgICAgICAgICBtb2RpZmllciA9IHNlbGYuX2dldF90ZXN0X21vZGlmaWVyKHRl c3RfbmFtZSkKICAgICAgICAgICAgICAgICBmb3IgcmVzdWx0IGluIHRlc3RbSlJHLlJFU1VMVFNd OgotICAgICAgICAgICAgICAgICAgICBpZiByZXN1bHRbMV0gPT0gbW9kaWZpZXI6CisgICAgICAg ICAgICAgICAgICAgIGlmIHJlc3VsdFsxXSA9PSBKUkcuRkFJTF9SRVNVTFQ6CiAgICAgICAgICAg ICAgICAgICAgICAgICBmYWlsZWQgPSByZXN1bHRbMF0KKwogICAgICAgICAgICAgICAgIHNlbGYu YXNzZXJ0RXF1YWwoMSwgZmFpbGVkKQogCiAgICAgICAgICAgICAgICAgdGltaW5nX2NvdW50ID0g MAo=