Bug 49702

Summary:

[Chromium] REGRESSION: Record actual test results in the JSON result summary for non-layout tests

Product:

WebKit

Reporter:

Kinuko Yasuda <kinuko>

Component:

Tools / Tests

Assignee:

Nobody <webkit-unassigned>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

eric, levin, ojan

Priority:

Version:

528+ (Nightly build)

Hardware:

OS:

OS X 10.5

Bug Depends on:

49784

Bug Blocks:

Attachments:

Description	Flags
Patch	none
Patch	ojan: review+

Kinuko Yasuda

Reported 2010-11-17 17:33:17 PST

REGRESSION: Record actual test results in the JSON result summary for non-layout tests Since r71952 we started recording test modifiers (FLAKY/FAILS) for failed tests in the failure summaries (fixableCounts), but the patch wrongly changed the way we record the failure details for each test too. We should record 1. actual failure/pass info for each test (this has gotten broken in r71952), and 2. FLAKY/FAILS modifier stats for failure summaries (this was introduced in r71952).

Attachments
Patch (3.31 KB, patch) 2010-11-17 18:28 PST, Kinuko Yasuda	no flags	Details Formatted Diff Diff
Patch (1.67 KB, patch) 2010-11-18 19:41 PST, Kinuko Yasuda	ojan: review+	Details Formatted Diff Diff
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.

Kinuko Yasuda

Comment 1 2010-11-17 18:28:34 PST

Created attachment 74188 [details] Patch

Kinuko Yasuda

Comment 2 2010-11-18 18:06:15 PST

Committed r72357: <http://trac.webkit.org/changeset/72357>

David Levin

Comment 3 2010-11-18 18:08:19 PST

There are no tests for this code?

Kinuko Yasuda

Comment 4 2010-11-18 18:12:34 PST

(In reply to comment #3) > There are no tests for this code? Oh, it does. I forgot to update the test expectations in the unittest. Am going to upload a follow-up patch.

Kinuko Yasuda

Comment 5 2010-11-18 19:41:57 PST

Created attachment 74347 [details] Patch

Eric Seidel (no email)

Comment 6 2010-11-18 20:22:13 PST

This appears to have broken the test-webkitpy tests.

Kinuko Yasuda

Comment 7 2010-11-18 20:46:24 PST

(In reply to comment #6) > This appears to have broken the test-webkitpy tests. I'm rolling out this change for now.

Ojan Vafai

Comment 8 2010-11-19 07:15:19 PST

Comment on attachment 74347 [details] Patch Looks like _get_test_modifier isn't used anymore. Can we delete it?

Kinuko Yasuda

Comment 9 2010-11-19 13:24:33 PST

(In reply to comment #8) > (From update of attachment 74347 [details]) > Looks like _get_test_modifier isn't used anymore. Can we delete it? It's used to record the summary info (fixable counts). _get_result_char is used to record each test result. For layout tests they're same but for non-layout tests for now we're recording different stats.

Kinuko Yasuda

Comment 10 2010-11-19 13:27:02 PST

(In reply to comment #9) > (In reply to comment #8) > > (From update of attachment 74347 [details] [details]) > > Looks like _get_test_modifier isn't used anymore. Can we delete it? > > It's used to record the summary info (fixable counts). _get_result_char is used to record each test result. For layout tests they're same but for non-layout tests for now we're recording different stats. Oops, of course you meant the one in the unittest. Yes it can be deleted.

Kinuko Yasuda

Comment 11 2010-11-19 15:08:49 PST

Committed r72445: <http://trac.webkit.org/changeset/72445>

Note You need to log in before you can comment on or make changes to this bug.