Bug 49702 – [Chromium] REGRESSION: Record actual test results in the JSON result summary for non-layout tests

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Kinuko Yasuda]

REGRESSION: Record actual test results in the JSON result summary for non-layout tests

Since r71952 we started recording test modifiers (FLAKY/FAILS) for failed tests in the failure summaries (fixableCounts), but the patch wrongly changed the way we record the failure details for each test too.

We should record 
1. actual failure/pass info for each test (this has gotten broken in r71952), and
2. FLAKY/FAILS modifier stats for failure summaries (this was introduced in r71952).

[Kinuko Yasuda]

Created attachment 74188 [details]
Patch

[Kinuko Yasuda]

Committed r72357: <http://trac.webkit.org/changeset/72357>

[David Levin]

There are no tests for this code?

[Kinuko Yasuda]

(In reply to comment #3)
> There are no tests for this code?

Oh, it does.  I forgot to update the test expectations in the unittest.
Am going to upload a follow-up patch.

[Kinuko Yasuda]

Created attachment 74347 [details]
Patch

[Eric Seidel (no email)]

This appears to have broken the test-webkitpy tests.

[Kinuko Yasuda]

(In reply to comment #6)
> This appears to have broken the test-webkitpy tests.

I'm rolling out this change for now.

[Ojan Vafai]

Comment on attachment 74347 [details]
Patch

Looks like _get_test_modifier isn't used anymore. Can we delete it?

[Kinuko Yasuda]

(In reply to comment #8)
> (From update of attachment 74347 [details])
> Looks like _get_test_modifier isn't used anymore. Can we delete it?

It's used to record the summary info (fixable counts).  _get_result_char is used  to record each test result.  For layout tests they're same but for non-layout tests for now we're recording different stats.

[Kinuko Yasuda]

(In reply to comment #9)
> (In reply to comment #8)
> > (From update of attachment 74347 [details] [details])
> > Looks like _get_test_modifier isn't used anymore. Can we delete it?
> 
> It's used to record the summary info (fixable counts).  _get_result_char is used  to record each test result.  For layout tests they're same but for non-layout tests for now we're recording different stats.

Oops, of course you meant the one in the unittest.  Yes it can be deleted.

[Kinuko Yasuda]

Committed r72445: <http://trac.webkit.org/changeset/72445>