Bug 76243 – [chromium] Reftests expected to be image mismatch still generate image mismatch when ran by NRWT --no-pixel-tests.

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Hao Zheng]

For example, fast/harness/sample-fail-mismatch-reftest.html
When ran by 'NRWT fast/harness/sample-fail-mismatch-reftest.html', all is fine: All 1 tests ran as expected.
When ran by 'NRWT --no-pixel-tests fast/harness/sample-fail-mismatch-reftest.html', output:
Regressions: Unexpected image mismatch : (1)
  fast/harness/sample-fail-mismatch-reftest.html = IMAGE

In TestExpectations.matches_an_expected_result, it removes pixel expectation when pixel tests is disabled.
        if not pixel_tests_are_enabled:
            expected_results = remove_pixel_failures(expected_results)
However, in test_failures.determine_result_type, reftest failure is treated as image failure.
        elif is_image_failure or is_reftest_failure:
            return test_expectations.IMAGE

[Hao Zheng]

Created attachment 122380 [details]
Proposed patch.

[Tony Chang]

Comment on attachment 122380 [details]
Proposed patch.

This change looks correct, but can you add a unit test for the change to manager.py?

[Hao Zheng]

Created attachment 122596 [details]
Add test.

[WebKit Review Bot]

Comment on attachment 122596 [details]
Add test.

Clearing flags on attachment: 122596

Committed r105169: <http://trac.webkit.org/changeset/105169>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.