TestResultWriter should pass tolerance=0 explicitly to port.diff_image in case of reftests. Mac port uses '0.1' as default value of tolerance. In case of reftests, we should use tolerance=0 in the phase of writing test results in addition to the phase of comparing images. See https://bugs.webkit.org/show_bug.cgi?id=73381 also.
This doesn't really help as ImageDiff itself has a default tolerance level.
Is there any ImageDiff which doesn't accept tolerance parameter? It seems that ImageDiff used by chromium port doesn't use tolerance parameter. I assumed its default is 'zero' though I've not investigated every ImageDiff program's behavior on every platform yet. (In reply to comment #1) > This doesn't really help as ImageDiff itself has a default tolerance level.
At least, passing tolerance=0 resolves mac port's issue and this doesn't affect other platforms because we don't pass tolerance parameter to ImageDiff on other platforms. Things should not become worse than now, I guess.
Created attachment 117143 [details] pass zero
Comment on attachment 117143 [details] pass zero View in context: https://bugs.webkit.org/attachment.cgi?id=117143&action=review > Tools/Scripts/webkitpy/layout_tests/controllers/test_result_writer_unittest.py:54 > + driver_output1, driver_output2, failures) Wrong indentation.
Comment on attachment 117143 [details] pass zero Attachment 117143 [details] did not pass chromium-ews (chromium-xvfb): Output: http://queues.webkit.org/results/10696325
Comment on attachment 117143 [details] pass zero Rejecting attachment 117143 [details] from commit-queue. Failed to run "['/mnt/git/webkit-commit-queue/Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '-..." exit_code: 2 Last 500 characters of output: and returned 256 at Tools/Scripts/update-webkit-chromium line 109. Re-trying 'depot_tools/gclient sync --force --reset --delete_unversioned_trees' No such file or directory at /mnt/git/webkit-commit-queue/Tools/Scripts/webkitdirs.pm line 2020. Failed to run "['Tools/Scripts/build-webkit', '--release', '--chromium', '--update-chromium']" exit_code: 2 sync --force --reset --delete_unversioned_trees' No such file or directory at /mnt/git/webkit-commit-queue/Tools/Scripts/webkitdirs.pm line 2020. Full output: http://queues.webkit.org/results/10695236
Is https://bugs.webkit.org/show_bug.cgi?id=55652 a duplicate of this bug?
Comment on attachment 117143 [details] pass zero Clearing flags on attachment: 117143 Committed r101739: <http://trac.webkit.org/changeset/101739>
All reviewed patches have been landed. Closing bug.
*** Bug 55652 has been marked as a duplicate of this bug. ***
Right. I marked it as duplicate of this bug. (In reply to comment #8) > Is https://bugs.webkit.org/show_bug.cgi?id=55652 a duplicate of this bug?