Bug 73406 - [NRWT] TestResultWriter should pass tolerance=0 to port.diff_image in case of reftests.
Summary: [NRWT] TestResultWriter should pass tolerance=0 to port.diff_image in case of...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Hayato Ito
URL:
Keywords:
: 55652 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-11-30 00:27 PST by Hayato Ito
Modified: 2011-12-04 18:54 PST (History)
5 users (show)

See Also:


Attachments
pass zero (11.44 KB, patch)
2011-11-30 01:48 PST, Hayato Ito
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Hayato Ito 2011-11-30 00:27:39 PST
TestResultWriter should pass tolerance=0 explicitly to port.diff_image in case of reftests.
Mac port uses '0.1' as default value of tolerance. In case of reftests, we should use tolerance=0 in the phase of writing test results in addition to the phase of comparing images.

See https://bugs.webkit.org/show_bug.cgi?id=73381 also.
Comment 1 Ryosuke Niwa 2011-11-30 00:54:47 PST
This doesn't really help as ImageDiff itself has a default tolerance level.
Comment 2 Hayato Ito 2011-11-30 01:27:32 PST
Is there any ImageDiff which doesn't accept tolerance parameter?
It seems that ImageDiff used by chromium port doesn't use tolerance parameter.
I assumed its default is 'zero' though I've not investigated every ImageDiff program's behavior on every platform yet.


(In reply to comment #1)
> This doesn't really help as ImageDiff itself has a default tolerance level.
Comment 3 Hayato Ito 2011-11-30 01:29:46 PST
At least, passing tolerance=0 resolves mac port's issue and this doesn't affect other platforms because we don't pass tolerance parameter to ImageDiff on other platforms. Things should not become worse than now, I guess.
Comment 4 Hayato Ito 2011-11-30 01:48:23 PST
Created attachment 117143 [details]
pass zero
Comment 5 Ryosuke Niwa 2011-11-30 10:07:29 PST
Comment on attachment 117143 [details]
pass zero

View in context: https://bugs.webkit.org/attachment.cgi?id=117143&action=review

> Tools/Scripts/webkitpy/layout_tests/controllers/test_result_writer_unittest.py:54
> +                                             driver_output1, driver_output2, failures)

Wrong indentation.
Comment 6 WebKit Review Bot 2011-11-30 13:23:02 PST
Comment on attachment 117143 [details]
pass zero

Attachment 117143 [details] did not pass chromium-ews (chromium-xvfb):
Output: http://queues.webkit.org/results/10696325
Comment 7 WebKit Review Bot 2011-12-01 06:03:57 PST
Comment on attachment 117143 [details]
pass zero

Rejecting attachment 117143 [details] from commit-queue.

Failed to run "['/mnt/git/webkit-commit-queue/Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '-..." exit_code: 2

Last 500 characters of output:
and returned 256 at Tools/Scripts/update-webkit-chromium line 109.
Re-trying 'depot_tools/gclient sync --force --reset --delete_unversioned_trees'
No such file or directory at /mnt/git/webkit-commit-queue/Tools/Scripts/webkitdirs.pm line 2020.

Failed to run "['Tools/Scripts/build-webkit', '--release', '--chromium', '--update-chromium']" exit_code: 2
 sync --force --reset --delete_unversioned_trees'
No such file or directory at /mnt/git/webkit-commit-queue/Tools/Scripts/webkitdirs.pm line 2020.

Full output: http://queues.webkit.org/results/10695236
Comment 8 Ryosuke Niwa 2011-12-01 19:57:25 PST
Is https://bugs.webkit.org/show_bug.cgi?id=55652 a duplicate of this bug?
Comment 9 WebKit Review Bot 2011-12-01 20:32:44 PST
Comment on attachment 117143 [details]
pass zero

Clearing flags on attachment: 117143

Committed r101739: <http://trac.webkit.org/changeset/101739>
Comment 10 WebKit Review Bot 2011-12-01 20:32:49 PST
All reviewed patches have been landed.  Closing bug.
Comment 11 Hayato Ito 2011-12-04 18:53:42 PST
*** Bug 55652 has been marked as a duplicate of this bug. ***
Comment 12 Hayato Ito 2011-12-04 18:54:46 PST
Right. I marked it as duplicate of this bug.

(In reply to comment #8)
> Is https://bugs.webkit.org/show_bug.cgi?id=55652 a duplicate of this bug?