REGRESSION(r123893): Reftest mismatches are run through ImageDiff with 0.1 tolerance

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Zan Dobersek]

After r123893[1] and the image diffing code refactoring, the default tolerance is set to 0.1 for the port when diffing images[2]. With the tolerance for diffing reftest output being 0, the expression on the next line[3] evaluates to 0.1. This means the ImageDiff operates with 0.1 tolerance, which can lead to false positives (i.e. ImageDiff reports that the two images' differences are within the allowed boundaries).

I guess it should explicitly be checked whether the tolerance argument passed into base.diff_image is None and act accordingly. I don't know why the default value for the tolerance option is being overridden, though (if it is actually overridden rather than just ensured it exists).


[1] - http://trac.webkit.org/changeset/123893
[2] - http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py?rev=123893#L325
[2] - http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py?rev=123893#L326

[Dirk Pranke]

Created attachment 155940 [details]
Patch

[Dirk Pranke]

whoops. good catch.

[WebKit Review Bot]

Comment on attachment 155940 [details]
Patch

Clearing flags on attachment: 155940

Committed r124401: <http://trac.webkit.org/changeset/124401>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.