Bug 92847 - REGRESSION(r123893): Reftest mismatches are run through ImageDiff with 0.1 tolerance
Summary: REGRESSION(r123893): Reftest mismatches are run through ImageDiff with 0.1 to...
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Dirk Pranke
Depends on:
Reported: 2012-08-01 00:44 PDT by Zan Dobersek
Modified: 2012-08-01 19:31 PDT (History)
7 users (show)

See Also:

Patch (3.12 KB, patch)
2012-08-01 18:13 PDT, Dirk Pranke
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Zan Dobersek 2012-08-01 00:44:49 PDT
After r123893[1] and the image diffing code refactoring, the default tolerance is set to 0.1 for the port when diffing images[2]. With the tolerance for diffing reftest output being 0, the expression on the next line[3] evaluates to 0.1. This means the ImageDiff operates with 0.1 tolerance, which can lead to false positives (i.e. ImageDiff reports that the two images' differences are within the allowed boundaries).

I guess it should explicitly be checked whether the tolerance argument passed into base.diff_image is None and act accordingly. I don't know why the default value for the tolerance option is being overridden, though (if it is actually overridden rather than just ensured it exists).

[1] - http://trac.webkit.org/changeset/123893
[2] - http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py?rev=123893#L325
[2] - http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py?rev=123893#L326
Comment 1 Dirk Pranke 2012-08-01 18:13:17 PDT
Created attachment 155940 [details]
Comment 2 Dirk Pranke 2012-08-01 18:13:49 PDT
whoops. good catch.
Comment 3 WebKit Review Bot 2012-08-01 19:31:35 PDT
Comment on attachment 155940 [details]

Clearing flags on attachment: 155940

Committed r124401: <http://trac.webkit.org/changeset/124401>
Comment 4 WebKit Review Bot 2012-08-01 19:31:39 PDT
All reviewed patches have been landed.  Closing bug.