Bug 92847

Summary: REGRESSION(r123893): Reftest mismatches are run through ImageDiff with 0.1 tolerance
Product: WebKit Reporter: Zan Dobersek <zan>
Component: Tools / TestsAssignee: Dirk Pranke <dpranke>
Severity: Normal CC: abarth, dpranke, eric, ojan, rniwa, tony, webkit.review.bot
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Description Flags
Patch none

Description Zan Dobersek 2012-08-01 00:44:49 PDT
After r123893[1] and the image diffing code refactoring, the default tolerance is set to 0.1 for the port when diffing images[2]. With the tolerance for diffing reftest output being 0, the expression on the next line[3] evaluates to 0.1. This means the ImageDiff operates with 0.1 tolerance, which can lead to false positives (i.e. ImageDiff reports that the two images' differences are within the allowed boundaries).

I guess it should explicitly be checked whether the tolerance argument passed into base.diff_image is None and act accordingly. I don't know why the default value for the tolerance option is being overridden, though (if it is actually overridden rather than just ensured it exists).

[1] - http://trac.webkit.org/changeset/123893
[2] - http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py?rev=123893#L325
[2] - http://trac.webkit.org/browser/trunk/Tools/Scripts/webkitpy/layout_tests/port/base.py?rev=123893#L326
Comment 1 Dirk Pranke 2012-08-01 18:13:17 PDT
Created attachment 155940 [details]
Comment 2 Dirk Pranke 2012-08-01 18:13:49 PDT
whoops. good catch.
Comment 3 WebKit Review Bot 2012-08-01 19:31:35 PDT
Comment on attachment 155940 [details]

Clearing flags on attachment: 155940

Committed r124401: <http://trac.webkit.org/changeset/124401>
Comment 4 WebKit Review Bot 2012-08-01 19:31:39 PDT
All reviewed patches have been landed.  Closing bug.