Bug 41644 – rebaseline-chromium-webkit-tests: wrong dup detection

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Kent Tamura]

If an image expectation for Linux and an image expectation for Windows are *similar*, the Linux expectation imae is unexpectedly removed.

How to repeat:
 Rebaseline Linux and Windows expectation on Mac Leopard
1. build ImageDiff (build-webkit and build-dumprendertree)
2. prepend "REBASELINE" to the following line in test_expectations.txt
  BUG48154 WIN LINUX: fast/forms/indeterminate.html = IMAGE
3. run "rebaseline-chromium-webkit-tests -w"

Expected result:
  Because the Linux image and the Windows image are different, I expect
   LayoutTests/platform/chromium-win/fast/forms/indeterminate-expected.checksum
   LayoutTests/platform/chromium-win/fast/forms/indeterminate-expected.png
   LayoutTests/platform/chromium-linux/fast/forms/indeterminate-expected.checksum
   LayoutTests/platform/chromium-linux/fast/forms/indeterminate-expected.png
are updated.

Actual result:
   LayoutTests/platform/chromium-win/fast/forms/indeterminate-expected.checksum
   LayoutTests/platform/chromium-win/fast/forms/indeterminate-expected.png
   LayoutTests/platform/chromium-linux/fast/forms/indeterminate-expected.checksum
are updated, and 
   LayoutTests/platform/chromium-linux/fast/forms/indeterminate-expected.png
is removed.

Log:
100706 12:14:27 rebaseline_chromium_webkit_tests.py:154 INFO ------------------ Extracting and adding new baselines: linux ------------------
100706 12:14:27 rebaseline_chromium_webkit_tests.py:423 INFO Test 1: fast/forms/indeterminate.html
100706 12:14:27 rebaseline_chromium_webkit_tests.py:434 INFO   .txt file not in archive.
100706 12:14:27 rebaseline_chromium_webkit_tests.py:438 INFO   .png file found in archive.
100706 12:14:27 rebaseline_chromium_webkit_tests.py:527 INFO   Found same baseline at /Users/tkent/WebKit/LayoutTests/platform/chromium-win/fast/forms/indeterminate-expected.png
100706 12:14:28 rebaseline_chromium_webkit_tests.py:438 INFO   .checksum file found in archive.
100706 12:14:29 rebaseline_chromium_webkit_tests.py:487 INFO   Rebaseline succeeded.

Root cause:
rebaseline-chromium-webkit-tests uses diff_image() in layout_tests_port/webkit.py, and it pass "--torelance 0.1" to ImageDiff.
So diff_image() returns False if the difference between two images are small,

[Kent Tamura]

Created attachment 60583 [details]
Patch

[Hajime Morrita]

Comment on attachment 60583 [details]
Patch

http://wkrietveld.appspot.com/41644/diff/1/4
File WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py (right):

http://wkrietveld.appspot.com/41644/diff/1/4#newcode119
WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py:119: diff_filename=None, tolerance=None):
how about just to pass 0.1 ?

http://wkrietveld.appspot.com/41644/diff/1/4#newcode124
WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py:124: # parameter, or make it go away and aways use exact matches.
nit: aways -> always

[Kent Tamura]

Created attachment 60594 [details]
Patch 2

[Kent Tamura]

Comment on attachment 60583 [details]
Patch

http://wkrietveld.appspot.com/41644/diff/1/4
File WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py (right):

http://wkrietveld.appspot.com/41644/diff/1/4#newcode119
WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py:119: diff_filename=None, tolerance=None):
On 2010/07/06 05:36:56, morrita wrote:
> how about just to pass 0.1 ?

Sounds good.  Done.

http://wkrietveld.appspot.com/41644/diff/1/4#newcode124
WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py:124: # parameter, or make it go away and aways use exact matches.
On 2010/07/06 05:36:56, morrita wrote:
> nit: aways -> always

Done.

[Shinichiro Hamaji]

Comment on attachment 60594 [details]
Patch 2

Looks good.

[Kent Tamura]

Comment on attachment 60594 [details]
Patch 2

Clearing flags on attachment: 60594

Committed r62620: <http://trac.webkit.org/changeset/62620>

[Kent Tamura]

All reviewed patches have been landed.  Closing bug.