Bug 41644

Summary: rebaseline-chromium-webkit-tests: wrong dup detection
Product: WebKit Reporter: Kent Tamura <tkent>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: hamaji, morrita, victorw
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Other   
OS: OS X 10.5   
Attachments:
Description Flags
Patch
none
Patch 2 none

Description Kent Tamura 2010-07-05 20:22:37 PDT
If an image expectation for Linux and an image expectation for Windows are *similar*, the Linux expectation imae is unexpectedly removed.

How to repeat:
 Rebaseline Linux and Windows expectation on Mac Leopard
1. build ImageDiff (build-webkit and build-dumprendertree)
2. prepend "REBASELINE" to the following line in test_expectations.txt
  BUG48154 WIN LINUX: fast/forms/indeterminate.html = IMAGE
3. run "rebaseline-chromium-webkit-tests -w"

Expected result:
  Because the Linux image and the Windows image are different, I expect
   LayoutTests/platform/chromium-win/fast/forms/indeterminate-expected.checksum
   LayoutTests/platform/chromium-win/fast/forms/indeterminate-expected.png
   LayoutTests/platform/chromium-linux/fast/forms/indeterminate-expected.checksum
   LayoutTests/platform/chromium-linux/fast/forms/indeterminate-expected.png
are updated.

Actual result:
   LayoutTests/platform/chromium-win/fast/forms/indeterminate-expected.checksum
   LayoutTests/platform/chromium-win/fast/forms/indeterminate-expected.png
   LayoutTests/platform/chromium-linux/fast/forms/indeterminate-expected.checksum
are updated, and 
   LayoutTests/platform/chromium-linux/fast/forms/indeterminate-expected.png
is removed.

Log:
100706 12:14:27 rebaseline_chromium_webkit_tests.py:154 INFO ------------------ Extracting and adding new baselines: linux ------------------
100706 12:14:27 rebaseline_chromium_webkit_tests.py:423 INFO Test 1: fast/forms/indeterminate.html
100706 12:14:27 rebaseline_chromium_webkit_tests.py:434 INFO   .txt file not in archive.
100706 12:14:27 rebaseline_chromium_webkit_tests.py:438 INFO   .png file found in archive.
100706 12:14:27 rebaseline_chromium_webkit_tests.py:527 INFO   Found same baseline at /Users/tkent/WebKit/LayoutTests/platform/chromium-win/fast/forms/indeterminate-expected.png
100706 12:14:28 rebaseline_chromium_webkit_tests.py:438 INFO   .checksum file found in archive.
100706 12:14:29 rebaseline_chromium_webkit_tests.py:487 INFO   Rebaseline succeeded.

Root cause:
rebaseline-chromium-webkit-tests uses diff_image() in layout_tests_port/webkit.py, and it pass "--torelance 0.1" to ImageDiff.
So diff_image() returns False if the difference between two images are small,
Comment 1 Kent Tamura 2010-07-05 21:41:29 PDT
Created attachment 60583 [details]
Patch
Comment 2 Hajime Morrita 2010-07-05 22:36:58 PDT
Comment on attachment 60583 [details]
Patch

http://wkrietveld.appspot.com/41644/diff/1/4
File WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py (right):

http://wkrietveld.appspot.com/41644/diff/1/4#newcode119
WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py:119: diff_filename=None, tolerance=None):
how about just to pass 0.1 ?

http://wkrietveld.appspot.com/41644/diff/1/4#newcode124
WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py:124: # parameter, or make it go away and aways use exact matches.
nit: aways -> always
Comment 3 Kent Tamura 2010-07-06 00:12:11 PDT
Created attachment 60594 [details]
Patch 2
Comment 4 Kent Tamura 2010-07-06 00:13:06 PDT
Comment on attachment 60583 [details]
Patch

http://wkrietveld.appspot.com/41644/diff/1/4
File WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py (right):

http://wkrietveld.appspot.com/41644/diff/1/4#newcode119
WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py:119: diff_filename=None, tolerance=None):
On 2010/07/06 05:36:56, morrita wrote:
> how about just to pass 0.1 ?

Sounds good.  Done.

http://wkrietveld.appspot.com/41644/diff/1/4#newcode124
WebKitTools/Scripts/webkitpy/layout_tests/port/webkit.py:124: # parameter, or make it go away and aways use exact matches.
On 2010/07/06 05:36:56, morrita wrote:
> nit: aways -> always

Done.
Comment 5 Shinichiro Hamaji 2010-07-06 05:46:56 PDT
Comment on attachment 60594 [details]
Patch 2

Looks good.
Comment 6 Kent Tamura 2010-07-06 21:42:06 PDT
Comment on attachment 60594 [details]
Patch 2

Clearing flags on attachment: 60594

Committed r62620: <http://trac.webkit.org/changeset/62620>
Comment 7 Kent Tamura 2010-07-06 21:42:14 PDT
All reviewed patches have been landed.  Closing bug.