Bug 85283

Summary: Reftest mismatches are providing undefined diff percent in results.html
Product: WebKit Reporter: Zan Dobersek <zan>
Component: Tools / TestsAssignee: Zan Dobersek <zan>
Status: RESOLVED FIXED    
Severity: Normal CC: abarth, dglazkov, dpranke, ojan, simon.fraser, webkit.review.bot
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Archive of layout-test-results from ec2-cr-linux-02
none
Patch
none
Patch none

Zan Dobersek
Reported 2012-05-01 07:50:57 PDT
When a reftest mismatch occurs in testing, results.html entry holds a link to the diffs.png that contains a text line 'diff (undefined%)'. It seems reftest mismatch failures are not given a diff percent attribute in JSON data as expected by the results.html file.
Attachments
Patch (1.59 KB, patch)
2012-05-01 07:56 PDT, Zan Dobersek 		no flags
DetailsFormatted DiffDiff
Archive of layout-test-results from ec2-cr-linux-02 (6.27 MB, application/zip)
2012-05-01 08:25 PDT, WebKit Review Bot 		no flags
Details
Patch (1.58 KB, patch)
2012-05-01 09:14 PDT, Zan Dobersek 		no flags
DetailsFormatted DiffDiff
Patch (5.97 KB, patch)
2012-05-01 22:59 PDT, Zan Dobersek 		no flags
DetailsFormatted DiffDiff
Show Obsolete (2) View All Add attachment proposed patch, testcase, etc.
Zan Dobersek
Comment 1 2012-05-01 07:56:15 PDT
Created attachment 139627 [details] Patch
Zan Dobersek
Comment 2 2012-05-01 07:59:09 PDT
Adding authors and reviewers of the last few changes to this file to get this reviewed. Thanks in advance!
WebKit Review Bot
Comment 3 2012-05-01 08:25:32 PDT
Comment on attachment 139627 [details] Patch Attachment 139627 [details] did not pass chromium-ews (chromium-xvfb): Output: http://queues.webkit.org/results/12588578 New failing tests: fast/harness/results.html
WebKit Review Bot
Comment 4 2012-05-01 08:25:39 PDT
Created attachment 139630 [details] Archive of layout-test-results from ec2-cr-linux-02 The attached test failures were seen while running run-webkit-tests on the chromium-ews. Bot: ec2-cr-linux-02 Port: <class 'webkitpy.common.config.ports.ChromiumXVFBPort'> Platform: Linux-2.6.35-28-virtual-x86_64-with-Ubuntu-10.10-maverick
Zan Dobersek
Comment 5 2012-05-01 09:14:09 PDT
Created attachment 139633 [details] Patch Sigh, forgot the percent sign.
Ojan Vafai
Comment 6 2012-05-01 09:30:52 PDT
Comment on attachment 139633 [details] Patch This seems like the wrong layer to fix this in. We should fix ImageDiff to include the diff percentage for reftests as well.
Zan Dobersek
Comment 7 2012-05-01 22:59:49 PDT
Created attachment 139752 [details] Patch
Ojan Vafai
Comment 8 2012-05-02 09:51:49 PDT
Comment on attachment 139752 [details] Patch Looks great. Thanks!
Zan Dobersek
Comment 9 2012-05-02 11:00:33 PDT
Comment on attachment 139752 [details] Patch Clearing flags on attachment: 139752 Committed r115853: <http://trac.webkit.org/changeset/115853>
Zan Dobersek
Comment 10 2012-05-02 11:00:59 PDT
All reviewed patches have been landed. Closing bug.
Dirk Pranke
Comment 11 2012-05-02 14:06:31 PDT
Comment on attachment 139752 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=139752&action=review > Tools/Scripts/webkitpy/layout_tests/controllers/test_result_writer.py:71 > + diff_result = port.diff_image(driver_output.image, expected_driver_output.image, tolerance=0) Minor quibble: I would've probably used a destructuring assignment here, like "diff_result, diff_percent = port.diff_image(...)"
Note You need to log in before you can comment on or make changes to this bug.