Bug 87187 - Acid3 fails in WebKit2 regression tests
Summary: Acid3 fails in WebKit2 regression tests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P1 Normal
Assignee: Andras Becsi
URL:
Keywords: InRadar, MakingBotsRed
: 18146 81621 (view as bug list)
Depends on:
Blocks:
 
Reported: 2012-05-22 17:33 PDT by Stephanie Lewis
Modified: 2012-05-25 14:41 PDT (History)
9 users (show)

See Also:


Attachments
proposed patch (319.76 KB, patch)
2012-05-25 07:14 PDT, Andras Becsi
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Stephanie Lewis 2012-05-22 17:33:40 PDT
Only happening on WebKit2.  Tracked down to between 117246-117249.  I suspect http://trac.webkit.org/changeset/117248.   Works correctly in browser.
Comment 1 Radar WebKit Bug Importer 2012-05-22 17:35:08 PDT
<rdar://problem/11511011>
Comment 2 Stephanie Lewis 2012-05-22 17:36:07 PDT
http://build.webkit.org/results/Lion%20Debug%20(WebKit2%20Tests)/r118075%20(7370)/results.html

Failing Results:

--- /Volumes/Data/slave/lion-intel-debug-tests-wk2/build/layout-test-results/http/tests/misc/acid3-expected.txt
+++ /Volumes/Data/slave/lion-intel-debug-tests-wk2/build/layout-test-results/http/tests/misc/acid3-actual.txt
@@ -229,7 +229,7 @@
         RenderBody {BODY} at (8,16) size 284x0
           RenderBlock {P} at (0,0) size 284x0
 layer at (17,18) size 80x36
-  RenderBlock (positioned) {A} at (17,18) size 80x36 [color=#FFFFFF]
+  RenderBlock (positioned) {A} at (17,18) size 80x36 [color=#FF0000]
     RenderText {#text} at (0,0) size 71x36
       text run at (0,0) width 67: "YOU SHOULD"
       text run at (0,12) width 71: "NOT SEE THIS"
Comment 3 Stephanie Lewis 2012-05-22 19:52:01 PDT
Committed failing results http://trac.webkit.org/projects/webkit/changeset/118101
Comment 4 Andras Becsi 2012-05-24 09:29:40 PDT
(In reply to comment #3)
> Committed failing results http://trac.webkit.org/projects/webkit/changeset/118101

The history of this test is really strange.

The checked in expected file now matches the platform independent expected file which was updated in http://trac.webkit.org/changeset/60999 together with a png pixel result which shows the red text "YOU SHOULD NOT SEE THIS AT ALL" in the top left corner. Although the linktest issue has already been there even before that (see https://bugs.webkit.org/show_bug.cgi?id=40428).

Further more the mac-wk2 specific expected file was already checked in as a "failing" result in http://trac.webkit.org/changeset/111309.
(https://bugs.webkit.org/show_bug.cgi?id=81621).

The issue seems to be "fixed" by enabling visited link tracking in the test with layoutTestController.keepWebHistory(), but this is obviously just hiding the real issue, which has already been reported four years ago. (https://bugs.webkit.org/show_bug.cgi?id=18146).

The linktest problem is also reproducible on http://acid3.acidtests.org with recent chromium in incognito mode, with clean history also outside incognito mode but it disappears if the page is reloaded.
The issues are not present neither in Firefox nor in Opera.
Comment 5 Andras Becsi 2012-05-24 09:34:54 PDT
*** Bug 81621 has been marked as a duplicate of this bug. ***
Comment 6 Andras Becsi 2012-05-24 09:35:24 PDT
*** Bug 18146 has been marked as a duplicate of this bug. ***
Comment 7 Alexey Proskuryakov 2012-05-24 11:35:22 PDT
> The issue seems to be "fixed" by enabling visited link tracking in the test with layoutTestController.keepWebHistory(), but this is obviously just hiding the real issue, which has already been reported four years ago. (https://bugs.webkit.org/show_bug.cgi?id=18146)

Could you please explain what the real issue is in more detail?

Bug 18146 has likely been resolved ages ago, as Acid3 has been fully passing in browser for years. So, it's not clear why there is any real issue besides regression test breakage that started with r117248.
Comment 8 Andras Becsi 2012-05-24 13:37:44 PDT
(In reply to comment #7)
> > The issue seems to be "fixed" by enabling visited link tracking in the test with layoutTestController.keepWebHistory(), but this is obviously just hiding the real issue, which has already been reported four years ago. (https://bugs.webkit.org/show_bug.cgi?id=18146)
> 
> Could you please explain what the real issue is in more detail?
> 
> Bug 18146 has likely been resolved ages ago, as Acid3 has been fully passing in browser for years. So, it's not clear why there is any real issue besides regression test breakage that started with r117248.

You are right, the issue is in fact that the acid3 test does rely on visited link tracking.
It passes in the browser which tracks visited links, but the test misses the layoutTestController.keepWebHistory() call so it fails in DRT which properly disables visited links before each test, however the checked in failing platform independent result disguised this issue.
Before r117248 this was "fixed" on WK2 by the fact that visited link tracking was not disabled properly which gave a correct result with WTR, thus mac-wk2 needed its own result.
This test is skipped on Qt because of font metric issues which is really unfortunate, Qt needs a platform dependent result until the font issues are fixed.

I'll prepare a patch for the test and update the result tomorrow.
Comment 9 Andras Becsi 2012-05-25 07:14:04 PDT
Created attachment 144064 [details]
proposed patch
Comment 10 Andras Becsi 2012-05-25 07:35:36 PDT
I opened a bug for the Qt issue, because it seems to be related to the issue that test infrastructure is not using monospace fonts: https://bugs.webkit.org/show_bug.cgi?id=87501
Comment 11 Alexey Proskuryakov 2012-05-25 10:14:22 PDT
Comment on attachment 144064 [details]
proposed patch

Makes sense to me.
Comment 12 WebKit Review Bot 2012-05-25 14:41:23 PDT
Comment on attachment 144064 [details]
proposed patch

Clearing flags on attachment: 144064

Committed r118566: <http://trac.webkit.org/changeset/118566>
Comment 13 WebKit Review Bot 2012-05-25 14:41:28 PDT
All reviewed patches have been landed.  Closing bug.