Bug 87187 – Acid3 fails in WebKit2 regression tests

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Stephanie Lewis]

Only happening on WebKit2.  Tracked down to between 117246-117249.  I suspect http://trac.webkit.org/changeset/117248.   Works correctly in browser.

[Radar WebKit Bug Importer]

<rdar://problem/11511011>

[Stephanie Lewis]

http://build.webkit.org/results/Lion%20Debug%20(WebKit2%20Tests)/r118075%20(7370)/results.html

Failing Results:

--- /Volumes/Data/slave/lion-intel-debug-tests-wk2/build/layout-test-results/http/tests/misc/acid3-expected.txt
+++ /Volumes/Data/slave/lion-intel-debug-tests-wk2/build/layout-test-results/http/tests/misc/acid3-actual.txt
@@ -229,7 +229,7 @@
         RenderBody {BODY} at (8,16) size 284x0
           RenderBlock {P} at (0,0) size 284x0
 layer at (17,18) size 80x36
-  RenderBlock (positioned) {A} at (17,18) size 80x36 [color=#FFFFFF]
+  RenderBlock (positioned) {A} at (17,18) size 80x36 [color=#FF0000]
     RenderText {#text} at (0,0) size 71x36
       text run at (0,0) width 67: "YOU SHOULD"
       text run at (0,12) width 71: "NOT SEE THIS"

[Stephanie Lewis]

Committed failing results http://trac.webkit.org/projects/webkit/changeset/118101

[Andras Becsi]

(In reply to comment #3)
> Committed failing results http://trac.webkit.org/projects/webkit/changeset/118101

The history of this test is really strange.

The checked in expected file now matches the platform independent expected file which was updated in http://trac.webkit.org/changeset/60999 together with a png pixel result which shows the red text "YOU SHOULD NOT SEE THIS AT ALL" in the top left corner. Although the linktest issue has already been there even before that (see https://bugs.webkit.org/show_bug.cgi?id=40428).

Further more the mac-wk2 specific expected file was already checked in as a "failing" result in http://trac.webkit.org/changeset/111309.
(https://bugs.webkit.org/show_bug.cgi?id=81621).

The issue seems to be "fixed" by enabling visited link tracking in the test with layoutTestController.keepWebHistory(), but this is obviously just hiding the real issue, which has already been reported four years ago. (https://bugs.webkit.org/show_bug.cgi?id=18146).

The linktest problem is also reproducible on http://acid3.acidtests.org with recent chromium in incognito mode, with clean history also outside incognito mode but it disappears if the page is reloaded.
The issues are not present neither in Firefox nor in Opera.

[Andras Becsi]

*** Bug 81621 has been marked as a duplicate of this bug. ***

[Andras Becsi]

*** Bug 18146 has been marked as a duplicate of this bug. ***

[Alexey Proskuryakov]

> The issue seems to be "fixed" by enabling visited link tracking in the test with layoutTestController.keepWebHistory(), but this is obviously just hiding the real issue, which has already been reported four years ago. (https://bugs.webkit.org/show_bug.cgi?id=18146)

Could you please explain what the real issue is in more detail?

Bug 18146 has likely been resolved ages ago, as Acid3 has been fully passing in browser for years. So, it's not clear why there is any real issue besides regression test breakage that started with r117248.

[Andras Becsi]

(In reply to comment #7)
> > The issue seems to be "fixed" by enabling visited link tracking in the test with layoutTestController.keepWebHistory(), but this is obviously just hiding the real issue, which has already been reported four years ago. (https://bugs.webkit.org/show_bug.cgi?id=18146)
> 
> Could you please explain what the real issue is in more detail?
> 
> Bug 18146 has likely been resolved ages ago, as Acid3 has been fully passing in browser for years. So, it's not clear why there is any real issue besides regression test breakage that started with r117248.

You are right, the issue is in fact that the acid3 test does rely on visited link tracking.
It passes in the browser which tracks visited links, but the test misses the layoutTestController.keepWebHistory() call so it fails in DRT which properly disables visited links before each test, however the checked in failing platform independent result disguised this issue.
Before r117248 this was "fixed" on WK2 by the fact that visited link tracking was not disabled properly which gave a correct result with WTR, thus mac-wk2 needed its own result.
This test is skipped on Qt because of font metric issues which is really unfortunate, Qt needs a platform dependent result until the font issues are fixed.

I'll prepare a patch for the test and update the result tomorrow.

[Andras Becsi]

Created attachment 144064 [details]
proposed patch

[Andras Becsi]

I opened a bug for the Qt issue, because it seems to be related to the issue that test infrastructure is not using monospace fonts: https://bugs.webkit.org/show_bug.cgi?id=87501

[Alexey Proskuryakov]

Comment on attachment 144064 [details]
proposed patch

Makes sense to me.

[WebKit Review Bot]

Comment on attachment 144064 [details]
proposed patch

Clearing flags on attachment: 144064

Committed r118566: <http://trac.webkit.org/changeset/118566>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.