Bug 242119 – [ iOS ] 2X fast/text/accessibility-bold-system-font (Layout-tests) are constant failures

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Hercules Hjalmarsson]

fast/text/accessibility-bold-system-font/accessibility-bold-system-font-2.html

Is a constant image failure on macOS and iOS.

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=fast%2Ftext%2Faccessibility-bold-system-font%2Faccessibility-bold-system-font-2.html

DIFF URL: 
https://build.webkit.org/results/Apple-Monterey-Release-WK2-Tests/251948%40main%20(4746)/fast/text/accessibility-bold-system-font/accessibility-bold-system-font-2-diffs.html

Diff images attached to this bug.

[Hercules Hjalmarsson]

Created attachment 460545 [details]
Diff images

[Radar WebKit Bug Importer]

<rdar://problem/96150140>

[Hercules Hjalmarsson]

I was able to reproduce the image failure at both Mac and iOS ToT running the test as follows:

run-webkit-tests fast/text/accessibility-bold-system-font/accessibility-bold-system-font-2.html

[Hercules Hjalmarsson]

The test was originally introduced at commit 189770@main, the history shows that it has been disabled for some time and then was re-enabled at commit 251883@main and has been failing ever since being re-enabled.

[Ryan Haddad]

(In reply to Hercules Hjalmarsson from comment #4)
> The test was originally introduced at commit 189770@main, the history shows
> that it has been disabled for some time and then was re-enabled at commit
> 251883@main and has been failing ever since being re-enabled.
Based on https://github.com/WebKit/WebKit/commit/3e3f9ab0f79b58eb597037a6de8ebf192ddc1728#diff-0b438e462819a7b0d06250189152c7bcfb07b7c39acd73938bdfccc7b25b8058R5323, I think the intention was for this test to continue failing on macOS. This is surfacing because this is an image failure instead of a text failure, so the [ Pass Fail ] expectation isn't suppressing it.

Myles, since these tests aren't expected to pass on macOS, is there any reason we shouldn't just skip them?

[Hercules Hjalmarsson]

Pull request: https://github.com/WebKit/WebKit/pull/1911

[Robert Jenner]

(In reply to Hercules Hjalmarsson from comment #6)
> Pull request: https://github.com/WebKit/WebKit/pull/1911

In this Pull Request, we have opted to [ Skip ] these tests for all platforms except iOS. This was because the previously set expectations mentioned that Accessibility Bold only exists on iOS. So in short, we've skipped them for all platforms. Then for iOS we re-enabled them by adding expectations.

This one is passing for iOS, so we set the expectation to [ Pass ] for this one:
fast/text/accessibility-bold-system-font/accessibility-bold-system-font-3.html


The initial test mentioned in this bug is a constant Image Failure for iOS, so we have marked the following test as [ Pass ImageOnlyFailure ] for iOS:
fast/text/accessibility-bold-system-font/accessibility-bold-system-font-2.html


The final test is a text failure, and has been marked as [ Pass Failure ] for iOS:
fast/text/accessibility-bold-system-font/accessibility-bold-system-font.html 

This is the DIFF for that text failure:
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/251945@main%20(3029)/fast/text/accessibility-bold-system-font/accessibility-bold-system-font-pretty-diff.html

-This test makes sure that text using the accessibility bold functionality has the correct weight.
-
-On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
-
-
-PASS window.getComputedStyle(target).getPropertyValue('font-weight') is "400"
-PASS successfullyParsed is true
-
-TEST COMPLETE
-Hello
+CONSOLE MESSAGE: ReferenceError: Can't find variable: description
+layer at (0,0) size 800x600
+  RenderView at (0,0) size 800x600
+layer at (0,0) size 800x98
+  RenderBlock {HTML} at (0,0) size 800x98
+    RenderBody {BODY} at (8,8) size 784x82
+      RenderBlock {DIV} at (0,0) size 784x82
+        RenderText {#text} at (0,0) size 156x82
+          text run at (0,0) width 156: "Hello"

[EWS]

Test gardening commit 251963@main (bd97a03885f9): <https://commits.webkit.org/251963@main>

Reviewed commits have been landed. Closing PR #1911 and removing active labels.

[Myles C. Maxfield]

*** Bug 242139 has been marked as a duplicate of this bug. ***

[Myles C. Maxfield]

Pull request: https://github.com/WebKit/WebKit/pull/1938

[Myles C. Maxfield]

(In reply to Ryan Haddad from comment #5)
> Myles, since these tests aren't expected to pass on macOS, is there any
> reason we shouldn't just skip them?

In general, it's better to mark flakey rather than skip, because flakey at least makes sure the tests don't crash.

[EWS]

Test gardening commit 251984@main (fa781327fe86): <https://commits.webkit.org/251984@main>

Reviewed commits have been landed. Closing PR #1938 and removing active labels.