Bug 277989 – [ iOS ] 3x imported/w3c/web-platform-tests/mathml/presentation-markup/* (layout-tests) are flaky failures

• Status: RESOLVED
• Resolution: DUPLICATE
• Component: MathML

[Dawn Morningstar]

imported/w3c/web-platform-tests/mathml/presentation-markup/mrow/stretch-along-block-axis-001.html
imported/w3c/web-platform-tests/mathml/presentation-markup/operators/mo-minsize-maxsize-001.html
imported/w3c/web-platform-tests/mathml/presentation-markup/operators/mo-stretch-properties-dynamic-001.html
are all flaky text failures after the import at 282098@main

Since these were 

History:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fmathml%2Fpresentation-markup%2Fmrow%2Fstretch-along-block-axis-001.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fmathml%2Fpresentation-markup%2Foperators%2Fmo-minsize-maxsize-001.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fmathml%2Fpresentation-markup%2Foperators%2Fmo-stretch-properties-dynamic-001.html

Diff:
--- /Volumes/Data/worker/Apple-iOS-17-Simulator-Release-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/mathml/presentation-markup/mrow/stretch-along-block-axis-001-expected.txt
+++ /Volumes/Data/worker/Apple-iOS-17-Simulator-Release-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/mathml/presentation-markup/mrow/stretch-along-block-axis-001-actual.txt
@@ -1,5 +1,5 @@
 
-PASS Taking into account stretch axis.
+FAIL Taking into account stretch axis. assert_approx_equals: vertical characters stretch vertically expected 150 +/- 2 but got 25
 PASS Taking into account stretchy property.
 FAIL Only operators with a stretchy property and block stretch axis. assert_approx_equals: small width expected 12.5 +/- 2 but got 19.4375
 FAIL Operators smaller and larger than non-stretchy siblings. assert_approx_equals: small width expected 12.5 +/- 2 but got 19.4375

Diff-url:
https://build.webkit.org/results/Apple-iOS-17-Simulator-Release-WK2-Tests/282130@main%20(4801)/imported/w3c/web-platform-tests/mathml/presentation-markup/mrow/stretch-along-block-axis-001-diff.txt

Test-run:
https://build.webkit.org/results/Apple-iOS-17-Simulator-Release-WK2-Tests/282130@main%20(4801)/results.html

[Dawn Morningstar]

These tests can't be rebaslined since they're flaky, they will likely need test development.

[Radar WebKit Bug Importer]

<rdar://problem/133722520>

[Dawn Morningstar]

Pull request: https://github.com/WebKit/WebKit/pull/32079

[EWS]

Test gardening commit 282147@main (0b004a3d129f): <https://commits.webkit.org/282147@main>

Reviewed commits have been landed. Closing PR #32079 and removing active labels.

[Alexey Proskuryakov]

It is strange for a mostly static text rendering test to be flaky, I wonder if there is some state leaking from preceding tests, or an actual WebKit bug.

[Dawn Morningstar]

(In reply to Alexey Proskuryakov from comment #5)
> It is strange for a mostly static text rendering test to be flaky, I wonder
> if there is some state leaking from preceding tests, or an actual WebKit bug.

Attempting reproduction to see if I can gather more insight

[Dawn Morningstar]

I attempted to reproduce with 10000 iterations;
run-webkit-tests --ios-simulator --root /Volumes/Data/Spades/Ios/Crystal/buildtoTest imported/w3c/web-platform-tests/mathml/presentation-markup/mrow/stretch-along-block-axis-001.html --iterations 10000 --force --child-processes 1 

This failed to reproduce, and I think that Alexey’s theory about leaking from preceding tests holds water.
I'll also try using a test-list with the preceding tests and see if that works as well.

[Alexey Proskuryakov]

I see a number of older (2021 or earlier) bugs about flaky MathML tests, some also about mathml/presentation-markup. Some of these could be related, hard to tell without knowing the root cause.

[Frédéric Wang (:fredw)]

Not sure that's the reason but MathML tests rely a lot of web fonts and need to wait for them to be ready. document.fonts.ready has issues in WebKit, so https://searchfox.org/wubkat/source/LayoutTests/imported/w3c/web-platform-tests/mathml/support/fonts.js was introduced to work around that. It's possible that some tests would need to use that helper function in order to avoid flakiness in WebKit.

[Alexey Proskuryakov]

This is a good theory, however the tests in question have it already:

<script src="/mathml/support/fonts.js"></script>

[Matt Woodrow]

*** This bug has been marked as a duplicate of bug 278933 ***