Bug 259157 – REGRESSION(264419@main): [ iOS ] 264419@Main introduced multiple flaky ImageOnlyFailures

• Component: WebKit Process Model
• Status: REOPENED

[Robert Jenner]

imported/w3c/web-platform-tests/html/dom/elements/global-attributes/dir_auto-textarea-script-mixed.html

is a flaky ImageOnlyFailure on iOS 16. 

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fdom%2Felements%2Fglobal-attributes%2Fdir_auto-textarea-script-mixed.html

The Diff shows that it appears to off by .01%
https://build.webkit.org/results/Apple-iOS-16-Simulator-Release-GPUProcess-WK2-Tests/265978@main%20(1887)/imported/w3c/web-platform-tests/html/dom/elements/global-attributes/dir_auto-textarea-script-mixed-diff.png

I have attached the diff files to this bug.

[Robert Jenner]

Created attachment 467027 [details]
Image Diff files

[Radar WebKit Bug Importer]

<rdar://problem/112153465>

[Robert Jenner]

I was able to reproduce the failure at iOS Release ToT running the test as follows:

run-webkit-tests --ios-simulator imported/w3c/web-platform-tests/html/dom/elements/global-attributes/dir_auto-textarea-script-mixed.html --child-process=1 --iterations 10

It doesn't fail every time, but it does consistently fail running the test as I did above.

[Robert Jenner]

Even though this is flaky, I was able to bisect to a regression point. 

This test will consistently fail at 264419@main, but won't fail at 264418@main. I ran the test the exact same way, up-to 1000 iterations, and it did not fail once. Where as at and after 264419@main it will fail roughly every 4 out of 10 times.

[EWS]

Committed 266003@main (1866fb9a9339): <https://commits.webkit.org/266003@main>

Reviewed commits have been landed. Closing PR #15792 and removing active labels.

[Robert Jenner]

I have found three more tests that become flaky ImageOnly Failures starting at 264419@main. They all passed when being run in 1000 iterations each at 264418@main, but became reproducible flaky failures starting at 264418@main. The additional tests found are:


fast/block/transparent-outline-with-and-without-border-radius.html
fast/forms/basic-textareas-quirks-simple-lines.html
imported/w3c/web-platform-tests/html/dom/elements/global-attributes/dir_auto-textarea-mixed.html

Observed at:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&test=fast%2Fblock%2Ftransparent-outline-with-and-without-border-radius.html&test=fast%2Fforms%2Fbasic-textareas-quirks-simple-lines.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fdom%2Felements%2Fglobal-attributes%2Fdir_auto-textarea-mixed.html

[EWS]

Test gardening commit 266014@main (738b85f95555): <https://commits.webkit.org/266014@main>

Reviewed commits have been landed. Closing PR #15799 and removing active labels.

[EWS]

Test gardening commit 265870.59@safari-7616-branch (1a4523f844fb): <https://commits.webkit.org/265870.59@safari-7616-branch>

Reviewed commits have been landed. Closing PR #697 and removing active labels.

[Alexey Proskuryakov]

Re-opening, as only TestExpectations were landed.