Bug 241205 – REGRESSION (250836@main): [ iOS ] fast/forms/textfield-outline.html is a flaky failure

• Component: New Bugs
• Status: RESOLVED FIXED

[Karl Rackler]

Description:
fast/forms/textfield-outline.html

The first failure that I saw on the dashboard was on 5/20/2022 at 250788@main.

REPRODUCTION STEPS
I can reproduce this on 250778@main, but unable to reproduce it on 250777@main or earlier.

Command: 
run-webkit-tests --ios-simulator --debug --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 --no-retry fast/forms/textfield-outline.html

Result: 
Unexpected flakiness: text-only failures (1)
  fast/forms/textfield-outline.html [ Failure Pass ]

History:
https://results.webkit.org/?suite=layout-tests&test=fast%2Fforms%2Ftextfield-outline.html&platform=ios&limit=50000

Diff:
--- /Volumes/Data/worker/Apple-iOS-15-Simulator-Debug-WK2-Tests/build/layout-test-results/fast/forms/textfield-outline-expected.txt
+++ /Volumes/Data/worker/Apple-iOS-15-Simulator-Debug-WK2-Tests/build/layout-test-results/fast/forms/textfield-outline-actual.txt
@@ -8,7 +8,7 @@
       RenderBR {BR} at (562,0) size 1x19
       RenderTextControl {INPUT} at (2,22) size 265x37 [bgcolor=#FFFFFF] [border: (1px solid #000000)]
       RenderText {#text} at (0,0) size 0x0
-layer at (21,35) size 243x25 scrollX 268 scrollWidth 511
+layer at (21,35) size 243x25 scrollWidth 511
   RenderBlock {DIV} at (11,5) size 243x25
     RenderText {#text} at (0,0) size 509x25
       text run at (0,0) width 509: "This tests that typing doesn't cut holes in the focus outline"

[Radar WebKit Bug Importer]

<rdar://problem/94255807>

[Karl Rackler]

250778@main caused the flaky failure.
Reverted at 250801@main and test passing again.
250836@main caused the flaky failure again.

[Karl Rackler]

REPRODUCTION STEPS
I can reproduce this on 250836@main but cannot reproduce it on 250835@main or back earlier to 250801@main.

Command: 
run-webkit-tests --ios-simulator --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 --no-retry fast/forms/textfield-outline.html

Result: 
Regressions: Unexpected text-only failures (1)
  fast/forms/textfield-outline.html [ Failure ]

[Karl Rackler]

I have marked this test as a flaky failure while this issue is investigated.

[Karl Rackler]

Pull request: https://github.com/WebKit/WebKit/pull/1240

[EWS]

Test gardening commit r295126 (251217@main): <https://commits.webkit.org/251217@main>

Reviewed commits have been landed. Closing PR #1240 and removing active labels.

[Ryan Reno]

I've been able to reproduce this locally. It's consistently the same diff - the scrollX is missing from the layer. Even though it's flaky it at least fails in the same way every time.

[Simon Fraser (smfr)]

Looks like the timer-based scroll on focus() is racing with test completion.

[Simon Fraser (smfr)]

See scheduleScrollToFocusedElement().

[Ryan Reno]

Pull request: https://github.com/WebKit/WebKit/pull/2339

[EWS]

Committed 252417@main (2e8bcf908a7d): <https://commits.webkit.org/252417@main>

Reviewed commits have been landed. Closing PR #2339 and removing active labels.

[Hercules Hjalmarsson]

Re-opening bug. It looks like after landing 252417@main all the tests are passing consistently except on iOS15 wk2 Release arm64 that is a flaky text failure.

DIFF:
--- /Volumes/Data/Builds/iOS/-/Production/testbuild252512@main/layout-test-results/fast/forms/textfield-outline-expected.txt
+++ /Volumes/Data/Builds/iOS/-/Production/testbuild252512@main/layout-test-results/fast/forms/textfield-outline-actual.txt
@@ -8,7 +8,7 @@
       RenderBR {BR} at (562,0) size 1x19
       RenderTextControl {INPUT} at (2,22) size 265x37 [bgcolor=#FFFFFF] [border: (1px solid #000000)]
       RenderText {#text} at (0,0) size 0x0
-layer at (21,35) size 243x25 scrollX 268 scrollWidth 511
+layer at (21,35) size 243x25 scrollWidth 511
   RenderBlock {DIV} at (11,5) size 243x25
     RenderText {#text} at (0,0) size 509x25
       text run at (0,0) width 509: "This tests that typing doesn't cut holes in the focus outline"

COMMAND:
(Using arm64 machine)

run-webkit-tests --ios-simulator --iterations=50 fast/forms/textfield-outline.html

RESULT:

~18% failure rate.

Unexpected flakiness: text-only failures (1)
  fast/forms/textfield-outline.html [ Failure Missing Pass ]

[Hercules Hjalmarsson]

Test ran on iOS ToT.

[Hercules Hjalmarsson]

Set expectation. Flaky text failure is now only happening on iOS15 wk2 Release arm64.

[EWS]

Test gardening commit 252575@main (945df646018d): <https://commits.webkit.org/252575@main>

Reviewed commits have been landed. Closing PR #2516 and removing active labels.

[Ryan Reno]

Pull request: https://github.com/WebKit/WebKit/pull/6620

[EWS]

Committed 256856@main (93ece5e17f0a): <https://commits.webkit.org/256856@main>

Reviewed commits have been landed. Closing PR #6620 and removing active labels.