Bug 241205 - REGRESSION (250836@main): [ iOS ] fast/forms/textfield-outline.html is a flaky failure
Summary: REGRESSION (250836@main): [ iOS ] fast/forms/textfield-outline.html is a flak...
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Karl Rackler
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2022-06-01 16:57 PDT by Karl Rackler
Modified: 2022-07-18 13:02 PDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Karl Rackler 2022-06-01 16:57:27 PDT
Description:
fast/forms/textfield-outline.html

The first failure that I saw on the dashboard was on 5/20/2022 at 250788@main.

REPRODUCTION STEPS
I can reproduce this on 250778@main, but unable to reproduce it on 250777@main or earlier.

Command: 
run-webkit-tests --ios-simulator --debug --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 --no-retry fast/forms/textfield-outline.html

Result: 
Unexpected flakiness: text-only failures (1)
  fast/forms/textfield-outline.html [ Failure Pass ]

History:
https://results.webkit.org/?suite=layout-tests&test=fast%2Fforms%2Ftextfield-outline.html&platform=ios&limit=50000

Diff:
--- /Volumes/Data/worker/Apple-iOS-15-Simulator-Debug-WK2-Tests/build/layout-test-results/fast/forms/textfield-outline-expected.txt
+++ /Volumes/Data/worker/Apple-iOS-15-Simulator-Debug-WK2-Tests/build/layout-test-results/fast/forms/textfield-outline-actual.txt
@@ -8,7 +8,7 @@
       RenderBR {BR} at (562,0) size 1x19
       RenderTextControl {INPUT} at (2,22) size 265x37 [bgcolor=#FFFFFF] [border: (1px solid #000000)]
       RenderText {#text} at (0,0) size 0x0
-layer at (21,35) size 243x25 scrollX 268 scrollWidth 511
+layer at (21,35) size 243x25 scrollWidth 511
   RenderBlock {DIV} at (11,5) size 243x25
     RenderText {#text} at (0,0) size 509x25
       text run at (0,0) width 509: "This tests that typing doesn't cut holes in the focus outline"
Comment 1 Radar WebKit Bug Importer 2022-06-01 16:57:46 PDT
<rdar://problem/94255807>
Comment 2 Karl Rackler 2022-06-01 17:14:25 PDT
250778@main caused the flaky failure.
Reverted at 250801@main and test passing again.
250836@main caused the flaky failure again.
Comment 3 Karl Rackler 2022-06-01 17:28:31 PDT
REPRODUCTION STEPS
I can reproduce this on 250836@main but cannot reproduce it on 250835@main or back earlier to 250801@main.

Command: 
run-webkit-tests --ios-simulator --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 --no-retry fast/forms/textfield-outline.html

Result: 
Regressions: Unexpected text-only failures (1)
  fast/forms/textfield-outline.html [ Failure ]
Comment 4 Karl Rackler 2022-06-02 06:49:39 PDT
I have marked this test as a flaky failure while this issue is investigated.
Comment 5 Karl Rackler 2022-06-02 06:53:29 PDT
Pull request: https://github.com/WebKit/WebKit/pull/1240
Comment 6 EWS 2022-06-02 06:57:19 PDT
Test gardening commit r295126 (251217@main): <https://commits.webkit.org/251217@main>

Reviewed commits have been landed. Closing PR #1240 and removing active labels.
Comment 7 Ryan Reno 2022-06-28 12:49:55 PDT
I've been able to reproduce this locally. It's consistently the same diff - the scrollX is missing from the layer. Even though it's flaky it at least fails in the same way every time.
Comment 8 Simon Fraser (smfr) 2022-07-06 11:52:34 PDT
Looks like the timer-based scroll on focus() is racing with test completion.
Comment 9 Simon Fraser (smfr) 2022-07-06 11:53:03 PDT
See scheduleScrollToFocusedElement().
Comment 10 Ryan Reno 2022-07-12 14:36:27 PDT
Pull request: https://github.com/WebKit/WebKit/pull/2339
Comment 11 EWS 2022-07-13 09:44:40 PDT
Committed 252417@main (2e8bcf908a7d): <https://commits.webkit.org/252417@main>

Reviewed commits have been landed. Closing PR #2339 and removing active labels.
Comment 12 Hercules Hjalmarsson 2022-07-15 14:14:08 PDT
Re-opening bug. It looks like after landing 252417@main all the tests are passing consistently except on iOS15 wk2 Release arm64 that is a flaky text failure.

DIFF:
--- /Volumes/Data/Builds/iOS/-/Production/testbuild252512@main/layout-test-results/fast/forms/textfield-outline-expected.txt
+++ /Volumes/Data/Builds/iOS/-/Production/testbuild252512@main/layout-test-results/fast/forms/textfield-outline-actual.txt
@@ -8,7 +8,7 @@
       RenderBR {BR} at (562,0) size 1x19
       RenderTextControl {INPUT} at (2,22) size 265x37 [bgcolor=#FFFFFF] [border: (1px solid #000000)]
       RenderText {#text} at (0,0) size 0x0
-layer at (21,35) size 243x25 scrollX 268 scrollWidth 511
+layer at (21,35) size 243x25 scrollWidth 511
   RenderBlock {DIV} at (11,5) size 243x25
     RenderText {#text} at (0,0) size 509x25
       text run at (0,0) width 509: "This tests that typing doesn't cut holes in the focus outline"

COMMAND:
(Using arm64 machine)

run-webkit-tests --ios-simulator --iterations=50 fast/forms/textfield-outline.html

RESULT:

~18% failure rate.

Unexpected flakiness: text-only failures (1)
  fast/forms/textfield-outline.html [ Failure Missing Pass ]
Comment 13 Hercules Hjalmarsson 2022-07-15 14:16:21 PDT
Test ran on iOS ToT.
Comment 14 Hercules Hjalmarsson 2022-07-18 13:01:38 PDT
Set expectation. Flaky text failure is now only happening on iOS15 wk2 Release arm64.
Comment 15 EWS 2022-07-18 13:02:06 PDT
Test gardening commit 252575@main (945df646018d): <https://commits.webkit.org/252575@main>

Reviewed commits have been landed. Closing PR #2516 and removing active labels.