Bug 228672 – [ iOS ] fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html is a flaky timeout

• Component: Scrolling
• Status: RESOLVED FIXED

fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html

Is flaky timing out on iOS 14 E Simulator wk2.

History: https://results.webkit.org/?platform=ios&suite=layout-tests&test=fast%2Fscrolling%2Fios%2Fclick-events-during-momentum-scroll-in-overflow-after-tap-on-body.html

Diff: 

--- /Volumes/Data/worker/ios-simulator-14-release-tests-wk2/build/layout-test-results/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body-expected.txt
+++ /Volumes/Data/worker/ios-simulator-14-release-tests-wk2/build/layout-test-results/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body-actual.txt
@@ -1,11 +1,4 @@
-Verifies that tapping outside of a subscrollable region during momentum scrolling dispatches click events. To run the test manually, swipe up on the black box to scroll it; while scrolling, tap outside of the scrollable area. The page should observe click events.
+#PID UNRESPONSIVE - WebKitTestRunnerApp (pid 35478)
+FAIL: Timed out waiting for notifyDone to be called
 
-On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
-
-
-PASS scroller.scrollTop passed 400
-PASS Observed click event.
-PASS successfullyParsed is true
-
-TEST COMPLETE
-
+#EOF

[Radar WebKit Bug Importer]

<rdar://problem/81348960>

I was not able to reproduce the timeout, but instead it failed on iOS 15 simulator on my local machine using: 

run-webkit-tests --ios-simulator --exit-after-n-failures 2 --exit-after-n-crashes-or-timeouts 2 --iterations 1000 fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html

Diff:

--- /Volumes/Data/Builds/iOS-builds/buildToTest/layout-test-results/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body-expected.txt
+++ /Volumes/Data/Builds/iOS-builds/buildToTest/layout-test-results/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body-actual.txt
@@ -3,8 +3,8 @@
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
+PASS Observed click event.
 PASS scroller.scrollTop passed 400
-PASS Observed click event.
 PASS successfullyParsed is true
 
 TEST COMPLETE

Updated test expectations https://trac.webkit.org/changeset/280532/webkit

[Wenson Hsieh]

(In reply to ayumi_kojima from comment #2)
> I was not able to reproduce the timeout, but instead it failed on iOS 15
> simulator on my local machine using: 
> 
> run-webkit-tests --ios-simulator --exit-after-n-failures 2
> --exit-after-n-crashes-or-timeouts 2 --iterations 1000
> fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-
> on-body.html
> 
> Diff:
> 
> ---
> /Volumes/Data/Builds/iOS-builds/buildToTest/layout-test-results/fast/
> scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-
> body-expected.txt
> +++
> /Volumes/Data/Builds/iOS-builds/buildToTest/layout-test-results/fast/
> scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-
> body-actual.txt
> @@ -3,8 +3,8 @@
>  On success, you will see a series of "PASS" messages, followed by "TEST
> COMPLETE".
>  
>  
> +PASS Observed click event.
>  PASS scroller.scrollTop passed 400
> -PASS Observed click event.
>  PASS successfullyParsed is true
>  
>  TEST COMPLETE

This looks like it's _probably_ a different issue, though it's hard to tell without more data for the case where it times out :/

[Wenson Hsieh]

Created attachment 434764 [details]
EWS run

[Wenson Hsieh]

(In reply to Wenson Hsieh from comment #5)
> Created attachment 434764 [details]
> EWS run

Results in https://ews-build.webkit.org/#/builders/51/builds/18493/steps/15/logs/stdio:

> 12:58:32.769 3089 Testing completed, Exit status: 0
> => Results: 50459/50470 tests passed (100.0%)
> => Tests to be fixed (23052):
>       7 timeouts                 ( 0.0%)
>       2 image-only failures      ( 0.0%)
> => Tests that will only be fixed if they crash (WONTFIX) (218):
> Unexpected flakiness: text-only failures (2)
>   http/tests/xmlhttprequest/resetting-timeout-to-zero.html [ Failure Pass ]
>   webrtc/peer-connection-audio-mute2.html [ Failure Pass ]
> Unexpected flakiness: image-only failures (2)
>   css3/filters/backdrop/backdrop-filter-with-border-radius-and-reflection-remove.html [ ImageOnlyFailure Pass ]
>   security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html [ ImageOnlyFailure Pass ]
> Unexpected flakiness: timeouts (7)
>   fast/scrolling/ios/cpd-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]
>   fast/scrolling/ios/ppw-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]
>   fast/scrolling/ios/von-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]
>   http/wpt/service-workers/persistent-modules.html [ Timeout Pass ]
>   imported/w3c/web-platform-tests/FileAPI/url/url-in-tags.window.html [ Timeout Pass ]
>   imported/w3c/web-platform-tests/clipboard-apis/feature-policy/clipboard-write/clipboard-write-enabled-by-feature-policy-attribute-cross-origin-tentative.https.sub.html [ Timeout Pass ]
>   imported/w3c/web-platform-tests/css/css-fonts/idlharness.html [ Timeout Pass ]

Repros about 3/1000 times — we should be able to use EWS to gather more data.

[Wenson Hsieh]

Created attachment 435300 [details]
Try to repro w/ logs (x1500)

[Wenson Hsieh]

(In reply to Wenson Hsieh from comment #7)
> Created attachment 435300 [details]
> Try to repro w/ logs (x1500)

This caught the bug three times:

> Unexpected flakiness: timeouts (3)
>   fast/scrolling/ios/aix-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]
>   fast/scrolling/ios/kpc-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]
>   fast/scrolling/ios/uvy-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]

Output from all three failures indicates that we're failing because we're...just not scrolling far enough to cross the threshold.

20:49:09.397 15295 worker/2 worker/2 fast/scrolling/ios/kpc-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html output stderr lines:
20:49:09.401 15295 worker/2   (CONSOLE) <WK>: 1a. Simulating drag
20:49:09.402 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 48
20:49:09.402 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 72
20:49:09.402 15295 worker/2   (CONSOLE) <WK>: 1b. Done simulating drag
20:49:09.403 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 102
20:49:09.403 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 111
20:49:09.403 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 120
20:49:09.403 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 128
20:49:09.404 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 136
20:49:09.404 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 144

    …(more scroll events)…

20:49:09.424 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 357
20:49:09.424 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 358
20:49:09.424 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 359
20:49:09.425 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 360
20:49:09.425 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 361
20:49:09.425 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 362
20:49:09.425 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 363
20:49:09.426 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 364
20:49:09.426 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 365
20:49:09.426 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 366
20:49:09.427 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 367
20:49:09.971 15295 worker/2 "ruby -I /Volumes/Data/worker/iOS-14-Simulator-WK2-Tests-EWS/build/Websites/bugs.webkit.org/PrettyPatch /Volumes/Data/worker/iOS-14-Simulator-WK2-Tests-EWS/build/Websites/bugs.webkit.org/PrettyPatch/prettify.rb /Volumes/Data/worker/iOS-14-Simulator-WK2-Tests-EWS/build/layout-test-results/fast/scrolling/ios/kpc-click-events-during-momentum-scroll-in-overflow-after-tap-on-body-diff.txt" took 0.54s
20:49:09.975 15295 worker/2 killing driver
20:49:10.125 15295 worker/2 fast/scrolling/ios/kpc-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html failed:
20:49:10.125 15295 worker/2 test timed out
20:49:10.126 15295 worker/2 text diff

[Wenson Hsieh]

Created attachment 435356 [details]
Test a fix

[Wenson Hsieh]

(In reply to Wenson Hsieh from comment #9)
> Created attachment 435356 [details]
> Test a fix

All 1000 runs of the new version of `fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html` passed. Looks like that fixes it!

[Wenson Hsieh]

Created attachment 435372 [details]
Patch

[Simon Fraser (smfr)]

Comment on attachment 435372 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=435372&action=review

> LayoutTests/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html:65
> +        await UIHelper.delayFor(200);

Don't we have a way to wait for scrolling to finish on iOS?

[Wenson Hsieh]

(In reply to Simon Fraser (smfr) from comment #12)
> Comment on attachment 435372 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=435372&action=review
> 
> > LayoutTests/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html:65
> > +        await UIHelper.delayFor(200);
> 
> Don't we have a way to wait for scrolling to finish on iOS?

It's possible to wait for a stable state update, but this test is intended specifically to tap while momentum scrolling is still ongoing.

[EWS]

Committed r280937 (240454@main): <https://commits.webkit.org/240454@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 435372 [details].