Bug 228672

Summary: [ iOS ] fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html is a flaky timeout
Product: WebKit Reporter: ayumi_kojima
Component: ScrollingAssignee: Wenson Hsieh <wenson_hsieh>
Status: RESOLVED FIXED    
Severity: Normal CC: simon.fraser, thorton, webkit-bot-watchers-bugzilla, webkit-bug-importer, wenson_hsieh
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: iPhone / iPad   
OS: Unspecified   
Attachments:
Description Flags
EWS run
none
Try to repro w/ logs (x1500)
ews-feeder: commit-queue-
Test a fix
none
Patch none

Description ayumi_kojima 2021-07-30 17:04:43 PDT
fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html

Is flaky timing out on iOS 14 E Simulator wk2.

History: https://results.webkit.org/?platform=ios&suite=layout-tests&test=fast%2Fscrolling%2Fios%2Fclick-events-during-momentum-scroll-in-overflow-after-tap-on-body.html

Diff: 

--- /Volumes/Data/worker/ios-simulator-14-release-tests-wk2/build/layout-test-results/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body-expected.txt
+++ /Volumes/Data/worker/ios-simulator-14-release-tests-wk2/build/layout-test-results/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body-actual.txt
@@ -1,11 +1,4 @@
-Verifies that tapping outside of a subscrollable region during momentum scrolling dispatches click events. To run the test manually, swipe up on the black box to scroll it; while scrolling, tap outside of the scrollable area. The page should observe click events.
+#PID UNRESPONSIVE - WebKitTestRunnerApp (pid 35478)
+FAIL: Timed out waiting for notifyDone to be called
 
-On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
-
-
-PASS scroller.scrollTop passed 400
-PASS Observed click event.
-PASS successfullyParsed is true
-
-TEST COMPLETE
-
+#EOF
Comment 1 Radar WebKit Bug Importer 2021-07-30 17:05:47 PDT
<rdar://problem/81348960>
Comment 2 ayumi_kojima 2021-07-30 17:16:51 PDT
I was not able to reproduce the timeout, but instead it failed on iOS 15 simulator on my local machine using: 

run-webkit-tests --ios-simulator --exit-after-n-failures 2 --exit-after-n-crashes-or-timeouts 2 --iterations 1000 fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html

Diff:

--- /Volumes/Data/Builds/iOS-builds/buildToTest/layout-test-results/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body-expected.txt
+++ /Volumes/Data/Builds/iOS-builds/buildToTest/layout-test-results/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body-actual.txt
@@ -3,8 +3,8 @@
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
+PASS Observed click event.
 PASS scroller.scrollTop passed 400
-PASS Observed click event.
 PASS successfullyParsed is true
 
 TEST COMPLETE
Comment 3 ayumi_kojima 2021-08-02 08:59:30 PDT
Updated test expectations https://trac.webkit.org/changeset/280532/webkit
Comment 4 Wenson Hsieh 2021-08-02 09:50:48 PDT
(In reply to ayumi_kojima from comment #2)
> I was not able to reproduce the timeout, but instead it failed on iOS 15
> simulator on my local machine using: 
> 
> run-webkit-tests --ios-simulator --exit-after-n-failures 2
> --exit-after-n-crashes-or-timeouts 2 --iterations 1000
> fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-
> on-body.html
> 
> Diff:
> 
> ---
> /Volumes/Data/Builds/iOS-builds/buildToTest/layout-test-results/fast/
> scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-
> body-expected.txt
> +++
> /Volumes/Data/Builds/iOS-builds/buildToTest/layout-test-results/fast/
> scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-
> body-actual.txt
> @@ -3,8 +3,8 @@
>  On success, you will see a series of "PASS" messages, followed by "TEST
> COMPLETE".
>  
>  
> +PASS Observed click event.
>  PASS scroller.scrollTop passed 400
> -PASS Observed click event.
>  PASS successfullyParsed is true
>  
>  TEST COMPLETE

This looks like it's _probably_ a different issue, though it's hard to tell without more data for the case where it times out :/
Comment 5 Wenson Hsieh 2021-08-02 09:55:26 PDT
Created attachment 434764 [details]
EWS run
Comment 6 Wenson Hsieh 2021-08-10 09:01:04 PDT
(In reply to Wenson Hsieh from comment #5)
> Created attachment 434764 [details]
> EWS run

Results in https://ews-build.webkit.org/#/builders/51/builds/18493/steps/15/logs/stdio:

> 12:58:32.769 3089 Testing completed, Exit status: 0
> => Results: 50459/50470 tests passed (100.0%)
> => Tests to be fixed (23052):
>       7 timeouts                 ( 0.0%)
>       2 image-only failures      ( 0.0%)
> => Tests that will only be fixed if they crash (WONTFIX) (218):
> Unexpected flakiness: text-only failures (2)
>   http/tests/xmlhttprequest/resetting-timeout-to-zero.html [ Failure Pass ]
>   webrtc/peer-connection-audio-mute2.html [ Failure Pass ]
> Unexpected flakiness: image-only failures (2)
>   css3/filters/backdrop/backdrop-filter-with-border-radius-and-reflection-remove.html [ ImageOnlyFailure Pass ]
>   security/contentSecurityPolicy/video-with-file-url-allowed-by-media-src-star-with-AllowContentSecurityPolicySourceStarToMatchAnyProtocol-enabled.html [ ImageOnlyFailure Pass ]
> Unexpected flakiness: timeouts (7)
>   fast/scrolling/ios/cpd-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]
>   fast/scrolling/ios/ppw-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]
>   fast/scrolling/ios/von-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]
>   http/wpt/service-workers/persistent-modules.html [ Timeout Pass ]
>   imported/w3c/web-platform-tests/FileAPI/url/url-in-tags.window.html [ Timeout Pass ]
>   imported/w3c/web-platform-tests/clipboard-apis/feature-policy/clipboard-write/clipboard-write-enabled-by-feature-policy-attribute-cross-origin-tentative.https.sub.html [ Timeout Pass ]
>   imported/w3c/web-platform-tests/css/css-fonts/idlharness.html [ Timeout Pass ]

Repros about 3/1000 times — we should be able to use EWS to gather more data.
Comment 7 Wenson Hsieh 2021-08-10 15:05:49 PDT
Created attachment 435300 [details]
Try to repro w/ logs (x1500)
Comment 8 Wenson Hsieh 2021-08-11 08:19:17 PDT
(In reply to Wenson Hsieh from comment #7)
> Created attachment 435300 [details]
> Try to repro w/ logs (x1500)

This caught the bug three times:

> Unexpected flakiness: timeouts (3)
>   fast/scrolling/ios/aix-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]
>   fast/scrolling/ios/kpc-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]
>   fast/scrolling/ios/uvy-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html [ Timeout Pass ]

Output from all three failures indicates that we're failing because we're...just not scrolling far enough to cross the threshold.

20:49:09.397 15295 worker/2 worker/2 fast/scrolling/ios/kpc-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html output stderr lines:
20:49:09.401 15295 worker/2   (CONSOLE) <WK>: 1a. Simulating drag
20:49:09.402 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 48
20:49:09.402 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 72
20:49:09.402 15295 worker/2   (CONSOLE) <WK>: 1b. Done simulating drag
20:49:09.403 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 102
20:49:09.403 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 111
20:49:09.403 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 120
20:49:09.403 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 128
20:49:09.404 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 136
20:49:09.404 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 144

    …(more scroll events)…

20:49:09.424 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 357
20:49:09.424 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 358
20:49:09.424 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 359
20:49:09.425 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 360
20:49:09.425 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 361
20:49:09.425 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 362
20:49:09.425 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 363
20:49:09.426 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 364
20:49:09.426 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 365
20:49:09.426 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 366
20:49:09.427 15295 worker/2   (CONSOLE) <WK>:   Scrolled to 367
20:49:09.971 15295 worker/2 "ruby -I /Volumes/Data/worker/iOS-14-Simulator-WK2-Tests-EWS/build/Websites/bugs.webkit.org/PrettyPatch /Volumes/Data/worker/iOS-14-Simulator-WK2-Tests-EWS/build/Websites/bugs.webkit.org/PrettyPatch/prettify.rb /Volumes/Data/worker/iOS-14-Simulator-WK2-Tests-EWS/build/layout-test-results/fast/scrolling/ios/kpc-click-events-during-momentum-scroll-in-overflow-after-tap-on-body-diff.txt" took 0.54s
20:49:09.975 15295 worker/2 killing driver
20:49:10.125 15295 worker/2 fast/scrolling/ios/kpc-click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html failed:
20:49:10.125 15295 worker/2 test timed out
20:49:10.126 15295 worker/2 text diff
Comment 9 Wenson Hsieh 2021-08-11 09:24:51 PDT
Created attachment 435356 [details]
Test a fix
Comment 10 Wenson Hsieh 2021-08-11 13:49:24 PDT
(In reply to Wenson Hsieh from comment #9)
> Created attachment 435356 [details]
> Test a fix

All 1000 runs of the new version of `fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html` passed. Looks like that fixes it!
Comment 11 Wenson Hsieh 2021-08-11 13:51:28 PDT
Created attachment 435372 [details]
Patch
Comment 12 Simon Fraser (smfr) 2021-08-11 13:53:53 PDT
Comment on attachment 435372 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=435372&action=review

> LayoutTests/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html:65
> +        await UIHelper.delayFor(200);

Don't we have a way to wait for scrolling to finish on iOS?
Comment 13 Wenson Hsieh 2021-08-11 13:58:20 PDT
(In reply to Simon Fraser (smfr) from comment #12)
> Comment on attachment 435372 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=435372&action=review
> 
> > LayoutTests/fast/scrolling/ios/click-events-during-momentum-scroll-in-overflow-after-tap-on-body.html:65
> > +        await UIHelper.delayFor(200);
> 
> Don't we have a way to wait for scrolling to finish on iOS?

It's possible to wait for a stable state update, but this test is intended specifically to tap while momentum scrolling is still ongoing.
Comment 14 EWS 2021-08-11 16:51:16 PDT
Committed r280937 (240454@main): <https://commits.webkit.org/240454@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 435372 [details].