Bug 195635 – [macOS WK2] REGRESSION (r242313): Layout Test scrollingcoordinator/mac/multiple-fixed.html is a flaky time out and image diff

• Status: REOPENED
• Component: Tools / Tests

[Shawn Roberts]

The following layout test is failing on Mac WK2 and Mac WK2 Debug

scrollingcoordinator/mac/multiple-fixed.html

Probable cause:

Test is failing as of https://trac.webkit.org/changeset/242313/webkit . Ran tests locally with newest revision 242800, and r242313 and got same timeout, image diff.
Test is failing locally with debug revision 242805 as well.

Reproduced with :

run-webkit-tests scrollingcoordinator/mac/multiple-fixed.html --iterations 500 -f --no-retry
run-webkit-tests scrollingcoordinator/mac/multiple-fixed.html -f --iterations 500 --no-retry --debug

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=scrollingcoordinator%2Fmac%2Fmultiple-fixed.html

Image Diff: 

https://build.webkit.org/results/Apple%20Mojave%20Debug%20WK2%20(Tests)/r242560%20(1841)/scrollingcoordinator/mac/multiple-fixed-diffs.html

[Radar WebKit Bug Importer]

<rdar://problem/48818269>

[Radar WebKit Bug Importer]

<rdar://problem/48818274>

[Simon Fraser (smfr)]

Image makes it look like we're catching it in a rubber-band.

[Truitt Savell]

marked test as timeout in https://trac.webkit.org/changeset/246806/webkit while waiting on fix

[Karl Rackler]

[ macOS wk2 arm64 ] scrollingcoordinator/mac/multiple-fixed.html is a flaky image failure

Description:
scrollingcoordinator/mac/multiple-fixed.html

This is affecting macOS wk2 arm64 only now.  This may still have to do with the originally outlined regression.  It is difficult to bisect due to the lack of spades around the regression time frame.  This test seems to have been flaky for the viewable dashboard history, first on Big Sur and then on Monterey.

REPRODUCTION STEPS
I can reproduce this on ToT r293167.

Command: 
run-webkit-tests --root 293167  scrollingcoordinator/mac/multiple-fixed.html --iterations 500 -f --no-retry

Result:   
Unexpected flakiness: image-only failures (1)
  scrollingcoordinator/mac/multiple-fixed.html [ ImageOnlyFailure Pass ]  

History:
https://results.webkit.org/?suite=layout-tests&test=scrollingcoordinator%2Fmac%2Fmultiple-fixed.html&limit=50000

Image Diff link:
https://build.webkit.org/results/Apple-Monterey-Release-AppleSilicon-WK2-Tests/249850@main%20(2806)/scrollingcoordinator/mac/multiple-fixed-diffs.html

I have marked this test as a flaky image failure while this issue is investigated.

[Karl Rackler]

Pull request: https://github.com/WebKit/WebKit/pull/350

[EWS]

Committed r293192 (249867@main): <https://commits.webkit.org/249867@main>

Reviewed commits have been landed. Closing PR #350 and removing active labels.

[Karl Rackler]

Reopened because the issue is still being investigated.

[Simon Fraser (smfr)]

Looks like the test is rubber banding at the bottom.