Bug 204045 – iOS: fast/events/scroll-subframe-in-rendering-update.html always fails

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Aakash Jain]

The following layout test is failing consistently on iOS Simulator Debug/Release: fast/events/scroll-subframe-in-rendering-update.html

Flakiness Dashboard:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=fast%2Fevents%2Fscroll-subframe-in-rendering-update.html
https://results.webkit.org/?suite=layout-tests&test=fast%2Fevents%2Fscroll-subframe-in-rendering-update.html


e.g.:

https://build.webkit.org/results/Apple%20iOS%2013%20Simulator%20Release%20WK2%20(Tests)/r252205%20(939)/results.html
https://build.webkit.org/results/Apple%20iOS%2013%20Simulator%20Release%20WK2%20(Tests)/r252231%20(947)/results.html
https://build.webkit.org/results/Apple%20iOS%2013%20Simulator%20Release%20WK2%20(Tests)/r252240%20(954)/results.html
https://build.webkit.org/results/Apple%20iOS%2013%20Simulator%20Release%20WK2%20(Tests)/r252262%20(960)/results.html
https://build.webkit.org/results/Apple%20iOS%2013%20Simulator%20Debug%20WK2%20(Tests)/r252299%20(810)/results.html
https://build.webkit.org/results/Apple%20iOS%2013%20Simulator%20Debug%20WK2%20(Tests)/r252205%20(788)/results.html
https://ews-build.webkit.org/results/iOS-13-Simulator-WK2-Tests-EWS/r383072-4464/results.html

[Aakash Jain]

Diff:
--- /Volumes/Data/slave/ios-simulator-13-release-tests-wk2/build/layout-test-results/fast/events/scroll-subframe-in-rendering-update-expected.txt
+++ /Volumes/Data/slave/ios-simulator-13-release-tests-wk2/build/layout-test-results/fast/events/scroll-subframe-in-rendering-update-actual.txt
@@ -8,9 +8,10 @@
 iframeAA.contentWindow.scrollTo(0, 1000); updateLayout(iframeAA)
 PASS logs.length is 0
 After requestAnimationFrame
-PASS logs.length is 3
-PASS logs.join(", ") is "A, AA, B"
+FAIL logs.length should be 3. Was 0.
+FAIL logs.join(", ") should be A, AA, B. Was .
 PASS successfullyParsed is true
+Some tests failed.
 
 TEST COMPLETE

[Radar WebKit Bug Importer]

<rdar://problem/57047296>

[Radar WebKit Bug Importer]

<rdar://problem/57047315>

[Aakash Jain]

This test was added 2 days back in http://trac.webkit.org/changeset/252205/webkit and has been failing on iOS since then.

[Ryosuke Niwa]

Ugh... frame flattening is in effect here.

[Ryosuke Niwa]

Created attachment 383229 [details]
Disable frame flattening

[Ryosuke Niwa]

Ugh... I need to download & install a new iOS SDK to test this out. Doing that now.

[Ryosuke Niwa]

Committed r252311: <https://trac.webkit.org/changeset/252311>

[Ryosuke Niwa]

Ugh... I can't either SSH into my machine or download the iOS SDK. Landing the fix speculatively for now. If this doesn't work, we probably need to skip the test on iOS for now until I regain access to a machine with functional iOS SDK.

[Aakash Jain]

(In reply to Ryosuke Niwa from comment #8)
> Committed r252311: <https://trac.webkit.org/changeset/252311>
Seems to be still failing. https://build.webkit.org/builders/Apple%20iOS%2013%20Simulator%20Release%20WK2%20%28Tests%29/builds/975

[Ryosuke Niwa]

Oh, I think we probably also need <!-- webkit-test-runner [ internal:AsyncOverflowScrollingEnabled=true ] -->.

[Ryosuke Niwa]

Added the failing expectation in https://trac.webkit.org/r252312 for now.

[Ryosuke Niwa]

Created attachment 383237 [details]
Fix attempt no.2

[Ryosuke Niwa]

It's really annoying that we need to enable random ass flags in the tests to make scrolling tests to work in iOS.

We should probably flip the default so that async scrolling is enabled by default so that many of existing layout tests for scrolling behavior can be enabled on iOS, and create a separate set of tests for frame flattening.

[Aakash Jain]

(In reply to Ryosuke Niwa from comment #13)
> Created attachment 383237 [details]
> Fix attempt no.2
Did you want to update the TestExpectations as well? 

I guess EWS results for this patch wouldn't have covered this test because we pass --skip-failing-tests to run-webkit-tests in EWS.

[Ryosuke Niwa]

(In reply to Aakash Jain from comment #15)
> (In reply to Ryosuke Niwa from comment #13)
> > Created attachment 383237 [details]
> > Fix attempt no.2
> Did you want to update the TestExpectations as well? 
> 
> I guess EWS results for this patch wouldn't have covered this test because
> we pass --skip-failing-tests to run-webkit-tests in EWS.

WTF!? When did we start doing that?

[Ryosuke Niwa]

Created attachment 383251 [details]
Patch for landing

[Ryosuke Niwa]

Comment on attachment 383251 [details]
Patch for landing

Wait for EWS.

[Aakash Jain]

(In reply to Ryosuke Niwa from comment #16)
> (In reply to Aakash Jain from comment #15)
> > (In reply to Ryosuke Niwa from comment #13)
> > > Created attachment 383237 [details]
> > > Fix attempt no.2
> > Did you want to update the TestExpectations as well? 
> > 
> > I guess EWS results for this patch wouldn't have covered this test because
> > we pass --skip-failing-tests to run-webkit-tests in EWS.
> 
> WTF!? When did we start doing that?
Seems like we started it in https://trac.webkit.org/changeset/106518/webkit

[WebKit Commit Bot]

Comment on attachment 383251 [details]
Patch for landing

Clearing flags on attachment: 383251

Committed r252317: <https://trac.webkit.org/changeset/252317>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.