REGRESSION (r244182) [Mac WK2] Layout Test imported/w3c/web-platform-tests/visual-viewport/viewport-resize-event-on-load-overflowing-page.html is a flaky failure

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Shawn Roberts]

The following layout test is flaky on Mac WK2 Release and Debug

imported/w3c/web-platform-tests/visual-viewport/viewport-resize-event-on-load-overflowing-page.html

Probable cause:

It appears changes in https://trac.webkit.org/changeset/244182/webkit have made this test more flaky. 

Local testing on Debug is yielding close to 90% failure in 500 iterations. Testing done on r244181 and older yield on average 1 failure in 500 iterations.

Reproduced with :

run-webkit-tests imported/w3c/web-platform-tests/visual-viewport/viewport-resize-event-on-load-overflowing-page.html --iterations 500 -f --debug --exit-after-n-failures=5

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fvisual-viewport%2Fviewport-resize-event-on-load-overflowing-page.html

Diff:

--- /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/visual-viewport/viewport-resize-event-on-load-overflowing-page-expected.txt
+++ /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/visual-viewport/viewport-resize-event-on-load-overflowing-page-actual.txt
@@ -3,5 +3,5 @@
 Test Description: This test ensures that we fire a resize event against window.visualViewport if the page has overflow (since this creates a scrollbar and thus changes the viewport size).
 
 
-PASS Resize event fired exactly once against window.visualViewport if scrollbars affect layout. 
+FAIL Resize event fired exactly once against window.visualViewport if scrollbars affect layout. assert_equals: expected 1 but got 0

[Shawn Roberts]

Marked flaky in https://trac.webkit.org/changeset/244651/webkit

[Radar WebKit Bug Importer]

<rdar://problem/50209891>

[Radar WebKit Bug Importer]

<rdar://problem/50209894>

[Said Abou-Hallawa]

Created attachment 370329 [details]
Patch

[Simon Fraser (smfr)]

Comment on attachment 370329 [details]
Patch

Does our behavior differ from other browsers here?

[Said Abou-Hallawa]

The test has an overflow scrolling:

body {
    /* Ensure overflow */
    height: 200%;
}

The script waits for the first layout to be done.

-- If the browser shows decides it is going to show classic scrollbar for the overflow scroll, the 'resize' event will fire.

-- If the bowser decided it is going to show overlay scrollbar, the the 'resize' event will not fire because overlay scroll does not affect the layout.

The test then asserts that "if the resize event was fired, the browser would show classic scrollbar for the overflow scroll".

Firing the 'resize' event should happen only once. But the test has to wait enough time  to ensure the first layout was done, the event was enqueued and the event was dispatched.

The patch here makes the waiting time a little bit longer, extra 16ms. So this should not affect other browsers if they adopt the same change.

[Said Abou-Hallawa]

Running: run-webkit-httpd LayoutTests/imported/w3c/web-platform-tests/ opt-tests

Then opening the modified test from "http://localhost:8800/visual-viewport/viewport-resize-event-on-load-overflowing-page.html" in both Safari and Chrome shows that the test passes.

But this bug is about the test being flaky.

[WebKit Commit Bot]

Comment on attachment 370329 [details]
Patch

Clearing flags on attachment: 370329

Committed r245877: <https://trac.webkit.org/changeset/245877>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.