Bug 197286 - REGRESSION (r244182) [Mac WK2] Layout Test imported/w3c/web-platform-tests/visual-viewport/viewport-resize-event-on-load-overflowing-page.html is a flaky failure
Summary: REGRESSION (r244182) [Mac WK2] Layout Test imported/w3c/web-platform-tests/vi...
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Said Abou-Hallawa
Keywords: InRadar
Depends on:
Reported: 2019-04-25 10:18 PDT by Shawn Roberts
Modified: 2019-05-29 16:23 PDT (History)
8 users (show)

See Also:

Patch (3.97 KB, patch)
2019-05-21 11:08 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Roberts 2019-04-25 10:18:40 PDT
The following layout test is flaky on Mac WK2 Release and Debug


Probable cause:

It appears changes in https://trac.webkit.org/changeset/244182/webkit have made this test more flaky. 

Local testing on Debug is yielding close to 90% failure in 500 iterations. Testing done on r244181 and older yield on average 1 failure in 500 iterations.

Reproduced with :

run-webkit-tests imported/w3c/web-platform-tests/visual-viewport/viewport-resize-event-on-load-overflowing-page.html --iterations 500 -f --debug --exit-after-n-failures=5

Flakiness Dashboard:



--- /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/visual-viewport/viewport-resize-event-on-load-overflowing-page-expected.txt
+++ /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/visual-viewport/viewport-resize-event-on-load-overflowing-page-actual.txt
@@ -3,5 +3,5 @@
 Test Description: This test ensures that we fire a resize event against window.visualViewport if the page has overflow (since this creates a scrollbar and thus changes the viewport size).
-PASS Resize event fired exactly once against window.visualViewport if scrollbars affect layout. 
+FAIL Resize event fired exactly once against window.visualViewport if scrollbars affect layout. assert_equals: expected 1 but got 0
Comment 1 Shawn Roberts 2019-04-25 10:24:57 PDT
Marked flaky in https://trac.webkit.org/changeset/244651/webkit
Comment 2 Radar WebKit Bug Importer 2019-04-25 10:25:13 PDT
Comment 3 Radar WebKit Bug Importer 2019-04-25 10:25:13 PDT
Comment 4 Said Abou-Hallawa 2019-05-21 11:08:45 PDT
Created attachment 370329 [details]
Comment 5 Simon Fraser (smfr) 2019-05-21 11:49:19 PDT
Comment on attachment 370329 [details]

Does our behavior differ from other browsers here?
Comment 6 Said Abou-Hallawa 2019-05-21 12:22:49 PDT
The test has an overflow scrolling:

body {
    /* Ensure overflow */
    height: 200%;

The script waits for the first layout to be done.

-- If the browser shows decides it is going to show classic scrollbar for the overflow scroll, the 'resize' event will fire.

-- If the bowser decided it is going to show overlay scrollbar, the the 'resize' event will not fire because overlay scroll does not affect the layout.

The test then asserts that "if the resize event was fired, the browser would show classic scrollbar for the overflow scroll".

Firing the 'resize' event should happen only once. But the test has to wait enough time  to ensure the first layout was done, the event was enqueued and the event was dispatched.

The patch here makes the waiting time a little bit longer, extra 16ms. So this should not affect other browsers if they adopt the same change.
Comment 7 Said Abou-Hallawa 2019-05-21 12:50:25 PDT
Running: run-webkit-httpd LayoutTests/imported/w3c/web-platform-tests/ opt-tests

Then opening the modified test from "http://localhost:8800/visual-viewport/viewport-resize-event-on-load-overflowing-page.html" in both Safari and Chrome shows that the test passes.

But this bug is about the test being flaky.
Comment 8 WebKit Commit Bot 2019-05-29 16:23:55 PDT
Comment on attachment 370329 [details]

Clearing flags on attachment: 370329

Committed r245877: <https://trac.webkit.org/changeset/245877>
Comment 9 WebKit Commit Bot 2019-05-29 16:23:57 PDT
All reviewed patches have been landed.  Closing bug.