Bug 167254 – LayoutTest media/modern-media-controls/media-controller/media-controller-auto-hide-mouse-leave-after-play.html is flaky

• Component: New Bugs
• Status: RESOLVED FIXED

[Ryan Haddad]

LayoutTest media/modern-media-controls/media-controller/media-controller-auto-hide-mouse-leave-after-play.html is flaky

https://build.webkit.org/results/Apple%20Sierra%20Debug%20WK1%20(Tests)/r210975%20(2596)/results.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Fmodern-media-controls%2Fmedia-controller%2Fmedia-controller-auto-hide-mouse-leave-after-play.html

--- /Volumes/Data/slave/sierra-debug-tests-wk1/build/layout-test-results/media/modern-media-controls/media-controller/media-controller-auto-hide-mouse-leave-after-play-expected.txt
+++ /Volumes/Data/slave/sierra-debug-tests-wk1/build/layout-test-results/media/modern-media-controls/media-controller/media-controller-auto-hide-mouse-leave-after-play-actual.txt
@@ -15,7 +15,7 @@
 Mouse leaving the media.
 
 The initial auto-hide timer started when we resumed playback should not have expired or be overriden by the mouse entering and leaving the media, the controls should remain visible.
-PASS controlsBar.classList.contains('faded') is false
+FAIL controlsBar.classList.contains('faded') should be false. Was true.
 
 The initial auto-hide timer started when we resumed playback should now have expired and the controls should be faded.
 PASS controlsBar.classList.contains('faded') is true

[Ryan Haddad]

Marked as flaky in http://trac.webkit.org/projects/webkit/changeset/210986

[Ryan Haddad]

This test seems to only be flaky on debug on the bots, but it flakes on release on EWS. See https://webkit-queues.webkit.org/results/2943490

[Antoine Quint]

This is tricky because this test, and a host of others, is timing-sensitive by design since it's testing the auto-hide timer. The test checks as close as possible to the auto-hide delay that the controls are still visible, and then again as soon as possible after the auto-hide delay has elapsed to check that the controls are no longer visible. So in certain cases, probably where the system is under pressure and timers run late, we get in cases where the check prior to the timer firing fails because we've actually run past auto-hide delay. We're going to have to find a different way to test this.

[Antoine Quint]

This particular test is extremely flaky and always fails at first run in certain configurations (--release -g, --release -1 -g, --debug -g).

[Radar WebKit Bug Importer]

<rdar://problem/30259293>

[Antoine Quint]

Created attachment 300110 [details]
Patch

[Antoine Quint]

Created attachment 300112 [details]
Patch

[Build Bot]

Comment on attachment 300112 [details]
Patch

Attachment 300112 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/2974418

New failing tests:
css3/filters/backdrop/dynamic-with-clip-path.html

[Build Bot]

Created attachment 300118 [details]
Archive of layout-test-results from ews123 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews123  Port: ios-simulator-wk2  Platform: Mac OS X 10.11.6

[Dean Jackson]

Comment on attachment 300112 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=300112&action=review

> Source/WebCore/ChangeLog:15
> +        When we would identify that we need to prolong an existing auto-hide timer, when the previous auto-hide timer was marked
> +        as non-cancelable, calling _cancelAutoHideTimer() would not actually cancel the previous timer, which would let it fire
> +        and hide the controls bar. We now have two methods, _cancelAutoHideTimer() which always cancels the current auto-hide
> +        timer, and _cancelNonEnforcedAutoHideTimer() which is used from all other existing call sites, which only cancels the
> +        current auto-hide timer if it was marked as cancelable. This, and revised timing in the test itself, makes the tetst
> +        at media/modern-media-controls/media-controller/media-controller-auto-hide-mouse-leave-after-play.html a lot more
> +        reliable.

Why is this a weirdly long line length?

> Source/WebCore/ChangeLog:18
> +        We also make a small drive-by fix where we ensure the "autoHideDelay" property is set first so that setting other
> +        members which may set a timer do not used an undefined value for the auto-hide timer delay.

I guess that's a downside of setters. What looks like a property is actually a function that has side effects.

[Antoine Quint]

Committed r211374: <http://trac.webkit.org/changeset/211374>