Bug 161491 – [WK2] LayoutTest media/video-main-content-allow-then-scroll.html is a flaky failure

• Status: NEW
• Component: Media

[Ryan Haddad]

LayoutTest media/video-main-content-allow-then-scroll.html is a flaky failure

This test is a flaky failure on ios-simulator-wk2

https://build.webkit.org/builders/Apple%20iOS%209%20Simulator%20Debug%20WK2%20(Tests)/builds/4887

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Fvideo-main-content-allow-then-scroll.html

--- /Volumes/Data/slave/ios-simulator-9-debug-tests-wk2/build/layout-test-results/media/video-main-content-allow-then-scroll-expected.txt
+++ /Volumes/Data/slave/ios-simulator-9-debug-tests-wk2/build/layout-test-results/media/video-main-content-allow-then-scroll-actual.txt
@@ -2,7 +2,7 @@
 EVENT(canplaythrough)
 EVENT(playing)
 RUN(document.body.scrollTop = document.body.clientHeight)
-EXPECTED (video.paused == 'false') OK
+EVENT(pause) TEST(false) FAIL
 END OF TEST

[Ryan Haddad]

Marked test as flaky in http://trac.webkit.org/projects/webkit/changeset/205296

[Matt Lewis]

This test has since started to be flaky on macOS and consistently fail on iOS according the the dashboard.

The first macOS build with a failure was:

https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK1%20(Tests)/builds/2992
https://build.webkit.org/results/Apple%20Sierra%20Debug%20WK1%20(Tests)/r220715%20(2992)/results.html

The macOS Diff:
--- /Volumes/Data/slave/sierra-debug-tests-wk1/build/layout-test-results/media/video-main-content-allow-then-scroll-expected.txt
+++ /Volumes/Data/slave/sierra-debug-tests-wk1/build/layout-test-results/media/video-main-content-allow-then-scroll-actual.txt
@@ -2,7 +2,7 @@
 EVENT(canplaythrough)
 EVENT(playing)
 RUN(document.body.scrollTop = document.body.clientHeight)
-EXPECTED (video.paused == 'false') OK
+EXPECTED (video.paused == 'false'), OBSERVED 'true' FAIL
 END OF TEST

[Matt Lewis]

Marked test as flaky on Mac and failing on iOS:
https://trac.webkit.org/changeset/221708/webkit