Bug 181565 – LayoutTests imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html and mediasource-config-change-mp4-v-framesize.html are flaky on EWS

• Status: NEW
• Component: Media

[Ryan Haddad]

LayoutTest imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate.html is a flaky failure on EWS

See https://bugs.webkit.org/show_bug.cgi?id=168447#c50

--- /Volumes/Data/EWS/WebKit/WebKitBuild/Debug/layout-test-results/imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-framesize-expected.txt
+++ /Volumes/Data/EWS/WebKit/WebKitBuild/Debug/layout-test-results/imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-framesize-actual.txt
@@ -1,3 +1,5 @@
 
-PASS Tests mp4 video-only frame size changes. 
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Tests mp4 video-only frame size changes. Test timed out
+

[Ryan Haddad]

Also flaked here https://webkit-queues.webkit.org/results/6041686

[Ryan Haddad]

Just realized that I pasted a diff from a different test: imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-framesize.html

Both flaked on the same patch here https://bugs.webkit.org/show_bug.cgi?id=168447

Here is the very similar diff for mediasource-config-change-mp4-v-bitrate.html
--- /Volumes/Data/EWS/WebKit/WebKitBuild/Release/layout-test-results/imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate-expected.txt
+++ /Volumes/Data/EWS/WebKit/WebKitBuild/Release/layout-test-results/imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-bitrate-actual.txt
@@ -1,3 +1,5 @@
 
-PASS Tests mp4 video-only bitrate changes. 
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Tests mp4 video-only bitrate changes. Test timed out
+

[Ryan Haddad]

These tests don't seem to be failing on the non-EWS bots. I haven't dug in to this yet, but there is a hardware difference between the general test machines (Mac mini) and EWS (Mac Pro).

[Alexey Proskuryakov]

*** Bug 181571 has been marked as a duplicate of this bug. ***

[Ryan Haddad]

Flakiness dashboard results for these tests:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fmedia-source%2Fmediasource-config-change-mp4

[WebKit Commit Bot]

The commit-queue just saw imported/w3c/web-platform-tests/media-source/mediasource-config-change-mp4-v-framesize.html flake (text diff) while processing attachment 331744 [details] on bug 179670.
Bot: webkit-cq-01  Port: <class 'webkitpy.common.config.ports.MacPort'>  Platform: Mac OS X 10.12.6

[WebKit Commit Bot]

Created attachment 331750 [details]
Archive of layout-test-results from webkit-cq-01

[Ryan Haddad]

Marked tests as flaky in https://trac.webkit.org/r227249 to help EWS.