Bug 179643 – [mac-wk1] Layout Test media/W3C/audio/networkState/networkState_during_loadstart.html is flaky

• Status: NEW
• Component: New Bugs

[Matt Lewis]

media/W3C/audio/networkState/networkState_during_loadstart.html is a flaky failure on macOS WK1 testers according to the flakiness dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2FW3C%2Faudio%2FnetworkState%2FnetworkState_during_loadstart.html

build:
https://build.webkit.org/results/Apple%20Sierra%20Release%20WK1%20(Tests)/r224776%20(6083)/results.html
https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK1%20(Tests)/builds/6083

I was able to reproduce the flake with run-webkit-test --no-retry-failures --verbose media/W3C/audio/networkState/networkState_during_loadstart.html --iterations=100 --debug -f -g -1

diff:
--- /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/media/W3C/audio/networkState/networkState_during_loadstart-expected.txt
+++ /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/media/W3C/audio/networkState/networkState_during_loadstart-actual.txt
@@ -3,7 +3,7 @@
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
-PASS "2" is "2"
+FAIL "1" should be 2. Was 1.
 
 TEST COMPLETE
 spec reference

[Matt Lewis]

I was able to reproduce the flaky failure as far back as

[Matt Lewis]

(In reply to Matt Lewis from comment #1)
> I was able to reproduce the flaky failure as far back as
revision https://trac.webkit.org/changeset/224340/webkit but I believe it would go back further.

the test became much more flaky after the revisions:

https://trac.webkit.org/changeset/224348/webkit
https://trac.webkit.org/changeset/224347/webkit

there is no archive of r224347 so I'm unable to determine between the two.

[Ryan Haddad]

This is also seen with media/W3C/video/networkState/networkState_during_loadstart.html
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2FW3C%2Fvideo%2FnetworkState%2FnetworkState_during_loadstart.html

[Ryan Haddad]

Marked test as flaky in https://trac.webkit.org/changeset/227183/webkit