media/W3C/audio/networkState/networkState_during_loadstart.html is a flaky failure on macOS WK1 testers according to the flakiness dashboard: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2FW3C%2Faudio%2FnetworkState%2FnetworkState_during_loadstart.html build: https://build.webkit.org/results/Apple%20Sierra%20Release%20WK1%20(Tests)/r224776%20(6083)/results.html https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK1%20(Tests)/builds/6083 I was able to reproduce the flake with run-webkit-test --no-retry-failures --verbose media/W3C/audio/networkState/networkState_during_loadstart.html --iterations=100 --debug -f -g -1 diff: --- /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/media/W3C/audio/networkState/networkState_during_loadstart-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk1/build/layout-test-results/media/W3C/audio/networkState/networkState_during_loadstart-actual.txt @@ -3,7 +3,7 @@ On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE". -PASS "2" is "2" +FAIL "1" should be 2. Was 1. TEST COMPLETE spec reference
I was able to reproduce the flaky failure as far back as
(In reply to Matt Lewis from comment #1) > I was able to reproduce the flaky failure as far back as revision https://trac.webkit.org/changeset/224340/webkit but I believe it would go back further. the test became much more flaky after the revisions: https://trac.webkit.org/changeset/224348/webkit https://trac.webkit.org/changeset/224347/webkit there is no archive of r224347 so I'm unable to determine between the two.
This is also seen with media/W3C/video/networkState/networkState_during_loadstart.html https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2FW3C%2Fvideo%2FnetworkState%2FnetworkState_during_loadstart.html
Marked test as flaky in https://trac.webkit.org/changeset/227183/webkit