http/tests/media/hls/video-controller-getStartDate.html Is flaky failing on macOS (mainly on wk2, but there are some failures on BigSur wk1 debug as well). History: https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Fmedia%2Fhls%2Fvideo-controller-getStartDate.html Result page: https://build.webkit.org/results/Apple-Catalina-Debug-WK2-Tests/r282087%20(16157)/results.html Diff: --- /Volumes/Data/worker/catalina-debug-tests-wk2/build/layout-test-results/http/tests/media/hls/video-controller-getStartDate-expected.txt +++ /Volumes/Data/worker/catalina-debug-tests-wk2/build/layout-test-results/http/tests/media/hls/video-controller-getStartDate-actual.txt @@ -4,6 +4,6 @@ EVENT(canplaythrough) EVENT(canplaythrough) EXPECTED (isNaN(video.getStartDate()) == 'true') OK -EXPECTED (video.getStartDate().toUTCString() == 'Wed, 03 Nov 2010 08:00:00 GMT') OK +EXPECTED (video.getStartDate().toUTCString() == 'Wed, 03 Nov 2010 08:00:00 GMT'), OBSERVED 'Invalid Date' FAIL END OF TEST
<rdar://problem/83246841>
Marked test expectations: https://trac.webkit.org/changeset/282670/webkit
Tried both on Release and Debug, but I was not able to reproduce the failure using run-webkit-tests --iterations 500 --force http/tests/media/hls/video-controller-getStartDate.html.
I was actually able to reproduce the failure with -f flag.
I tried bisecting again and found that I couldn't reproduce the failure with the revision that I was able to reproduce before (r281960). I was still able to reproduce the failure with TOT. The test has been flaky for long time, but it looks like it became more flaky recently. That might be why. I was able to reproduce the failure with r283204 but it passed with r283199. (No revisions in between). It looks like https://trac.webkit.org/changeset/283204/webkit caused the test to be more flaky.