Bug 230411

Summary: [ macOS ] http/tests/media/hls/video-controller-getStartDate.html is flaky failing
Product: WebKit Reporter: ayumi_kojima
Component: MediaAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: eric.carlson, pvollan, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   

Description ayumi_kojima 2021-09-17 10:36:19 PDT
http/tests/media/hls/video-controller-getStartDate.html

Is flaky failing on macOS (mainly on wk2, but there are some failures on BigSur wk1 debug as well).

History: https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Fmedia%2Fhls%2Fvideo-controller-getStartDate.html

Result page: https://build.webkit.org/results/Apple-Catalina-Debug-WK2-Tests/r282087%20(16157)/results.html

Diff: 

--- /Volumes/Data/worker/catalina-debug-tests-wk2/build/layout-test-results/http/tests/media/hls/video-controller-getStartDate-expected.txt
+++ /Volumes/Data/worker/catalina-debug-tests-wk2/build/layout-test-results/http/tests/media/hls/video-controller-getStartDate-actual.txt
@@ -4,6 +4,6 @@
 EVENT(canplaythrough)
 EVENT(canplaythrough)
 EXPECTED (isNaN(video.getStartDate()) == 'true') OK
-EXPECTED (video.getStartDate().toUTCString() == 'Wed, 03 Nov 2010 08:00:00 GMT') OK
+EXPECTED (video.getStartDate().toUTCString() == 'Wed, 03 Nov 2010 08:00:00 GMT'), OBSERVED 'Invalid Date' FAIL
 END OF TEST
Comment 1 Radar WebKit Bug Importer 2021-09-17 10:38:43 PDT
<rdar://problem/83246841>
Comment 2 ayumi_kojima 2021-09-17 10:45:06 PDT
Marked test expectations: https://trac.webkit.org/changeset/282670/webkit
Comment 3 ayumi_kojima 2021-09-17 15:18:44 PDT
Tried both on Release and Debug, but I was not able to reproduce the failure using run-webkit-tests --iterations 500 --force http/tests/media/hls/video-controller-getStartDate.html.
Comment 4 ayumi_kojima 2021-09-17 15:28:18 PDT
I was actually able to reproduce the failure with -f flag.
Comment 5 ayumi_kojima 2021-10-14 14:40:37 PDT
I tried bisecting again and found that I couldn't reproduce the failure with the revision that I was able to reproduce before (r281960).

I was still able to reproduce the failure with TOT. The test has been flaky for long time, but it looks like it became more flaky recently. That might be why.

I was able to reproduce the failure with r283204 but it passed with r283199. (No revisions in between). It looks like https://trac.webkit.org/changeset/283204/webkit caused the test to be more flaky.