imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize.html Is flakey failing on MacOS Big Sur Wk2. HISTORY URL: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fmedia-source%2Fmediasource-config-change-webm-v-framesize.html DIFF URL: https://build.webkit.org/results/Apple-BigSur-Release-WK2-Tests/r272143%20(1311)/imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize-diff.txt DIFF TEXT: --- /Volumes/Data/slave/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize-expected.txt +++ /Volumes/Data/slave/bigsur-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize-actual.txt @@ -1,3 +1,5 @@ -PASS Tests webm video-only frame size changes. +Harness Error (TIMEOUT), message = null +TIMEOUT Tests webm video-only frame size changes. Test timed out Bug Reproducible on r272141, same failure verified. Bug Produced using test: run-webkit-test imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize.html --iterations 1000 -f Regression point appears r272140, where the test did not crash. Bug appears to have been introduced in r272141
<rdar://problem/73837300>
We are unable to find a true regression point. This test appears to be very flakey. Tested back to r271039 just to see, and it did fail.
Created attachment 420026 [details] Patch
(In reply to Robert Jenner from comment #2) > We are unable to find a true regression point. This test appears to be very > flakey. > > Tested back to r271039 just to see, and it did fail. I think this test has just been flaky since it was unskipped as part of https://trac.webkit.org/changeset/270720/webkit
Comment on attachment 420026 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=420026&action=review > LayoutTests/platform/mac-wk2/TestExpectations:1329 > +webkit.org/b/221218 [ BigSur ] imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize.html [ Pass Failure ] Let's mark this BigSur+, since presumably it wold affect future releases too. > LayoutTests/platform/mac-wk2/TestExpectations:1331 > + Nit: extra newline here
Created attachment 420059 [details] Patch
(In reply to Ryan Haddad from comment #5) > Comment on attachment 420026 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=420026&action=review > > > LayoutTests/platform/mac-wk2/TestExpectations:1329 > > +webkit.org/b/221218 [ BigSur ] imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-v-framesize.html [ Pass Failure ] > > Let's mark this BigSur+, since presumably it wold affect future releases too. > > > LayoutTests/platform/mac-wk2/TestExpectations:1331 > > + > > Nit: extra newline here Changes made, how does it look now?
Created attachment 420062 [details] Patch
It does appear to be flakey failing in wk1 as well. Updating title and patch.
Created attachment 420754 [details] Update TestExpectations
Comment on attachment 420754 [details] Update TestExpectations Clearing flags on attachment: 420754 Committed r273045 (234246@main): <https://commits.webkit.org/234246@main>
The test is consistently failing on Big Sur WK1 with the following diff: -PASS Tests webm video-only frame size changes. +FAIL Tests webm video-only frame size changes. assert_true: video/webm;codecs="vp8" is supported. expected true got false perhaps that is expected, though. I don't see any failures on Big Sur WK2 since 02/09/21: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fmedia-source%2Fmediasource-config-change-webm-v-framesize.html&flavor=wk2&platform=mac We may be able to remove the expectation for WK2 and close this one out.