Bug 244677 – New Test?: [ macOS wk1 ] media/media-source/media-source-istypesupported-vp8-without-vp9.html is a constant text failure

• Status: RESOLVED
• Resolution: FIXED
• Component: Media

[Robert Jenner]

media/media-source/media-source-istypesupported-vp8-without-vp9.html

is a constant text failure on macOS wk1. 

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=media%2Fmedia-source%2Fmedia-source-istypesupported-vp8-without-vp9.html

It is also slowing down EWS. Observed here:
https://ews-build.webkit.org/#/builders/73


TEXT DIFF:
 These tests may be expected to fail if the WebKit port does not support the format.
 
 Forcing VP9 decoder to be enabled.
-EXPECTED (MediaSource.isTypeSupported('video/webm; codecs=vp8') == 'true') OK
+EXPECTED (MediaSource.isTypeSupported('video/webm; codecs=vp8') == 'true'), OBSERVED 'false' FAIL
 Forcing VP9 decoder to be disabled.
-EXPECTED (MediaSource.isTypeSupported('video/webm; codecs=vp8.0') == 'true') OK
+EXPECTED (MediaSource.isTypeSupported('video/webm; codecs=vp8.0') == 'true'), OBSERVED 'false' FAIL
 END OF TEST

DIFF URL:
https://build.webkit.org/results/Apple-BigSur-Release-AppleSilicon-WK1-Tests/254041@main%20(12600)/media/media-source/media-source-istypesupported-vp8-without-vp9-pretty-diff.html

[Radar WebKit Bug Importer]

<rdar://problem/99446320>

[Karl Rackler]

I have marked this test as a flaky failure while this issue is investigated.

[EWS]

Test gardening commit 254054@main (066fa32708f5): <https://commits.webkit.org/254054@main>

Reviewed commits have been landed. Closing PR #3921 and removing active labels.

[Karl Rackler]

REPRODUCTION STEPS
I can reproduce this on 254044@main ToT.

Command: 
run-webkit-tests --debug -1 --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 -f --no-retry media/media-source/media-source-istypesupported-vp8-without-vp9.html

Result: 
Regressions: Unexpected text-only failures (1)
  media/media-source/media-source-istypesupported-vp8-without-vp9.html [ Failure ]

[Ryan Haddad]

This test is failing on release as well, we need to remove the [ Debug ] flag from the expectation.

However, I don't know if webm is expected to work on WK1. Maybe this test needs to be skipped.

[Jonathan Bedard]

Pull request: https://github.com/WebKit/WebKit/pull/3921

[EWS]

Test gardening commit 254270@main (683811c96127): <https://commits.webkit.org/254270@main>

Reviewed commits have been landed. Closing PR #4128 and removing active labels.

[Jean-Yves Avenard [:jya]]

Pull request: https://github.com/WebKit/WebKit/pull/7543

[Jean-Yves Avenard [:jya]]

VP8 and VP9 are not supported in WK1, so the test is expected to fail. Amend the expectations to expect failure.

So the failure is to be expected

[EWS]

Committed 258071@main (96015181185f): <https://commits.webkit.org/258071@main>

Reviewed commits have been landed. Closing PR #7543 and removing active labels.

[Robert Jenner]

As stated above the failure is expected for WK1 platforms. So marking the expectation as Failure is the correct fix. Closing.