Bug 217263 – [ iOS ] 2 mediacapture-record tests are a constant failure

• Component: New Bugs
• Status: REOPENED

[Hector Lopez]

imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-no-sink.https.html
imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html

Tests are constant failures according to history on iOS. First occurrence of failures on iOS coincide with first testing on iOS. 

Both test were passing on macOS until r267870. From that point test imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html became flaky on macOS Catalina wk2 Debug/Release.

History:

https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fmediacapture-record%2FMediaRecorder-no-sink.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fmediacapture-record%2FMediaRecorder-peerconnection.https.html

Diff iOS for imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-no-sink.https.html:

--- /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-no-sink.https-expected.txt
+++ /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-no-sink.https-actual.txt
@@ -1,5 +1,7 @@
+
+Harness Error (TIMEOUT), message = null
 
 PASS PeerConnection MediaRecorder records audio from PeerConnection without sinks
 PASS PeerConnection MediaRecorder records video from PeerConnection without sinks
-PASS PeerConnection MediaRecorder records audio/video from PeerConnection without sinks
+TIMEOUT PeerConnection MediaRecorder records audio/video from PeerConnection without sinks Test timed out

Diff iOS for imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html

--- /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https-expected.txt
+++ /Volumes/Data/slave/ios-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https-actual.txt
@@ -6,8 +6,8 @@
 PASS PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":false} with format [passthrough].
 PASS PeerConnection MediaRecorder receives data after onstart, {"video":false,"audio":true} with format [passthrough].
 PASS PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":false,"audio":true} with format [passthrough].
-PASS PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":true} with format [passthrough].
-TIMEOUT PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":true} with format [passthrough]. Test timed out
+TIMEOUT PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":true} with format [passthrough]. Test timed out
+NOTRUN PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":true} with format [passthrough].
 NOTRUN PeerConnection MediaRecorder receives data after onstart, {"video":false,"audio":true} with format video/webm;codecs=vp8.
 NOTRUN PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":false,"audio":true} with format video/webm;codecs=vp8.
 NOTRUN PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":false} with format video/webm;codecs=vp8.

Diff macOS for imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html

--- /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https-actual.txt
@@ -1,23 +1,21 @@
 
-
-Harness Error (TIMEOUT), message = null
 
 PASS PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":false} with format [passthrough].
 PASS PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":false} with format [passthrough].
 PASS PeerConnection MediaRecorder receives data after onstart, {"video":false,"audio":true} with format [passthrough].
 PASS PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":false,"audio":true} with format [passthrough].
 PASS PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":true} with format [passthrough].
-TIMEOUT PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":true} with format [passthrough]. Test timed out
-NOTRUN PeerConnection MediaRecorder receives data after onstart, {"video":false,"audio":true} with format video/webm;codecs=vp8.
-NOTRUN PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":false,"audio":true} with format video/webm;codecs=vp8.
-NOTRUN PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":false} with format video/webm;codecs=vp8.
-NOTRUN PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":false} with format video/webm;codecs=vp8.
-NOTRUN PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":true} with format video/webm;codecs=vp8.
-NOTRUN PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":true} with format video/webm;codecs=vp8.
-NOTRUN PeerConnection MediaRecorder receives data after onstart, {"video":false,"audio":true} with format video/webm;codecs=vp9.
-NOTRUN PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":false,"audio":true} with format video/webm;codecs=vp9.
-NOTRUN PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":false} with format video/webm;codecs=vp9.
-NOTRUN PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":false} with format video/webm;codecs=vp9.
-NOTRUN PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":true} with format video/webm;codecs=vp9.
-NOTRUN PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":true} with format video/webm;codecs=vp9.
+PASS PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":true} with format [passthrough].
+FAIL PeerConnection MediaRecorder receives data after onstart, {"video":false,"audio":true} with format video/webm;codecs=vp8. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":false,"audio":true} with format video/webm;codecs=vp8. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":false} with format video/webm;codecs=vp8. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":false} with format video/webm;codecs=vp8. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":true} with format video/webm;codecs=vp8. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":true} with format video/webm;codecs=vp8. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder receives data after onstart, {"video":false,"audio":true} with format video/webm;codecs=vp9. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":false,"audio":true} with format video/webm;codecs=vp9. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":false} with format video/webm;codecs=vp9. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":false} with format video/webm;codecs=vp9. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder receives data after onstart, {"video":true,"audio":true} with format video/webm;codecs=vp9. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"
+FAIL PeerConnection MediaRecorder gets ondata on stopping recorded tracks {"video":true,"audio":true} with format video/webm;codecs=vp9. promise_test: Unhandled rejection with value: object "NotSupportedError: mimeType is not supported"



Reproduction for imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html on macOS Catalina wk 2 Release on r267897

run-webkit-tests imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html --iterations 1000 --exit-after-n-failures 3

Result:

Baseline search path: platform/mac-catalina -> Internal/Tests/LayoutTestsResultsForUnreleasedSoftware/mac-wk2 -> platform/mac-wk2 -> Internal/Tests/LayoutTestsResultsForUnreleasedSoftware/mac -> platform/mac -> platform/wk2 -> generic

Running 1 test

Running 1 WebKitTestRunner.

[1/1000] imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html failed unexpectedly (text diff)
[2/1000] imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html failed unexpectedly (text diff)
[3/1000] imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html failed unexpectedly (text diff)
Exiting early after 3 failures. 3 tests run.


0 tests ran as expected, 3 didn't (997 didn't run):


Regressions: Unexpected text-only failures (1)
  imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-peerconnection.https.html [ Failure ]

[Radar WebKit Bug Importer]

<rdar://problem/69904225>

[Hector Lopez]

See webkit.org/b/217262 for test expectations for Mac

Test expectations for iOS:

[Hector Lopez]

Test expectations for iOS:

https://trac.webkit.org/changeset/267905/webkit

See test expectation for Mac:

https://trac.webkit.org/changeset/267904/webkit

[youenn fablet]

Created attachment 411109 [details]
Patch

[EWS]

commit-queue failed to commit attachment 411109 [details] to WebKit repository.

[EWS]

Committed r268361: <https://trac.webkit.org/changeset/268361>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 411109 [details].

[youenn fablet]

Reopening for further investigation

[Hector Lopez]

imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-no-sink.https.html

Test expectation adjustment:

https://trac.webkit.org/changeset/268737/webkit