Bug 263227 – REGRESSION(264111@main): imported/w3c/web-platform-tests/webrtc/simulcast/basic.https.html is a flaky failure

• Status: RESOLVED
• Resolution: CONFIGURATION CHANGED
• Component: Tools / Tests

[Ben Schwartz]

The following tests are consistently failing on macOS wk2:

imported/w3c/web-platform-tests/webrtc/simulcast/basic.https.html
imported/w3c/web-platform-tests/webrtc/protocol/handover-datachannel.html

HISTORY:

https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2Fsimulcast%2Fbasic.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2Fprotocol%2Fhandover-datachannel.html&recent=false&platform=mac

TEXT DIFFs:

-PASS Basic simulcast setup with two spatial layers
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Basic simulcast setup with two spatial layers Test timed out
+
_________

-PASS Handover with datachannel reinitiated from new callee completes
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Handover with datachannel reinitiated from new callee completes Test timed out
+

DIFF URLs:

https://ews-build.s3-us-west-2.amazonaws.com/macOS-Monterey-Release-WK2-Tests-EWS/d3496a07-20520/imported/w3c/web-platform-tests/webrtc/simulcast/basic.https-pretty-diff.html
https://ews-build.s3-us-west-2.amazonaws.com/macOS-Monterey-Release-WK2-Tests-EWS/d3496a07-20520/imported/w3c/web-platform-tests/webrtc/protocol/handover-datachannel-pretty-diff.html

REPRODUCIBILITY:

I was able to reproduce this issue on macOS Ventura (13.6, 22G120/arm64) ToT using the following command:

run-webkit-tests --debug --iterations 50 --child-processes 5 -f imported/w3c/web-platform-tests/webrtc/simulcast/basic.https.html imported/w3c/web-platform-tests/webrtc/protocol/handover-datachannel.html

REGRESSION:

I was able to identify a potential initial regression point at 264111@main; however, it appears that the failures became more frequent at 269302@main.

[Radar WebKit Bug Importer]

<rdar://problem/117045403>

[EWS]

Test gardening commit 269391@main (c193b95a25cf): <https://commits.webkit.org/269391@main>

Reviewed commits have been landed. Closing PR #19141 and removing active labels.

[Ryan Haddad]

It looks like imported/w3c/web-platform-tests/webrtc/simulcast/basic.https.html is still a flaky failure, but imported/w3c/web-platform-tests/webrtc/protocol/handover-datachannel.html appears to be passing now.

[youenn fablet]

imported/w3c/web-platform-tests/webrtc/simulcast/basic.https.html is flaky on Monterey x86_64 only.
imported/w3c/web-platform-tests/webrtc/protocol/handover-datachannel.html is no longer flaky.

Updating test expectations in https://bugs.webkit.org/show_bug.cgi?id=270791.

I filed https://bugs.webkit.org/show_bug.cgi?id=270790 to continue the investigation since the same bug appears in several tests.