263227 2023-10-16 17:23:19 -0700 REGRESSION(264111@main): imported/w3c/web-platform-tests/webrtc/simulcast/basic.https.html is a flaky failure 2024-03-11 06:01:19 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED CONFIGURATION CHANGED https://bugs.webkit.org/show_bug.cgi?id=256780 https://bugs.webkit.org/show_bug.cgi?id=263060 https://bugs.webkit.org/show_bug.cgi?id=263221 InRadar P2 Normal --- 1 ben_schwartz webkit-unassigned achristensen webkit-bot-watchers-bugzilla webkit-bug-importer youennf oldest_to_newest 1985464 0 ben_schwartz 2023-10-16 17:23:19 -0700 The following tests are consistently failing on macOS wk2: imported/w3c/web-platform-tests/webrtc/simulcast/basic.https.html imported/w3c/web-platform-tests/webrtc/protocol/handover-datachannel.html HISTORY: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2Fsimulcast%2Fbasic.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2Fprotocol%2Fhandover-datachannel.html&recent=false&platform=mac TEXT DIFFs: -PASS Basic simulcast setup with two spatial layers +Harness Error (TIMEOUT), message = null +TIMEOUT Basic simulcast setup with two spatial layers Test timed out + _________ -PASS Handover with datachannel reinitiated from new callee completes +Harness Error (TIMEOUT), message = null +TIMEOUT Handover with datachannel reinitiated from new callee completes Test timed out + DIFF URLs: https://ews-build.s3-us-west-2.amazonaws.com/macOS-Monterey-Release-WK2-Tests-EWS/d3496a07-20520/imported/w3c/web-platform-tests/webrtc/simulcast/basic.https-pretty-diff.html https://ews-build.s3-us-west-2.amazonaws.com/macOS-Monterey-Release-WK2-Tests-EWS/d3496a07-20520/imported/w3c/web-platform-tests/webrtc/protocol/handover-datachannel-pretty-diff.html REPRODUCIBILITY: I was able to reproduce this issue on macOS Ventura (13.6, 22G120/arm64) ToT using the following command: run-webkit-tests --debug --iterations 50 --child-processes 5 -f imported/w3c/web-platform-tests/webrtc/simulcast/basic.https.html imported/w3c/web-platform-tests/webrtc/protocol/handover-datachannel.html REGRESSION: I was able to identify a potential initial regression point at 264111@main; however, it appears that the failures became more frequent at 269302@main. 1985465 1 webkit-bug-importer 2023-10-16 17:24:00 -0700 <rdar://problem/117045403> 1985467 2 ews-feeder 2023-10-16 17:35:43 -0700 Test gardening commit 269391@main (c193b95a25cf): <https://commits.webkit.org/269391@main> Reviewed commits have been landed. Closing PR #19141 and removing active labels. 1999063 3 ryanhaddad 2023-12-12 12:42:23 -0800 It looks like imported/w3c/web-platform-tests/webrtc/simulcast/basic.https.html is still a flaky failure, but imported/w3c/web-platform-tests/webrtc/protocol/handover-datachannel.html appears to be passing now. 2020083 4 youennf 2024-03-11 06:01:19 -0700 imported/w3c/web-platform-tests/webrtc/simulcast/basic.https.html is flaky on Monterey x86_64 only. imported/w3c/web-platform-tests/webrtc/protocol/handover-datachannel.html is no longer flaky. Updating test expectations in https://bugs.webkit.org/show_bug.cgi?id=270791. I filed https://bugs.webkit.org/show_bug.cgi?id=270790 to continue the investigation since the same bug appears in several tests.