Bug 240348 - REGRESSION (250343@main): [iOS] imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-videoDetectorTest.html is a consistent failure
Summary: REGRESSION (250343@main): [iOS] imported/w3c/web-platform-tests/webrtc/RTCPee...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Karl Rackler
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2022-05-12 08:46 PDT by Karl Rackler
Modified: 2022-05-16 16:50 PDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Karl Rackler 2022-05-12 08:46:54 PDT
Description:
imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-videoDetectorTest.html

The first failure that I saw on the dashboard was 5/6/2022 at r293885, and the dashboard shows passing on r293882.  Between r293883 and r293885 probably introduced the consistent failure. 

REPRODUCTION STEPS
I can reproduce this on r293885, but repro against all earlier commits are also failures which is different from the dashboard history.

Command: 
run-webkit-tests --root 293885 --ios-simulator --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 --no-retry imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-videoDetectorTest.html

Result:  
Regressions: Unexpected text-only failures (1)
  imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-videoDetectorTest.html [ Failure ]

History:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebrtc%2FRTCPeerConnection-videoDetectorTest.html&limit=50000&platform=ios

Diff:
--- /Volumes/Data/worker/Apple-iOS-15-Simulator-Debug-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-videoDetectorTest-expected.txt
+++ /Volumes/Data/worker/Apple-iOS-15-Simulator-Debug-WK2-Tests/build/layout-test-results/imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-videoDetectorTest-actual.txt
@@ -1,3 +1,5 @@
 
-PASS Signal detector detects track change within reasonable time
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT Signal detector detects track change within reasonable time Test timed out
+

stderr:
(webrtc_video_engine.cc:3482): Absent receive stream; ignoring clearing encoded frame sink for ssrc 0
(video_send_stream_impl.cc:187): ERROR: Initial encoder max bitrate = -1 which is <= 0!
Comment 1 Radar WebKit Bug Importer 2022-05-12 08:47:13 PDT
<rdar://problem/93176919>
Comment 2 Karl Rackler 2022-05-12 08:55:48 PDT
imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-videoDetectorTest.html ran on average 24 ms before r293885 and is now running on average 135 ms.
Comment 3 Alexey Proskuryakov 2022-05-12 09:29:32 PDT
This would be seconds, not milliseconds.

I see the same stderr message in passing builds, so that part is not a regression.
Comment 4 Karl Rackler 2022-05-12 10:50:06 PDT
I have marked this test as slow while this issue is investigated.
Comment 5 Karl Rackler 2022-05-12 10:53:08 PDT
Pull request: https://github.com/WebKit/WebKit/pull/600
Comment 6 EWS 2022-05-12 10:55:56 PDT
Test gardening commit r294110 (250493@main): <https://commits.webkit.org/250493@main>

Reviewed commits have been landed. Closing PR #600 and removing active labels.
Comment 7 Karl Rackler 2022-05-16 09:15:49 PDT
Pull request: https://github.com/WebKit/WebKit/pull/636
Comment 8 Karl Rackler 2022-05-16 09:16:06 PDT
I have now marked this test as a consistent failure while this issue is investigated.
Comment 9 EWS 2022-05-16 09:18:46 PDT
Test gardening commit r294236 (250594@main): <https://commits.webkit.org/250594@main>

Reviewed commits have been landed. Closing PR #636 and removing active labels.
Comment 10 Ryan Haddad 2022-05-16 14:58:16 PDT
The test history does indeed suggest that this started with r293885 / 250343@main, and that is the only WebRTC related commit in the range.