Bug 210335 – Add a timer to AVVideoCaptureSource to verify reception of frames

• Status: RESOLVED
• Resolution: FIXED
• Component: WebRTC

[youenn fablet]

Add a timer to AVVideoCaptureSource to verify reception of frames

[youenn fablet]

Created attachment 396085 [details]
Patch

[youenn fablet]

Created attachment 396087 [details]
Patch

[youenn fablet]

Created attachment 396197 [details]
Patch

[Eric Carlson]

Comment on attachment 396197 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=396197&action=review

> Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h:143
> +    static constexpr Seconds verifyCaptureInterval = 3_s;

Is three seconds long enough?

> Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h:144
> +    static const uint64_t framesToDropWhenStarting = 4;

Shouldn't this be relative to the capture rate?

> Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm:163
> +    RELEASE_LOG_ERROR(WebRTC, "AVVideoCaptureSource::verifyIsCapturing - no frame received in %d seconds, failing", static_cast<int>(m_verifyCapturingTimer.repeatInterval().value()));
> +    captureFailed();

Failing capture completely is quite severe. Should we try to stop and restart the camera one time?

[youenn fablet]

(In reply to Eric Carlson from comment #4)
> Comment on attachment 396197 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=396197&action=review
> 
> > Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h:143
> > +    static constexpr Seconds verifyCaptureInterval = 3_s;
> 
> Is three seconds long enough?

I tried to pick something not too short but not too long either.
Timer starts when session is running so frames should flow very quickly in theory.


> > Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.h:144
> > +    static const uint64_t framesToDropWhenStarting = 4;
> 
> Shouldn't this be relative to the capture rate?

Might be the case.
IIRC, from your testing, the first frames were bad, probably at 30 fps.
I am guessing that we could reduce this number at lower frame rate.

> > Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm:163
> > +    RELEASE_LOG_ERROR(WebRTC, "AVVideoCaptureSource::verifyIsCapturing - no frame received in %d seconds, failing", static_cast<int>(m_verifyCapturingTimer.repeatInterval().value()));
> > +    captureFailed();
> 
> Failing capture completely is quite severe. Should we try to stop and
> restart the camera one time?

I was wondering the same thing and went this way following what we are doing for audio capture. At the same time, the application can always call getUserMedia if it wants to restart capture.

Let's try that as a follow-up.

[EWS]

Committed r260064: <https://trac.webkit.org/changeset/260064>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 396197 [details].

[Radar WebKit Bug Importer]

<rdar://problem/61762873>