Bug 236558 - [GStreamer] Initial MediaRecorder implementation
Summary: [GStreamer] Initial MediaRecorder implementation
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Platform (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Philippe Normand
URL:
Keywords: InRadar
Depends on:
Blocks: 213699
  Show dependency treegraph
 
Reported: 2022-02-13 04:23 PST by Philippe Normand
Modified: 2022-02-22 11:33 PST (History)
19 users (show)

See Also:


Attachments
Patch (49.29 KB, patch)
2022-02-13 06:31 PST, Philippe Normand
no flags Details | Formatted Diff | Diff
Patch (49.76 KB, patch)
2022-02-14 06:03 PST, Philippe Normand
no flags Details | Formatted Diff | Diff
Patch (50.05 KB, patch)
2022-02-18 09:35 PST, Philippe Normand
no flags Details | Formatted Diff | Diff
Patch (53.86 KB, patch)
2022-02-19 02:47 PST, Philippe Normand
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Philippe Normand 2022-02-13 04:23:19 PST
.
Comment 1 Philippe Normand 2022-02-13 06:31:40 PST
Created attachment 451815 [details]
Patch
Comment 2 Philippe Normand 2022-02-13 09:10:38 PST
Comment on attachment 451815 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=451815&action=review

> Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.cpp:59
> +#if HAVE(AVASSETWRITERDELEGATE)

Looks like this introduces test failures on the mac-wk2 EWS. Maybe this should be PLATFORM(COCOA)... or !USE(GSTREAMER_TRANSCODER)
Comment 3 Philippe Normand 2022-02-14 03:12:59 PST
Comment on attachment 451815 [details]
Patch

There's a deadlock during the transcoder shutdown, need to debug this.
Comment 4 Philippe Normand 2022-02-14 06:03:28 PST
Created attachment 451896 [details]
Patch
Comment 5 Eric Carlson 2022-02-14 08:25:43 PST
Comment on attachment 451896 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=451896&action=review

> Source/WebCore/Modules/mediarecorder/MediaRecorder.cpp:51
> +#if PLATFORM(COCOA) || USE(GSTREAMER_TRANSCODER)

Maybe it is time to define an ENABLE flag?
Comment 6 Philippe Normand 2022-02-15 04:06:32 PST
Comment on attachment 451896 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=451896&action=review

>> Source/WebCore/Modules/mediarecorder/MediaRecorder.cpp:51
>> +#if PLATFORM(COCOA) || USE(GSTREAMER_TRANSCODER)
> 
> Maybe it is time to define an ENABLE flag?

Maybe so! Is ENABLE(MEDIA_RECORDER) a good name?
Comment 7 Xabier Rodríguez Calvar 2022-02-16 05:13:54 PST
Comment on attachment 451896 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=451896&action=review

Would be interesting to honor what Eric is saying.

> Source/WebCore/platform/mediarecorder/MediaRecorderPrivateGStreamer.cpp:85
> +    auto scopeExit = makeScopeExit([this, weakThis = WeakPtr { this }, completionHandler = WTFMove(completionHandler)]() mutable {

Interesting to see creating a WeakPtr for a scope exit. Can this really happen? Can this object be destroyed while executing this method? If this can happen I would be even more worried about accessing the other attributes, like m_eos, m_eosLock, etc. If this is so, then I guess what we should do is doing a hard ref and making up some invalidation system or at least locking in the destructor. Calling this method on a dead object would be the worst scenario, obviously.

> Source/WebCore/platform/mediarecorder/MediaRecorderPrivateGStreamer.cpp:110
> +    bool isEOS = false;
> +    {
> +        LockHolder lock(m_eosLock);
> +        isEOS = m_eos;
> +    }
> +    while (!isEOS) {
> +        LockHolder lock(m_eosLock);
> +        m_eosCondition.waitFor(m_eosLock, 200_ms, [weakThis = WeakPtr { this }]() -> bool {

I don't think you need to do this because the waitFor checks the predicate before waiting.

> Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp:198
> +        callOnMainThread([&] {
> +            if (m_track.hasAudio())
> +                m_track.source().removeAudioSampleObserver(*this);
> +            else if (m_track.hasVideo())
> +                m_track.source().removeVideoSampleObserver(*this);
> +            m_track.removeObserver(*this);
> +        });

Nope! You call this from the destructor, deferring this to the main thread would mean you're accessing stuff after the object is destroyed. Maybe AndWait can help, but this means r- .
Comment 8 Philippe Normand 2022-02-16 06:15:38 PST
Comment on attachment 451896 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=451896&action=review

>> Source/WebCore/platform/mediarecorder/MediaRecorderPrivateGStreamer.cpp:110
>> +        m_eosCondition.waitFor(m_eosLock, 200_ms, [weakThis = WeakPtr { this }]() -> bool {
> 
> I don't think you need to do this because the waitFor checks the predicate before waiting.

"this"?

>> Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp:198
>> +        });
> 
> Nope! You call this from the destructor, deferring this to the main thread would mean you're accessing stuff after the object is destroyed. Maybe AndWait can help, but this means r- .

Not only from the destructor. As mentioned in the comment a few lines above, the transcoder uses a dedicated thread :) The pipeline state is changed from that thread.
Comment 9 Philippe Normand 2022-02-16 06:16:51 PST
(In reply to Xabier Rodríguez Calvar from comment #7)
> Comment on attachment 451896 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=451896&action=review
> 
> Would be interesting to honor what Eric is saying.

https://bugs.webkit.org/show_bug.cgi?id=236652 :)
Comment 10 Xabier Rodríguez Calvar 2022-02-16 07:12:20 PST
(In reply to Philippe Normand from comment #8)
> Comment on attachment 451896 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=451896&action=review
> 
> >> Source/WebCore/platform/mediarecorder/MediaRecorderPrivateGStreamer.cpp:110
> >> +        m_eosCondition.waitFor(m_eosLock, 200_ms, [weakThis = WeakPtr { this }]() -> bool {
> > 
> > I don't think you need to do this because the waitFor checks the predicate before waiting.
> 
> "this"?

You don't need the first part, you can initialize as false, let the loop begin, then do the waitFor because it will check the predicate before blocking.

> >> Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp:198
> >> +        });
> > 
> > Nope! You call this from the destructor, deferring this to the main thread would mean you're accessing stuff after the object is destroyed. Maybe AndWait can help, but this means r- .
> 
> Not only from the destructor. As mentioned in the comment a few lines above,
> the transcoder uses a dedicated thread :) The pipeline state is changed from
> that thread.

Well, the problem here is just the destructor, right?
Comment 11 Philippe Normand 2022-02-16 07:57:14 PST
Comment on attachment 451896 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=451896&action=review

>>>> Source/WebCore/platform/mediastream/gstreamer/GStreamerMediaStreamSource.cpp:198
>>>> +        });
>>> 
>>> Nope! You call this from the destructor, deferring this to the main thread would mean you're accessing stuff after the object is destroyed. Maybe AndWait can help, but this means r- .
>> 
>> Not only from the destructor. As mentioned in the comment a few lines above, the transcoder uses a dedicated thread :) The pipeline state is changed from that thread.
> 
> Well, the problem here is just the destructor, right?

There is no issue with the destructor. The issue is that webkitMediaStreamSrcChangeState() is not always called from the main thread, and it has this:

    if (transition == GST_STATE_CHANGE_PAUSED_TO_READY) {
        GST_OBJECT_LOCK(self);
        for (auto& item : self->priv->sources)
            item->stopObserving();
        GST_OBJECT_UNLOCK(self);
    }

We could do the main thread dispatch there, but I thought it was nicer to do it in stopObserving() because that's the method that directly calls to RealtimeMediaSource.
Comment 12 Philippe Normand 2022-02-18 09:35:57 PST
Created attachment 452529 [details]
Patch
Comment 13 Philippe Normand 2022-02-19 02:47:34 PST
Created attachment 452633 [details]
Patch
Comment 14 EWS 2022-02-22 11:32:42 PST
Committed r290323 (247645@main): <https://commits.webkit.org/247645@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 452633 [details].
Comment 15 Radar WebKit Bug Importer 2022-02-22 11:33:18 PST
<rdar://problem/89307131>