Bug 240113 - Allow MediaPlayerPrivateMediaStreamAVFObjC to avoid pixel conformers if IOSurfaces are not allowed
Summary: Allow MediaPlayerPrivateMediaStreamAVFObjC to avoid pixel conformers if IOSur...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebRTC (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: youenn fablet
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2022-05-05 01:32 PDT by youenn fablet
Modified: 2022-05-06 02:53 PDT (History)
11 users (show)

See Also:


Attachments
Patch (28.78 KB, patch)
2022-05-05 02:09 PDT, youenn fablet
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (29.04 KB, patch)
2022-05-05 02:42 PDT, youenn fablet
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (29.08 KB, patch)
2022-05-05 02:52 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch (29.01 KB, patch)
2022-05-05 04:23 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch for landing (29.09 KB, patch)
2022-05-05 09:34 PDT, youenn fablet
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch for landing (29.35 KB, patch)
2022-05-05 10:36 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch for landing (32.15 KB, patch)
2022-05-06 02:10 PDT, youenn fablet
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description youenn fablet 2022-05-05 01:32:13 PDT
Allow MediaPlayerPrivateMediaStreamAVFObjC to avoid pixel conformers if IOSurfaces are not allowed
Comment 1 youenn fablet 2022-05-05 02:06:06 PDT
<rdar://92629845>
Comment 2 youenn fablet 2022-05-05 02:09:21 PDT
Created attachment 458862 [details]
Patch
Comment 3 youenn fablet 2022-05-05 02:42:28 PDT
Created attachment 458863 [details]
Patch
Comment 4 youenn fablet 2022-05-05 02:52:44 PDT
Created attachment 458864 [details]
Patch
Comment 5 youenn fablet 2022-05-05 04:23:52 PDT
Created attachment 458865 [details]
Patch
Comment 6 Eric Carlson 2022-05-05 05:42:43 PDT
Comment on attachment 458865 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=458865&action=review

r=me once the bots are happy

> Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp:142
> +        m_connection->send(Messages::RemoteVideoFrameObjectHeapProxyProcessor::NewConvertedVideoFrameBuffer { { } }, 0);

It looks like SharedVideoFrameReader doesn't log errors, so it might be worth logging an error here to help debug unexpected failures.

> Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp:149
> +        m_connection->send(Messages::RemoteVideoFrameObjectHeapProxyProcessor::NewConvertedVideoFrameBuffer { { } }, 0);

Ditto, PixelBufferConformerCV also doesn't log failures.

> Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.mm:32
> +

Nit: unneeded white space
Comment 7 youenn fablet 2022-05-05 05:51:52 PDT
> It looks like SharedVideoFrameReader doesn't log errors, so it might be
> worth logging an error here to help debug unexpected failures.

Let's fix this in https://bugs.webkit.org/show_bug.cgi?id=236066, I just uploaded a patch.
Comment 8 youenn fablet 2022-05-05 05:53:17 PDT
> > Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp:149
> > +        m_connection->send(Messages::RemoteVideoFrameObjectHeapProxyProcessor::NewConvertedVideoFrameBuffer { { } }, 0);
> 
> Ditto, PixelBufferConformerCV also doesn't log failures.

Will add logging for this one.
Comment 9 Simon Fraser (smfr) 2022-05-05 08:47:48 PDT
Comment on attachment 458865 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=458865&action=review

> Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxyProcessor.cpp:145
> +    connection.send(Messages::RemoteVideoFrameObjectHeap::ConvertBuffer { WTFMove(*buffer) }, 0);
> +    m_conversionSemaphore.wait();

Why this and not just a sync message? Sync IPC has logic to avoid deadlocking, but this doesn't.
Comment 10 youenn fablet 2022-05-05 09:34:07 PDT
Created attachment 458882 [details]
Patch for landing
Comment 11 youenn fablet 2022-05-05 10:36:58 PDT
Created attachment 458888 [details]
Patch for landing
Comment 12 youenn fablet 2022-05-05 11:22:57 PDT
(In reply to Simon Fraser (smfr) from comment #9)
> Comment on attachment 458865 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=458865&action=review
> 
> > Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxyProcessor.cpp:145
> > +    connection.send(Messages::RemoteVideoFrameObjectHeap::ConvertBuffer { WTFMove(*buffer) }, 0);
> > +    m_conversionSemaphore.wait();
> 
> Why this and not just a sync message? Sync IPC has logic to avoid
> deadlocking, but this doesn't.

We send a message from main thread but we receive the answer from a work queue.
Comment 13 youenn fablet 2022-05-06 02:10:46 PDT
Created attachment 458941 [details]
Patch for landing
Comment 14 youenn fablet 2022-05-06 02:31:22 PDT
(In reply to youenn fablet from comment #12)
> (In reply to Simon Fraser (smfr) from comment #9)
> > Comment on attachment 458865 [details]
> > Patch
> > 
> > View in context:
> > https://bugs.webkit.org/attachment.cgi?id=458865&action=review
> > 
> > > Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxyProcessor.cpp:145
> > > +    connection.send(Messages::RemoteVideoFrameObjectHeap::ConvertBuffer { WTFMove(*buffer) }, 0);
> > > +    m_conversionSemaphore.wait();
> > 
> > Why this and not just a sync message? Sync IPC has logic to avoid
> > deadlocking, but this doesn't.
> 
> We send a message from main thread but we receive the answer from a work
> queue.

Thinking more about it, this will create a deadlock in case of GPUProcess crash, so we might need to use sync IPC, though I am unsure how sync IPC and work queue message receivers do things consistently.
I'll file a bug to keep track of this.
Comment 15 youenn fablet 2022-05-06 02:32:24 PDT
I filed https://bugs.webkit.org/show_bug.cgi?id=240161.
Comment 16 EWS 2022-05-06 02:53:49 PDT
Committed r293886 (250344@main): <https://commits.webkit.org/250344@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 458941 [details].