Bug 235600

Summary: Capturing WebGL canvas through peer connection shows the image flipped
Product: WebKit Reporter: Kimmo Kinnunen <kkinnunen>
Component: WebGLAssignee: Kimmo Kinnunen <kkinnunen>
Status: RESOLVED FIXED    
Severity: Normal CC: dino, eric.carlson, ews-watchlist, glenn, hta, jer.noble, kbr, kkinnunen, kondapallykalyan, philipj, sergio, tommyw, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: Other   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=230613
https://bugs.webkit.org/show_bug.cgi?id=235708
https://bugs.webkit.org/show_bug.cgi?id=235765
Bug Depends on:    
Bug Blocks: 232727    
Attachments:
Description Flags
WIP
ews-feeder: commit-queue-
Patch
none
Simpler version
none
Simpler version
none
For debugging
none
Patch
none
Patch
none
Patch none

Description Kimmo Kinnunen 2022-01-25 10:43:56 PST
Capturing WebGL canvas through peer connection shows the image flipped
Comment 1 Radar WebKit Bug Importer 2022-01-25 14:04:05 PST
<rdar://problem/88043325>
Comment 2 Kimmo Kinnunen 2022-01-26 09:11:43 PST
https://youennf.github.io/webrtc-tests/src/content/capture/canvas-pc/ <-- shows the bug

Note:
The newer version of this fails to show anything:
https://webrtc.github.io/samples/src/content/capture/canvas-pc/ <-- does not show stuff
Comment 3 Kimmo Kinnunen 2022-01-26 09:14:52 PST
Youenn Fablet  7:12 PM
canvas-pc is using legacy APIs that we do not support and it messes up the script. https://github.com/webrtc/samples/pull/1511
Comment 4 Kimmo Kinnunen 2022-01-26 10:29:51 PST
Created attachment 450036 [details]
WIP
Comment 5 Kimmo Kinnunen 2022-01-26 11:26:54 PST
Created attachment 450043 [details]
Patch
Comment 6 Dean Jackson 2022-01-26 14:03:18 PST
Comment on attachment 450043 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=450043&action=review

> Source/WebCore/platform/graphics/cocoa/GraphicsContextGLCocoa.mm:556
> +    // TODO: When this is moved to GraphicsContextGLCocoa, set m_mediaSampleRotationSession = nullptr.

s/TODO/FIXME/

> LayoutTests/webrtc/canvas-to-peer-connection.html:80
> +// FIXME: Tthis should be filled when 2D context supports HTMLCanvasElement.captureStream().

Typo: Tthis
Comment 7 Kimmo Kinnunen 2022-01-27 00:18:16 PST
Created attachment 450107 [details]
Simpler version
Comment 8 Kimmo Kinnunen 2022-01-27 00:43:44 PST
Created attachment 450112 [details]
Simpler version
Comment 9 youenn fablet 2022-01-27 01:19:31 PST
Comment on attachment 450112 [details]
Simpler version

View in context: https://bugs.webkit.org/attachment.cgi?id=450112&action=review

> Source/WebCore/platform/graphics/cocoa/GraphicsContextGLCocoa.mm:791
> +    return MediaSampleAVFObjC::createImageSample(WTFMove(mediaSamplePixelBuffer), MediaSampleAVFObjC::VideoRotation::None, false);

When canvas is in GPUProcess, aren't we going the RemoteGraphicsContextGLProxyCocoa::paintCompositedResultsToMediaSample code path?
It seems that RemoteGraphicsContextGLCocoa::prepareForDisplay should somehow do the flip if needed to remove the need for RemoteGraphicsContextGLProxyCocoa::paintCompositedResultsToMediaSample to make the sample as upsidedown+flipped.
Comment 10 Kimmo Kinnunen 2022-01-27 04:16:55 PST
Created attachment 450124 [details]
For debugging
Comment 11 Kimmo Kinnunen 2022-01-27 05:53:18 PST
Created attachment 450129 [details]
Patch
Comment 12 Kimmo Kinnunen 2022-01-28 05:11:48 PST
Created attachment 450227 [details]
Patch
Comment 13 Kimmo Kinnunen 2022-01-28 06:28:58 PST
Created attachment 450229 [details]
Patch
Comment 14 EWS 2022-01-29 03:50:55 PST
Committed r288789 (246566@main): <https://commits.webkit.org/246566@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 450229 [details].