Capability to duplicate WKWebView to multiple views

• Component: WebKit2
• Status: NEW ---

[Dan]

We have scenarios that require the same web content (WKWebView) to be rendered in multiple windows of the same Mac application simultaneously, and synchronized rendering of that content between the windows with high fidelity. The aspect ratio for the web content will be the same in all views, but the scale could be different. We don't own all of the web content, so we cannot utilize state synchronization JavaScript libraries with multiple instances of WKWebView.  Some of the web content may be hardware accelerated, including videos. The user also needs to be able to interact with the web content in all windows, and for it to be capable of being accessible through VoiceOver navigation.

With Chromium we can achieve these multiple view scenarios because Chromium browser supports off-screen rendering. At a high level, Chromium is rendering from the web content process to an IOSurface that can share hardware-accelerated buffer data across multiple processes. Chromium also gives control for the application to subscribe to changes in the pixel buffer, and render the content to multiple views in the application. Some applications have even rendered Chromium web content to an OpenGL texture with custom transforms to include in a visualizer application.

For our scenario the highest priority is to have a duplicate view into the web content, with real-time synchronization. Having direct access to the pixel buffer would allow for more flexibility, but providing a new API to give a duplicate NSView for a WKWebView instance could be sufficient as long as it also allows user to interact with the content, and provide proper accessibility.

[Kevin Neal]

Thank you for filing. The appropriate engineers have been notified.

[Radar WebKit Bug Importer]

<rdar://problem/83890788>

[Tim Horton]

This is a bit tricky in the general case, because we use the system compositor, so to WebKit's eyes there isn't a single pixel buffer that you can just slap in multiple places, but a whole layer tree.

It's actually all doable in the UI-side-compositing (read: iOS family, currently) world (and done, e.g. for our debug tile indicator, which duplicates all of the layers + their contents inside itself -- see `RemoteLayerTreeDrawingAreaProxy::initializeDebugIndicator`), /except/ for video (which you specifically call out), since CALayerHost can't host the same tree in two places. macOS WebKit has the same issue, except it applies to all layers, not just video.

> as long as it also allows user to interact with the content

Ah, they'd both be interactive, too? That is an interesting additional constraint (not covered by any of the partial versions of this we have).