[GTK] Software-only basic compositing

• Component: WebKitGTK
• Status: RESOLVED WONTFIX

[Emanuele Aina]

At the moment WebKit depends on OpenGL until bug 146511 gets fixed. Unfortunately, disabling OpenGL means disabling composition too.

On our old 2.4.1-based branch for the Raspberry Pi we had some features and some really ugly hacks in place to reduce memory copies as much as possible, since those where one of the most severe performance bottlenecks on a device with a rather slow memory bus: we enabled the Tiled Backing Store with a Cairo backend to improve scrolling responsiveness, and we punched some holes through several layers to make sure video frames get blit to screen directly, without hitting the backing store first.

Now that we're (finally!) moving to the modern world I'd like to get rid of such hacks: my very high level idea would be to have some form of restricted compositing where only rectangular, non-transformed, fully-opaque areas are promoted to layers. This way, compositing would roughly consist in a series of clipped memcpys on the windowing surface.

The main use-case I'd like to target is video playing: with our hacked up branch at the moment we're able to play 720p videos on the Pi 1 and I'd like to retain such ability while getting rid of the hacks. What currently slows things down is that without compositing each frame is copied to the backing store first and only then blit to windowing surface, doubling the amount of memcpy'ed data for each drawing loop.

Another use-case that would benefit from such sort of composition would be the scrolling of the vast majority of pages with position:fixed headers or sidebars.

Implementation-wise I started looking at reviving the TextureMapperImageBuffer that was removed in http://trac.webkit.org/changeset/183807 (see https://lists.webkit.org/pipermail/webkit-dev/2015-April/027384.html for some context).

I'm now a bit confused about which infrastructure I should use to develop such feature, in LayerTreeHost.cpp[1] there seems to be three choices: USE(COORDINATED_GRAPHICS_MULTIPROCESS), USE(COORDINATED_GRAPHICS_THREADED) and USE(TEXTURE_MAPPER_GL). LayerTreeHostGtk assumes GL, so I guess I  will need to add a software-only implementation. I'm not sure about the status of Coordinated Graphics and the GTK+ port: building with -DENABLE_THREADED_COMPOSITOR on my amd64 laptop failed due to a mismatch with the ThreadedCoordinatedLayerTreeHost::create() prototype. :/

I guess that Coordinated Graphics may be more interesting for me: it already has a Tiled Backing Store implementation and the prospect of being able to use the threaded compositor to use the four cores of the Pi 2 to render the layers is very interesting (we have a special multi-threading feature where we render to a Cairo recording surface and dispatch the real drawing to a thread pool for each tile).

Any guidance is appreciated! :)

[1] https://trac.webkit.org/browser/trunk/Source/WebKit2/WebProcess/WebPage/LayerTreeHost.cpp?rev=187332#L41

[Emanuele Aina]

From what I can see, the only thing that uses GL directly with Coordinated Graphics is ThreadedCompositor.cpp, so I guess I should start there, replacing GL usage with GraphicsContext.

Yoon, if you have any moment to spare, any suggestion would be welcome. :)

[Gwang Yoon Hwang]

(In reply to comment #1)
> From what I can see, the only thing that uses GL directly with Coordinated
> Graphics is ThreadedCompositor.cpp, so I guess I should start there,
> replacing GL usage with GraphicsContext.
> 
> Yoon, if you have any moment to spare, any suggestion would be welcome. :)


Plz blame my laziness. I fixed build of threaded compositor.

Anyway, It looks like I couldn't understand your use cases properly.
Why not using GL accelerated compositing in RPi2?
AFAIK, it has enough hardware power.

If you want to use threaded-compositor with TextureMapperImageBuffer,
You don't have to do lots of things.

just do not use ensureGLContext and glContext()->swapBuffers();
and make a GraphicsContext from ThreadedCompositor:m_nativeSurfaceHandle and
pass it to TextureMapperImageBuffer implementation.

However, I think it is enough to use HW accelerated compositing in RPi2.

[Emanuele Aina]

> Plz blame my laziness. I fixed build of threaded compositor.

Ah ah, no worries, I just picked an unfortunate timing. :)

> Anyway, It looks like I couldn't understand your use cases properly. Why not using GL accelerated compositing in RPi2? AFAIK, it has enough hardware power.

The current (closed) GL stack has been deemed not reliable enough for WebKit. It works well enough for the limited usage seen in Kodi/XBMC, but WebKit may stress it too much and it has been decided that we will need to get away without GL until the new open stack based on Mesa will be viable.

> If you want to use threaded-compositor with TextureMapperImageBuffer,
> You don't have to do lots of things.
> just do not use ensureGLContext and glContext()->swapBuffers();
> and make a GraphicsContext from ThreadedCompositor:m_nativeSurfaceHandle and
pass it to TextureMapperImageBuffer implementation.

Indeed that was enough to get the threaded-compositor to work without GL when not actually compositing layers, now I've got position:fixed elements on their own layers (but with a lot of flickering probably due to overdraw) and will look at <video> elements once I have a clear understanding of what causes the flickering.

Thanks!

[Emanuele Aina]

Just a quick update: I've currently got it running with opaque position:fixed elements, I still need to fix elements with alpha, clean up the hugely messy patch and after that I will start looking at <video>.

[Emanuele Aina]

Created attachment 262186 [details]
Patch

[Emanuele Aina]

Created attachment 262187 [details]
Reintroduce TextureMapperImageBuffer

I submitted the current work in progress rough patch to have some sort of software only compositing using the threaded compositor.

The patch needs the reintroduction of TextureMapperImageBuffer to be applied first, should I upload it as a separate bug?

Comments on the general approach would be very welcome. :)

[Zan Dobersek]

(In reply to comment #3)
> > Plz blame my laziness. I fixed build of threaded compositor.
> 
> Ah ah, no worries, I just picked an unfortunate timing. :)
> 
> > Anyway, It looks like I couldn't understand your use cases properly. Why not using GL accelerated compositing in RPi2? AFAIK, it has enough hardware power.
> 
> The current (closed) GL stack has been deemed not reliable enough for
> WebKit. It works well enough for the limited usage seen in Kodi/XBMC, but
> WebKit may stress it too much and it has been decided that we will need to
> get away without GL until the new open stack based on Mesa will be viable.
> 

Can you roughly estimate how long this software compositing backend would have to be maintained in trunk, i.e. how long until you'd be able to switch back to OpenGL-based implementation?

For the last few years the desire was to move over completely towards depending on hardware acceleration for compositing, which is why TextureMapperImageBuffer was removed in the first place.

[Emanuele Aina]

My current understanding is that we're still one or two year from having a GL implementation we can rely on.

The current patch didn't turn out to be extremely invasive: I guess it can also benefit from a couple of refinements to further lower its impact if the approach chosen seems sensible.

[Emanuele Aina]

Interest in this a dried up, and hopefully some sort of usable, free GL support for the Raspberry Pi should be available soon. Closing.

[Michael Catanzaro]

Comment on attachment 262186 [details]
Patch

At the web engines hackfest, you mentioned that you're no longer interested in software-only compositing. For a better future....