Bug 250314 – [WPE][WebGL][Canvas][Compositing] Threaded compositor may use a 3D canvas target FBO texture before the WebGL rendering has finished

• Status: NEW
• Component: Compositing

[Loïc Le Page]

Created attachment 464421 [details]
Example video

When rendering an heavy WebGL scene as fast as possible in a 3D canvas, the final composited image may show a black content or a partially rendered content in place of the canvas surface.

This has been observed with WPE and the threaded compositor on Linux PC with different models of NVidia cards, and on Android with different hardwares.

Logically it should happen on any configuration using the GraphicsContextGLOpenGL context to draw a 3D scene as the texture attached to the FBO target is passed to the threaded compositor without doing any synchronization first.

So, if the 3D scene rendering takes longer than the threaded compositor drawing cycle, the texture passed to the compositor doesn't contain the full scene.

Before passing the canvas FBO target texture to the threaded compositor, the client code should wait for all previous OpenGL calls to be executed by the rendering server in the current context.

The attached video shows the visual effect of this bug.

[Loïc Le Page]

Created attachment 464422 [details]
Working fix for wpe-webkit 2.38.3

[Loïc Le Page]

I've got a working fix for wpe-webkit 2.38.3 but I need to adapt it for the main branch.

[Loïc Le Page]

Fixed on main branch.
Pull Request: https://github.com/WebKit/WebKit/pull/8520

[Loïc Le Page]

You can repoduce this issue with any kind of complex 3D scene (like https://alteredqualia.com/three/examples/webgl_terrain_dynamic.html), using the WPE Minibrowser.

Depending on the 3D hardware, it may be necessary to disable vsync in the driver configuration in order to render the scene as fast as possible.

[Loïc Le Page]

After rethinking about the whole issue I've reached those conclusions:

1- The cause of the issue is that the compositor uses the WebGL canvas target texture BEFORE all drawing commands of the canvas GL context have been executed by the GL server.

2- It is happenning with the EGL renderer on NVidia card and on Android. I haven't done advanced testing on WebKitGTK or with other 3D hardware. Indeed, it seems to work well with software rendering.

3- Independently of the 3D hardware, in the OpenGL/ES specifications neither glBindFramebuffer nor glFramebufferTexture2D guaranty any kind of implicit synchronization, so without explicit synchronization code the real behavior may be up to the drivers implementation.

Conclusion: if we want to ensure frame completeness in all situations we have to add our own synchronization code.

I can see two opposite approaches:

1- We compose only on a new frame, then somebody has to wait.

1a- Or the rendering thread waits for frame completeness before giving the texture to the compositor (this is my current approach until now), in this case we block the rendering thread until the frame is complete which can block other important stuff if the rendering thread is the same as the main thread. But we allow the compositor to follow the same pace as the fastest rendered canvas in the web page.

1b- Or the compositor thread waits for the frame completeness before using the WebGL canvas target texture, in this case the compositor pace is limited to the slowest rendering canvas in the web page and if there is a fastest rendering canvas some of its frames will just be dropped.

2- We compose whenever the compositor thread wants and we just ignore the canvas that have not a new frame complete at this moment. In this case nobody waits, but if a canvas target texture is not complete when the compositor is drawing, it will redraw the same previous frame for this canvas.

From the implementation point of view, if GL fences are available all solutions 1a, 1b and 2 can be implemented. If fences are not available and only glFinish is available, only 1a can be implemented.

[Loïc Le Page]

Solution 2 in previous comment is not viable because it introduces a desynchronization between the WebGL rendering cycle and the output compositing.

When drawing in a canvas on requestAnimationFrame() on Javascript side, we expect that the actual drawing would happen before the next requestAnimationFrame() event. Solution 2 would break that contract.

So far, the optimal solution seems to be 1b if fences are available, with an automatic fallback to 1a if not.

[Loïc Le Page]

Created attachment 464471 [details]
New optimized fix for wpewebkit 2.38.3

This new patch uses the 1b approach with automatic fallback to 1a if fences are not available.

[Loïc Le Page]

Previous patch works perfectly when the final rendering target is a Window because fences are waited on the GL server side and so the final swapBuffers done by EGL ends up by waiting for all commands to be executed.

But when doing full offscreen rendering without a final Window (no swapBuffers available), we still encounter incomplete frames. In this case, I need to add a final synchronization point.

[Loïc Le Page]

Created attachment 464489 [details]
Patch for wpe-webkit 2.38.3

Implements solution 1b with automatic fallback to 1a if fences are not available.
Ensures proper synchronization if final target is not a window (full offscreen rendering).

[Zan Dobersek]

I was finally able to reproduce this with
 - NVIDIA 525.78.01
 - patched-up Cog
 - vsync disabled
 - ~only~ the birds demo

[Miguel Gomez]

This should be fixed now after https://bugs.webkit.org/show_bug.cgi?id=272663 was closed.