Bug 183720 – Complete OffscreenCanvas implementation

• Status: NEW
• Component: Canvas

[Zan Dobersek]

SSIA.

[Sanjay Kumar]

Is this planed for near future?

Since webkit does not support Webp, OffscreenCanvas is the only performant way to convert such images on a background thread. Not having Offscreen canvas means no Webp (or any other image format not supported y Safari).

OffScreen canvas will also enables mapping engines such as Mapbox to render tiles on background threads - significantly speeding up maps.

It would be great if Webkit were to implement Offscreen canvas - it will unlock huge potential for the web.

Thank you for considering it a priority.

[Chris Lord]

I'm working on updating/finishing Zan's patches.

[Chris Lord]

Just to note that I'm still working on this, and my current work is available on GitHub here: https://github.com/Cwiiis/webkit/tree/offscreen-canvas

Many tests are passing and the 2d functionality is fairly complete, but there's still a fair amount of work to tidy it up and start submitting patches for review. I expect some of the choices I've made will need to be inspected and re-thought by people with more knowledge of the code-base.

[Myles C. Maxfield]

Finishing OffscreenCanvas support with 2D rendering would be a fantastic addition to WebKit! Thanks for working on it! The WebKit team at Apple would be happy to help and answer any questions you have. We're really looking forward to seeing this happen!

[Simon Fraser (smfr)]

One note of caution: it's going to be very hard for us to verify, via review, that you'd done all the correct locking to make off-main-thread canvas drawing, and all the called code (text, fonts, images etc) thread-safe. Anything you can do to convince us that you got it right will help with review. "I added some locking and it seems to work" won't be entirely convincing :)

[Chris Lord]

(In reply to Simon Fraser (smfr) from comment #5)
> One note of caution: it's going to be very hard for us to verify, via
> review, that you'd done all the correct locking to make off-main-thread
> canvas drawing, and all the called code (text, fonts, images etc)
> thread-safe. Anything you can do to convince us that you got it right will
> help with review. "I added some locking and it seems to work" won't be
> entirely convincing :)

This is absolutely fair :) I've been making sure that all tests run in debug mode (there are a *lot* of thread-related asserts), but really I'm not sure how to best fully guarantee this. I'm reasonably confident that it's only the font bits that are worrisome...

Do you have any suggestions beyond relying on tests? Assuming we can get to the point where everyone is happy that there aren't any obvious concurrency issues, is this something we could land (perhaps disabled on release by default? What's the best way to go about that?) to increase testing?

[Ashley Gullen]

I'd like to point out that user gesture requirements equivalent to Chrome's "User Activation v2" are important for some OffscreenCanvas use cases. In our case we can host our entire game engine in a web worker, forwarding input events over postMessage, and posting back for certain DOM APIs not available in workers (e.g. requestFullscreen). If user activation is limited to synchronous callbacks in input events, this blocks the use of any input-restricted APIs (like requestFullscreen), which would block our use of OffscreenCanvas. With User Activation v2, the restrictions are relaxed enough that it keeps working and means we can use OffscreenCanvas. I'm not sure what the status of user gestures are in WebKit but it might be worth reviewing this for OffscreenCanvas.

[Simon Fraser (smfr)]

<rdar://39882956>

[Chris Lord]

Now that bug 224178 has landed, all major OffscreenCanvas features are now implemented, I believe, and enabled on Linux platforms. There's still a fair amount of work to solidify and optimise this implementation, and of course to enable it on all platforms, but the foundations are there. Hurray!

[Kenneth Russell]

That's great news!

Looking forward to being able to do effective off-main-thread rendering via OffscreenCanvas in WebKit!

[Jeffrey Gilbert]

Cross linking Mozilla ticket for this feature to track support level of this feature on the big 3: https://bugzilla.mozilla.org/show_bug.cgi?id=1390089

[Jen Simmons]

*** Bug 254071 has been marked as a duplicate of this bug. ***

[Michael Hagar]

I'm seeing that the WebGL context is supported on Mac OS Ventura on the main thread, but not the worker threads. This is causing a problem because we were doing the detection of the WebGL context on the main thread, and not the worker. And we are actually trying to use the WebGL context from a worker.

1. Was it intended that WebGL context support be added to Mac OS Ventura 17.1 (it wasn't in 17.0 on Ventura)

2. Can this be fixed in future releases of Safari on Ventura, so that either the WebGL context is not supported at all (main thread or worker), or the WebGL context be supported in the main thread *and* workers.

[Michael Hagar]

The specific check we're doing (on the main thread) is this:

if (typeof OffscreenCanvas !== 'function') {
      return false;
    }

    const ofs = new OffscreenCanvas(1, 1);
    if (ofs.getContext('webgl')) {
      return true;
    }

    return false;

[Ashley Gullen]

That sounds similar to a problem we had with Safari 16.4-16.6 - it shipped OffscreenCanvas but with no support for WebGL in a worker, which broke all of our feature detection. It was a big problem for us and I wrote about it here: https://www.construct.net/en/blogs/ashleys-blog-2/safari-releases-development-1616

Even though this was not web compatible Apple shipped it anyway, but added a hack for our engine only (which we're now trying to get removed in issue 263010), so it will still have broken other content like yours.

However AFAIK from Safari 17.0+ WebGL is supported in OffscreenCanvas. So things ought to be working from there.

[Kimmo Kinnunen]

>Can this be fixed in future releases of Safari on Ventura, so that either the WebGL context is not supported at all (main thread or worker), or the WebGL context be supported in the main thread *and* workers.

Given not everybody updates when the update is available, some clients would still run into the issue.

The recommended way would be to detect the Worker feature by running the feature detection in a Worker task.

OffscreenCanvas and Worker are related but orthogonal concepts, so disabling OffscreenCanvas on main thread is likely not feasible.

[Kimmo Kinnunen]

Safari 17.1 on macOS 13 Ventura does not support WebGL in Web Workers.

(You can see the setting in Settings, Advanced, Show features for web developers then Feature Flags, search for "WebGL")

[Michael Hagar]

Yes, we're now updating our code to run the OffscreenCanvas feature detection in the worker itself. 

I think my confusion was that on Safari 17.1 on Mac OS Sonoma, WebGL in Workers is supported, but it is not on Safari 17.1 on Mac OS Ventura. So I thought it was maybe a bug that WebGL in Workers is not supported on Safari 17.1 on Mac OS Ventura, but it sounds like that's the intended behavior.

I guess there's nothing for Safari to do now, but here are my thoughts:

It is confusing for developers when feature support differs by OS instead of Safari version, and doubly so when this is not mentioned in the release notes. I was going off of the release notes here: https://developer.apple.com/documentation/safari-release-notes/safari-17-release-notes#New-Features , which I took to mean that full (main thread and worker)WebGL OffscreenCanvas is supported on Safari 17+, regardless of the OS. While it is still our responsibility to test with the beta versions, I would have not thought to test on both Sonoma and Ventura for this.