Bug 236173 – Big drop in canvas performance above 3840px

• Component: Canvas
• Status: RESOLVED FIXED

Created attachment 450976 [details]
FPS vs canvas size

**Background**
While working on a complex full-screen 2D animation utilizing multiple <canvas> layers I found out a problem with Safari. The rendering performance drops sharply after the canvas gets bigger than exactly 3840x3840. I ran multiple tests (benchmark I wrote: https://codepen.io/kiler129/full/Exbgrqp) and obtained this peculiar graph attached to this bug report.

To make this graph I used:
 - MacBook Pro (2021, 16" M1 Max version): tried Safari 15.3 (17612.4.9.1.5), Safari TP Release 139 (Safari 15.4, WebKit 17613.1.14.41.3), as well as latest build of WebKit (r289148) on Monterey (12.2 (21D49))
 - Brave running on the MBP with macOS mentioned above
 - iPad Pro (3rd, 2018), running iOS 15.0
  

**Data analysis**
 - Steady 60 FPS/realtime is kept until 3840x3840, then it drops to unusable 2-25 FPS
 - There's a correlation between larger fill % (100% vs 20%) of the canvas and FPS, as well as larger number of draw calls per frame (d/f) and FPS -> nothing unusual here, it's pretty linear
 - Buffered canvases (drawing to off-screen & then copying) are way less performant for simple operations
 - Brave, and any other Chromium-based browsers, keep steady 60 FPS basically forever (even with screen-scaled 10000x10000 plane)
 - 2018 iPad running Safari behaves identically to 2021 MBP with M1 Max, just a little bit slower overall
 - The 3840px is exactly 4K. While I have a 4K monitor disconnecting it and rebooting doesn't change the steep drop point (MBP has a 3456x2234 screen). Even assuming that the 4K resolution is cached somewhere this doesn't explain the same drop on the iPad which has nothing to do with 4K screens.

I also checked what actually takes time in Safari and it seems that the rendering to the screen and not the actual drawing on canvas is the culprit. That will explain why using buffering gives such bad results. The times for drawing are actually slower on Chromium-based browsers, yet the times are hardly correlated to anything, i.e. the calls like fillRect() are fast regardless of the size but the actual paint is getting VERY slow.


**Hardcoded size?**
This to me looks weird, like WebKit has some hardcoded 3840px boundary and switches how it renders <canvas> with 2D context above that size. Unfortunately that switch absolutely kills the performance.

After digging deeper this behavior seems ... intentional, but imho misguided. The canvas seems to be dropped from being GPU-accelerated based on HTMLCanvasElement::shouldAccelerate (link#1) due to MaximumAccelerated2dCanvasSize being set to 5120*2880 (link#2). Digging deeper this setting was originally introduced in the patch for https://bugs.webkit.org/show_bug.cgi?id=167968 (link#3). Such canvas falling off the acceleration causes a large performance drop, which isn't surprising.

I don't see the rationale behind bug 167968 as it's probably only in Radar. The limit for the accelerated canvas is documented as "5120x2880 which is the 5K resolution for 16:9" in the commit. While this kind of makes sense the resolution is insufficient to render display-scaled full-screen canvas on a 4K display (which will be ~6720x3780).

Can someone shed some light on this?
Is there a way to change the limit on the JS side?
Is there a way to tell if the canvas is accelerated from JS?


Links to code for easy clicks:
1. https://github.com/WebKit/WebKit/blob/31bd02411a15b0c59150a09677d27d721079dcbf/Source/WebCore/html/HTMLCanvasElement.cpp#L841
2. https://github.com/WebKit/WebKit/blob/6bbe48d7161ec52d6b644950cfca5883ab874d2a/Source/WebCore/page/Settings.yaml#L323
3. https://github.com/WebKit/WebKit/commit/ae4a8115b3efde63f0dd0dfa32855d82e87fb6a3

[Myles C. Maxfield]

How big would be sufficient for you?

[Radar WebKit Bug Importer]

<rdar://problem/88536159>

(In reply to Myles C. Maxfield from comment #2)
> How big would be sufficient for you?

Correct me if I'm wrong but I believe the largest supported display on any macOS hardware now will be Pro Display XDR, which can result in a framebuffer size of 2x native so 6K*2 => 12032x6768 (~81Mpx). However, this this may be too much for the most hardware to handle anyway. In comparison Blink is able to handle ~132Mpx on M1 Max before frames are dropped.

I think, seeing as a lot of people run 4K screens scaled, the size of 7680x4320 would be reasonable, allowing for @x2 on canvases.

Do you maybe happen to know why this limit exists, and is there still a valid usecase for its existence? Exposing that limit will also be useful as it took me quite a while to figure out why the app magically halves its performance when a window is resized.

[Myles C. Maxfield]

> a framebuffer size of 2x native

This is surprising. What is the reasoning for needing twice the native resolution in each dimension? Especially in 2D graphics where drawing operations are already antialiased.

[Myles C. Maxfield]

Limits like this, in general, are meant to make sure misbehaving pages can’t take all the resources on the system thereby making your computer unusable.

One path forward here might be to use different limits on different machines. Or maybe we could use something like https://developer.apple.com/documentation/metal/mtldevice/2369280-recommendedmaxworkingsetsize. Or maybe we just re-evaluate the perf&memory on current machines to see if we can just bump up the hardcoded limit.

Whatever we do, we should be careful about fingerprinting. We don’t want to give trackers more information about the user’s device. So exposing a number that’s queryable from JavaScript is probably not great.

Due to reassignment I stopped working on the project, which doesn't have a huge user-base anyway. However, now the problem is I think much bigger: Google Spreadsheets is unusable when Safari window is maximized on a 4K screen running in scaled resolution mode. The rendering performance is in single digits when scrolling through a document containing any data.

To replicate this open a new Google Spreadsheet, go somewhere reasonably far (e.g. P200) and start typing. Click a few cells and try typing - by the 2nd-3rd cell the web app becomes unusable until the window is resized to a size smaller than 3840x3840.

[Kimmo Kinnunen]

Pull request: https://github.com/WebKit/WebKit/pull/18080

[EWS]

Committed 268440@main (45446c595376): <https://commits.webkit.org/268440@main>

Reviewed commits have been landed. Closing PR #18080 and removing active labels.