Bug 260460 – Slow performance when pushing ImageBitmaps to VRAM from a video source

• Status: NEW
• Component: WebGL

[Wladislav Artsimovich]

iOS struggles when pushing ImageBitmaps created from video sources to VRAM via WebGL. I have a live demo with debug timers, where you can see this: https://mirrorball.frost.kiwi/ with the source code being https://github.com/FrostKiwi/Mirrorball .
In the demo, images and videos are converted via https://developer.mozilla.org/en-US/docs/Web/API/createImageBitmap to ImageBitmaps, the modern object that is supposed to allow performant interaction with WebGL.

For static images, image Bitmaps are created here: https://github.com/FrostKiwi/Mirrorball/blob/3b3b9426f8935f1e1a7980a770f5fb97c723fe86/src/js/media.js#L261 and pushed with a brand new created texture to VRAM here: https://github.com/FrostKiwi/Mirrorball/blob/3b3b9426f8935f1e1a7980a770f5fb97c723fe86/src/js/media.js#L276

For videos, a different path is chosen to only update the texture, not recreate it. The bitmap creation happens here: https://github.com/FrostKiwi/Mirrorball/blob/3b3b9426f8935f1e1a7980a770f5fb97c723fe86/src/js/main.js#L88 and the update to the texture via texSubImage2D() happens here: https://github.com/FrostKiwi/Mirrorball/blob/3b3b9426f8935f1e1a7980a770f5fb97c723fe86/src/js/media.js#L303C9-L303C22

To see the resulting performance, you can go in the controls side menu -> Settings -> Debug -> Eruda Debug console. This enables measurement and printing of time taken to create the ImageBitmaps and the upload to VRAM. It also allows one to see the timers on iOS devices without debug consoles via https://github.com/liriliri/eruda , the small square icon appearing on the bottom of the page.

Choosing any video from the media gallery results in the following performance:
On an iPad 9th Gen, iOS Version 16.6: ~5ms to create the ImageBitmap (excellent), ~36ms to update an already existing 2048²px texture via texSubImage2D(). This unfortunately tanks all performance below 15fps. Not only that, it also makes input delay skyrocket. Even though the Video's framerate is supposed to be decoupled from the WebApps WebGL renderloop, the upload takes so long as to make the whole interface a laggy experience.

This is strange, since weak and outdated non-iOS devices do not struggle in the same manner.

This issue potentially corroborates these issues:
https://bugs.webkit.org/show_bug.cgi?id=135387
https://bugs.webkit.org/show_bug.cgi?id=223740
https://bugs.webkit.org/show_bug.cgi?id=232296

[Radar WebKit Bug Importer]

<rdar://problem/114545931>