Bug 235043 – texImage2D with ImageBitmap from a video element is slow

• Status: NEW
• Component: WebGL

[Kimmo Kinnunen]

texImage2D with ImageBitmap from a video element is slow

Has multiple copies of the data
WebGL in GPUP off causes extra copies due to IPC

[Radar WebKit Bug Importer]

<rdar://problem/87687340>

[Erik Zivkovic]

This is hurting our web app in a BIG way: we have thousands of users watching live streamed video 3D projected in Safari - Every day!

It takes around 72ms to upload _ONE_ video frame which blows our entire frame budget.

[Simon Taylor]

Erik - I'm not sure how you're getting your ImageBitmap, but if you can do the texImage2d directly from a video element you'll get much better performance.

[Erik Zivkovic]

That's exactly what I'm doing

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, this.videoElement);

The performance is very bad, 70+ ms / requestAnimationFrame

[Simon Taylor]

Using RGBA as format and internal format is recommended on Safari, RGB has a bit of a performace penalty.

Your timings do seem high though - what hardware / Safari version are you seeing this with? What's the src for the video element? What's the video resolution?

It's possible you're seeing bug 236699.

This bug is specifically about ImageBitmap from a video element being slow to upload:
createImageBitmap(this.videoElement).then(bitmap => {
   gl.texImage2d(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, bitmap);
});

[Erik Zivkovic]

Using gl.RGBA made texImage2D drop from 64.6ms to 6.9ms in my latest test.

I am using a MacBook Pro (16-inch, 2019) 2,6 GHz 6-Core Intel Core i7, 32 GB 2667 MHz DDR4, AMD Radeon Pro 5300M 4 GB / Intel UHD Graphics 630 1536 MB

We are not using createImageBitmap anywhere in our code. The video source is a HLS video (about 4k resolution) and using HLS.js (MediaSource extension).

Using gl.RGBA mitigates the issue, so thanks for that!

[Erik Zivkovic]

Sorry forgot the Safari version: Version 15.3 (17612.4.9.1.8)