texImage2D with ImageBitmap from a video element is slow Has multiple copies of the data WebGL in GPUP off causes extra copies due to IPC
<rdar://problem/87687340>
This is hurting our web app in a BIG way: we have thousands of users watching live streamed video 3D projected in Safari - Every day! It takes around 72ms to upload _ONE_ video frame which blows our entire frame budget.
Erik - I'm not sure how you're getting your ImageBitmap, but if you can do the texImage2d directly from a video element you'll get much better performance.
That's exactly what I'm doing gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, this.videoElement); The performance is very bad, 70+ ms / requestAnimationFrame
Using RGBA as format and internal format is recommended on Safari, RGB has a bit of a performace penalty. Your timings do seem high though - what hardware / Safari version are you seeing this with? What's the src for the video element? What's the video resolution? It's possible you're seeing bug 236699. This bug is specifically about ImageBitmap from a video element being slow to upload: createImageBitmap(this.videoElement).then(bitmap => { gl.texImage2d(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, bitmap); });
Using gl.RGBA made texImage2D drop from 64.6ms to 6.9ms in my latest test. I am using a MacBook Pro (16-inch, 2019) 2,6 GHz 6-Core Intel Core i7, 32 GB 2667 MHz DDR4, AMD Radeon Pro 5300M 4 GB / Intel UHD Graphics 630 1536 MB We are not using createImageBitmap anywhere in our code. The video source is a HLS video (about 4k resolution) and using HLS.js (MediaSource extension). Using gl.RGBA mitigates the issue, so thanks for that!
Sorry forgot the Safari version: Version 15.3 (17612.4.9.1.8)