Bug 235043 - texImage2D with ImageBitmap from a video element is slow
Summary: texImage2D with ImageBitmap from a video element is slow
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebGL (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks: 234920
  Show dependency tree / graph
 
Reported: 2022-01-10 12:37 PST by Kimmo Kinnunen
Modified: 2022-03-07 02:00 PST (History)
6 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Kimmo Kinnunen 2022-01-10 12:37:40 PST 
texImage2D with ImageBitmap from a video element is slow

Has multiple copies of the data
WebGL in GPUP off causes extra copies due to IPC
Comment 1 Radar WebKit Bug Importer 2022-01-17 12:38:15 PST 
<rdar://problem/87687340>
Comment 2 Erik Zivkovic 2022-03-05 04:45:40 PST 
This is hurting our web app in a BIG way: we have thousands of users watching live streamed video 3D projected in Safari - Every day!

It takes around 72ms to upload _ONE_ video frame which blows our entire frame budget.
Comment 3 Simon Taylor 2022-03-06 07:05:27 PST 
Erik - I'm not sure how you're getting your ImageBitmap, but if you can do the texImage2d directly from a video element you'll get much better performance.
Comment 4 Erik Zivkovic 2022-03-06 09:42:56 PST 
That's exactly what I'm doing

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, this.videoElement);

The performance is very bad, 70+ ms / requestAnimationFrame
Comment 5 Simon Taylor 2022-03-07 00:46:12 PST 
Using RGBA as format and internal format is recommended on Safari, RGB has a bit of a performace penalty.

Your timings do seem high though - what hardware / Safari version are you seeing this with? What's the src for the video element? What's the video resolution?

It's possible you're seeing bug 236699.

This bug is specifically about ImageBitmap from a video element being slow to upload:
createImageBitmap(this.videoElement).then(bitmap => {
   gl.texImage2d(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, bitmap);
});
Comment 6 Erik Zivkovic 2022-03-07 01:58:16 PST 
Using gl.RGBA made texImage2D drop from 64.6ms to 6.9ms in my latest test.

I am using a MacBook Pro (16-inch, 2019) 2,6 GHz 6-Core Intel Core i7, 32 GB 2667 MHz DDR4, AMD Radeon Pro 5300M 4 GB / Intel UHD Graphics 630 1536 MB

We are not using createImageBitmap anywhere in our code. The video source is a HLS video (about 4k resolution) and using HLS.js (MediaSource extension).

Using gl.RGBA mitigates the issue, so thanks for that!
Comment 7 Erik Zivkovic 2022-03-07 02:00:05 PST 
Sorry forgot the Safari version: Version 15.3 (17612.4.9.1.8)