Bug 135387 – [Mac, iOS] Video upload to texture performance way too slow

• Component: WebGL
• Status: RESOLVED FIXED

[Klaus Reinfeld]

On iOS 8 beta 4 and also in the OSX 10.10 Safari the video to texture uploads are way too slow!
An iPad 3 renders only with unusable 2-3 FPS!

The WebGL rendering itself is very fast, 60 FPS on all devices, only the video frame texture uploading is too slow.

Here a simplified test example with a FPS counter:

http://krpano.com/ios/bugs/ios8-webgl-video-performance/

[MooLee]

是的  是在是太慢了

[Daniel Rossi]

I am on IOS9 with Ipad 3. Confirming this is a problem. 

I thought it was something I was doing. The video will play back at full frames but within webgl its a problem. 

This ticket is really old. Any ideas what is going on ?

[Daniel Rossi]

I've added an example which includes the CORS proxy work around and a stats monitor. 

http://dev.electroteque.org/threejs/

It says 5 FPS ! 

I could try and reduce the video size and see if that helps ? 

This animation example from the stats example gives me 14 FPS compared to 60 FPS. 

http://dev.electroteque.org/webgl/examples/basic.html

[Marcin]

It is 2020, Safari TP 115
Copying video to WebGL texture is still very slow, way slower than in other browsers (Chrome and Firefox).

Checked with: 
Safari: QuickTime[HEVC with Alpha HD (1-1-1)] 540x960
Chrome/Firefox: Webm[VP09 with Alpha] 540x960.

Both texSubImage2D and texImage2D performance is unacceptable on Safari.

[Kimmo Kinnunen]

Thanks for the reports!
The base-case reported in this issue should now be fixed and the test case runs 60fps on iOS devices and on Mac.

Marcin, if you have time, please file a new bug, preferably with the test-case you're seeing the problem. Certain formats do have a fast path, and certain formats or conditions cause the video to texture upload to take a slower path.

[Marcin]

I gave it a quick check:
My test uses 2 textures to ensure upload and render does not need to wait for each other, and uploads new frame every second rendered frame (video is 30fps, 540x960, QuickTime HEVC with alpha channel).

On iMac ~2017, I'm able to hit 60fps easily while tex[Sub]Imaging video but I think Safari presents only half of rendered frames. Looks like bottle neck is still there but have moved to another thread.

On the other hand, iPhone SE 2017 is definitely blocked right in the tex[Sub]Image2D call, not much changed here, max 50 fps.

And the winner is iPad Air 2020, works really well (60fps, all frames are presented), I didn't have a chance to test it with previous version of iOS.

Could it be that performance on older devices (2017) is limited by lack of hardware accelerated H265 codec?

[Kimmo Kinnunen]

(In reply to Marcin from comment #6)
> I gave it a quick check:
> My test uses 2 textures to ensure upload and render does not need to wait
> for each other, and uploads new frame every second rendered frame (video is
> 30fps, 540x960, QuickTime HEVC with alpha channel).
> 
> On iMac ~2017, I'm able to hit 60fps easily while tex[Sub]Imaging video but
> I think Safari presents only half of rendered frames. Looks like bottle neck
> is still there but have moved to another thread.

Skipping half of the frames sounds very curious. If you have the repro case I can have a look. If not, thanks, I'll keep this in mind once I'll do next round of tests.

[Marcin]

Sure Kimmo, I'll try to prepare a case test for you during this week.

One more thing I found, using texImage2D (instead of texSubImage2D) is way faster and non-blocking on iPhone SE! I'm able to run 60fps without presentation drops! Unfortunately iMac is still throttling.

[Marcin]

Created attachment 424126 [details]
Zip containing html and video files

[Marcin]

Hey Kimmo, please find attached thing.

First of all epilepsy warning, the test blinks a lot to make it easier to detect presentation drops.

Also please note blinking background temporarily burns your screen (only apple screens) it goes away after 15 mins or so. This is super interesting whats going on here but it's out of this issue scope :)

Provided video files come in two formats (mov and webm) so you can test it against chrome or firefox as well.

1) Results on my iPhone SE ~ 2017 14.4.1 and iPad Air 2020 14.4.1

   1.1 paused movie 
       - performance 60fps
       - no visual presentation drops

   1.2 playing movie with texture updates at 10FPS
       - performance 60fps
       - HEAVY presentation drops!!!!!!! why?

   1.3 playing movie with texture updates at 60FPS
       - performance 60fps
       - no visual presentation drops (interesting right?)

2) Results on my iMac ~ 2017 Catalina 10.15.7, Safari 14.2 TP 122

   1.1 paused movie 
       - performance 60fps
       - no visual presentation drops

   1.1 playing movie (texture update rate makes no difference here)
       - performance 60fps
       - random visual presentation drops, 
         (intensifies with more pressure on CPU/GPU)

Zip file contains alternate video files (puppets), results with it are much better on iMac but causes same heavy drops on iPhone.

[Kimmo Kinnunen]

Thank you very much for the report. Added it as new bug 223740