Bug 135387 - [Mac, iOS] Video upload to texture performance way too slow
Summary: [Mac, iOS] Video upload to texture performance way too slow
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebGL (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P2 Critical
Assignee: Nobody
URL: http://krpano.com/ios/bugs/ios8-webgl...
Keywords:
Depends on:
Blocks:
 
Reported: 2014-07-29 08:12 PDT by Klaus Reinfeld
Modified: 2021-03-25 03:21 PDT (History)
12 users (show)

See Also:


Attachments
Zip containing html and video files (6.60 MB, application/zip)
2021-03-24 05:04 PDT, Marcin
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Klaus Reinfeld 2014-07-29 08:12:51 PDT
On iOS 8 beta 4 and also in the OSX 10.10 Safari the video to texture uploads are way too slow!
An iPad 3 renders only with unusable 2-3 FPS!

The WebGL rendering itself is very fast, 60 FPS on all devices, only the video frame texture uploading is too slow.

Here a simplified test example with a FPS counter:

http://krpano.com/ios/bugs/ios8-webgl-video-performance/
Comment 1 MooLee 2014-08-17 22:34:38 PDT
是的  是在是太慢了
Comment 2 Daniel Rossi 2016-07-25 23:38:10 PDT
I am on IOS9 with Ipad 3. Confirming this is a problem. 

I thought it was something I was doing. The video will play back at full frames but within webgl its a problem. 

This ticket is really old. Any ideas what is going on ?
Comment 3 Daniel Rossi 2016-07-27 07:53:00 PDT
I've added an example which includes the CORS proxy work around and a stats monitor. 

http://dev.electroteque.org/threejs/

It says 5 FPS ! 

I could try and reduce the video size and see if that helps ? 

This animation example from the stats example gives me 14 FPS compared to 60 FPS. 

http://dev.electroteque.org/webgl/examples/basic.html
Comment 4 Marcin 2020-10-29 06:48:32 PDT
It is 2020, Safari TP 115
Copying video to WebGL texture is still very slow, way slower than in other browsers (Chrome and Firefox).

Checked with: 
Safari: QuickTime[HEVC with Alpha HD (1-1-1)] 540x960
Chrome/Firefox: Webm[VP09 with Alpha] 540x960.

Both texSubImage2D and texImage2D performance is unacceptable on Safari.
Comment 5 Kimmo Kinnunen 2021-03-18 04:22:36 PDT
Thanks for the reports!
The base-case reported in this issue should now be fixed and the test case runs 60fps on iOS devices and on Mac.

Marcin, if you have time, please file a new bug, preferably with the test-case you're seeing the problem. Certain formats do have a fast path, and certain formats or conditions cause the video to texture upload to take a slower path.
Comment 6 Marcin 2021-03-23 07:50:41 PDT
I gave it a quick check:
My test uses 2 textures to ensure upload and render does not need to wait for each other, and uploads new frame every second rendered frame (video is 30fps, 540x960, QuickTime HEVC with alpha channel).

On iMac ~2017, I'm able to hit 60fps easily while tex[Sub]Imaging video but I think Safari presents only half of rendered frames. Looks like bottle neck is still there but have moved to another thread.

On the other hand, iPhone SE 2017 is definitely blocked right in the tex[Sub]Image2D call, not much changed here, max 50 fps.

And the winner is iPad Air 2020, works really well (60fps, all frames are presented), I didn't have a chance to test it with previous version of iOS.

Could it be that performance on older devices (2017) is limited by lack of hardware accelerated H265 codec?
Comment 7 Kimmo Kinnunen 2021-03-23 08:08:25 PDT
(In reply to Marcin from comment #6)
> I gave it a quick check:
> My test uses 2 textures to ensure upload and render does not need to wait
> for each other, and uploads new frame every second rendered frame (video is
> 30fps, 540x960, QuickTime HEVC with alpha channel).
> 
> On iMac ~2017, I'm able to hit 60fps easily while tex[Sub]Imaging video but
> I think Safari presents only half of rendered frames. Looks like bottle neck
> is still there but have moved to another thread.

Skipping half of the frames sounds very curious. If you have the repro case I can have a look. If not, thanks, I'll keep this in mind once I'll do next round of tests.
Comment 8 Marcin 2021-03-23 08:41:13 PDT
Sure Kimmo, I'll try to prepare a case test for you during this week.

One more thing I found, using texImage2D (instead of texSubImage2D) is way faster and non-blocking on iPhone SE! I'm able to run 60fps without presentation drops! Unfortunately iMac is still throttling.
Comment 9 Marcin 2021-03-24 05:04:45 PDT
Created attachment 424126 [details]
Zip containing html and video files
Comment 10 Marcin 2021-03-24 05:06:14 PDT
Hey Kimmo, please find attached thing.

First of all epilepsy warning, the test blinks a lot to make it easier to detect presentation drops.

Also please note blinking background temporarily burns your screen (only apple screens) it goes away after 15 mins or so. This is super interesting whats going on here but it's out of this issue scope :)

Provided video files come in two formats (mov and webm) so you can test it against chrome or firefox as well.

1) Results on my iPhone SE ~ 2017 14.4.1 and iPad Air 2020 14.4.1

   1.1 paused movie 
       - performance 60fps
       - no visual presentation drops

   1.2 playing movie with texture updates at 10FPS
       - performance 60fps
       - HEAVY presentation drops!!!!!!! why?

   1.3 playing movie with texture updates at 60FPS
       - performance 60fps
       - no visual presentation drops (interesting right?)

2) Results on my iMac ~ 2017 Catalina 10.15.7, Safari 14.2 TP 122

   1.1 paused movie 
       - performance 60fps
       - no visual presentation drops

   1.1 playing movie (texture update rate makes no difference here)
       - performance 60fps
       - random visual presentation drops, 
         (intensifies with more pressure on CPU/GPU)

Zip file contains alternate video files (puppets), results with it are much better on iMac but causes same heavy drops on iPhone.
Comment 11 Kimmo Kinnunen 2021-03-25 03:21:20 PDT
Thank you very much for the report. Added it as new bug 223740