Bug 216250 - Major WebGL video texture performance regression in iOS 14 beta
Summary: Major WebGL video texture performance regression in iOS 14 beta
Status: RESOLVED DUPLICATE of bug 215908
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebGL (show other bugs)
Version: Other
Hardware: iPhone / iPad Other
: P2 Major
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-09-07 12:32 PDT by Simon Taylor
Modified: 2020-10-06 03:16 PDT (History)
10 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Taylor 2020-09-07 12:32:58 PDT
Based on the current iOS 14 beta (beta 7) the performance of texImage2d from a video element has suffered a significant regression since iOS 13.

Some examples pages that have the very simplest shader and render loop for getting a 720p video into a WebGL canvas:
https://tango-bravo.net/WebGLVideoPerformance/texImage2d_video.html (uses gl.RGB format and internal format)
https://tango-bravo.net/WebGLVideoPerformance/texImage2d_video_rgba.html (uses gl.RGBA format and internal format)

On an iPod 7th gen running iOS 13.6, I've observed the reported total average draw time under 2ms for both of these and it has no difficulty maintaining a 60FPS render loop (aside: iOS seems to downclock when hitting 60FPS; opening the app switcher and interacting with the page (scrolling, zooming, etc) looks to bring the timings down).

With an iPhone XR on the iOS 14 beta 7, things are significantly worse. With the first example the render loop drops under 60 FPS (and the bulk of the time seems to be spent in drawElements). With the RGBA format (second link) 60 FPS is maintained but the timings remain high (above 7ms).
Comment 1 Radar WebKit Bug Importer 2020-09-08 09:48:11 PDT
<rdar://problem/68513491>
Comment 2 Alexey Proskuryakov 2020-09-08 15:27:24 PDT
Could you please clarify if this affects user facing websites or apps?
Comment 3 Simon Taylor 2020-09-09 02:56:39 PDT
As demonstrated by the test case, any page making use of video textures for WebGL will suffer a performance regression.

Here's a user-facing site that is affected. It's a regression test (with a 1080p video) for our web-based AR platform that shows very poor performance in the beta on an iPhone XR, but is perfectly smooth on the iOS 13 iPod 7th gen.
https://web.zappar.com/?zid=z/wwMc1c

Both camera and video textures go via texImage2d from video elements. The black flashing background appears to be another WebGL related regression in iOS 14 that I'm currently attempting to make a minimal test case for so I can submit another bug report.

Most higher-res video content would use HLS, so clearly Bug 215908 is a bigger priority and breaks many live WebGL sites. However it seems whatever changes have been made to the WebGL video texture pipeline even outside of HLS have a significant performance impact too.
Comment 4 Dustin Kerstein 2020-09-17 17:54:09 PDT
I can also replicate this regression on https://my.panomoments.com/u/dustinkerstein/m/grand-central using a first generation iPad Pro on the iPadOS 14 GM. With the UHD and HD resolutions the render framerate is about 25-50% of what I see on iPadOS 13.
Comment 5 Dustin Kerstein 2020-09-17 18:11:26 PDT
Here is a pure WebGL-based JSFiddle that should replicate on iOS/iPadOS (though you should make sure to click into the preview window to avoid an iFrame iOS power saving feature) - https://jsfiddle.net/1obyw68k

I believe I'm also able to replicate this regression in YouTube's 360 viewer, but I no longer have a test device on iOS 13 so I can't directly compare performance.
Comment 6 Dustin Kerstein 2020-09-17 18:49:17 PDT
Here's a more generic WebGL texture upload test using a benchmark video that contains solid colors - https://jsfiddle.net/qz9ka6xm

The test uses gl.readPixels to count how many unique colors are rendered and the average upload time. I'm only able to get the 1080p test running on my iOS devices (I believe due to the aspect ratio of the 2160p file going beyond the mobile WebGL texture size limits - I can fix that if so desired), but I'm able to replicate the regression using the 1080p file:

iPadOS 14 GM on iPad Pro ~21fps 
iOS 12.4.8 on iPhone 6 Plus ~24fps
Comment 7 Dustin Kerstein 2020-09-23 09:22:33 PDT
Quick update, I can also replicate this regression in the current iPadOS 14.2 Beta (18B5052h)
Comment 8 rigel 2020-09-29 09:49:43 PDT
We are seeing this behavior as well and can be reproduced here: https://8w.8thwall.app/ios14-video-perf

I've included videos of this test site running without issue on the older iPhone XS Max running iOS. 13.7 (https://www.youtube.com/watch?v=MLNgYmj-_5s) but drops to half the frame rate on the newer
iPhone 11 running iOS 14.0.1 (https://www.youtube.com/watch?v=L-v0q7iiY7E).
Comment 9 Kimmo Kinnunen 2020-09-29 23:32:32 PDT
Thanks for the reports and testcases. This performance regression was due to video -> texture teximage2d failing the HW code path and reverting to SW conversion. It is the same root cause as bug 215908.

*** This bug has been marked as a duplicate of bug 215908 ***