Bug 227081 - Using a video as a source for a WebGL texture is slow and hangs on iOS 15
Summary: Using a video as a source for a WebGL texture is slow and hangs on iOS 15
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebGL (show other bugs)
Version: Other
Hardware: iPhone / iPad Other
: P2 Major
Assignee: Kimmo Kinnunen
URL:
Keywords: InRadar
Depends on:
Blocks: webgl2perfproblem
  Show dependency treegraph
 
Reported: 2021-06-16 10:34 PDT by Paris Morgan
Modified: 2021-10-22 11:36 PDT (History)
12 users (show)

See Also:


Attachments
Video of the repro app running on an iPhone12 Pro Max with iOS 15 (58.67 MB, video/mp4)
2021-06-16 10:34 PDT, Paris Morgan
no flags Details
Video of the repro app running on an Phone 12 Pro with iOS 14.6 (31.07 MB, video/mp4)
2021-06-16 10:36 PDT, Paris Morgan
no flags Details
Patch (39.81 KB, patch)
2021-06-23 04:50 PDT, Kimmo Kinnunen
no flags Details | Formatted Diff | Diff
Patch for landing (56.69 KB, patch)
2021-06-24 05:37 PDT, Kimmo Kinnunen
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Paris Morgan 2021-06-16 10:34:59 PDT
Created attachment 431562 [details]
Video of the repro app running on an iPhone12 Pro Max with iOS 15

On iOS 15 using a video as a texture is very slow and hangs. This is a regression from iOS 14.6. 

Repro app: https://8w.8thwall.app/performance-test-video-texture
Repro code: https://codesandbox.io/s/video-texture-performance-test-hmybu

This sample code uses a Three.js THREE.VideoTexture as the texture for a material on a cube. Result:
- iPhone 12 Pro Max iOS 15.0: The render loop takes ~60 ms and has constant hangs with both WebGL 1 and WebGL 2, and with "WebGL via Metal" = true / false. 
- iPhone 12 Pro iOS 14.6: The render loops takes 4.2 ms with no hangs with both WebGL 1 and WebGL 2.
As noted above I have tested with WebGL 1 / 2 and "WebGL via Metal" = true / false (on iOS 15), and no combination fixes the issue on iOS 15 or causes it to repro on iOS 14.6.

Attached is a video of the repro app. Please let me know if I can provide more info, thank you.
Comment 1 Paris Morgan 2021-06-16 10:36:31 PDT
Created attachment 431563 [details]
Video of the repro app running on an Phone 12 Pro with iOS 14.6
Comment 2 Kenneth Russell 2021-06-16 12:46:34 PDT
Kimmo, would you be able to triage this since it's happening on hardware? May be related to Bug 226830, though that one seems specific to video elements created from getUserMedia.
Comment 3 Kenneth Russell 2021-06-16 12:48:01 PDT
Note that this is a key code path that was carefully ensured to retain performance in the ANGLE/GLES code path. Not sure whether the GPU process has been deployed in Safari on iOS 15, or whether this might be a regression in ANGLE/Metal.
Comment 4 Kenneth Russell 2021-06-16 12:49:33 PDT
Actually, this may be a duplicate of Bug 223740.
Comment 5 Kimmo Kinnunen 2021-06-23 04:50:37 PDT
Created attachment 432031 [details]
Patch
Comment 6 Radar WebKit Bug Importer 2021-06-23 10:35:41 PDT
<rdar://problem/79673064>
Comment 7 Eric Carlson 2021-06-23 12:07:37 PDT
Comment on attachment 432031 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=432031&action=review

This looks great to me, thanks!

> Source/WebKit/Scripts/webkit/tests/TestWithCVPixelBufferMessageReceiver.cpp:2
> + * Copyright (C) 2010-2021 Apple Inc. All rights reserved.

Probably only need `2021` for a new file.

> Source/WebKit/Scripts/webkit/tests/TestWithCVPixelBufferMessages.h:2
> + * Copyright (C) 2010-2021 Apple Inc. All rights reserved.

Ditto

> Source/WebKit/Scripts/webkit/tests/TestWithCVPixelBufferMessagesReplies.h:2
> + * Copyright (C) 2010-2021 Apple Inc. All rights reserved.

Ditto
Comment 8 Kenneth Russell 2021-06-23 16:44:53 PDT
Amazing diagnosis and fix Kimmo!
Comment 9 Kimmo Kinnunen 2021-06-24 05:37:21 PDT
Created attachment 432153 [details]
Patch for landing
Comment 10 EWS 2021-06-24 07:38:25 PDT
Committed r279221 (239107@main): <https://commits.webkit.org/239107@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 432153 [details].
Comment 11 Kimmo Kinnunen 2021-06-24 11:04:51 PDT
Thanks for the report and the test case!