Bug 227081

Summary:

Using a video as a source for a WebGL texture is slow and hangs on iOS 15

Product:

WebKit

Reporter:

Paris Morgan <paris>

Component:

WebGL

Assignee:

Kimmo Kinnunen <kkinnunen>

Status:

RESOLVED FIXED

Severity:

Major

CC:

dino, dustin.kerstein, eric.carlson, ews-watchlist, glenn, jer.noble, kbr, kkinnunen, philipj, sergio, tony, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

Other

Hardware:

iPhone / iPad

OS:

Other

See Also:

https://bugs.webkit.org/show_bug.cgi?id=223740

Bug Depends on:

Bug Blocks:

223434

Attachments:

Description	Flags
Video of the repro app running on an iPhone12 Pro Max with iOS 15	none
Video of the repro app running on an Phone 12 Pro with iOS 14.6	none
Patch	none
Patch for landing	none

Description Paris Morgan 2021-06-16 10:34:59 PDT

Created attachment 431562 [details]
Video of the repro app running on an iPhone12 Pro Max with iOS 15

On iOS 15 using a video as a texture is very slow and hangs. This is a regression from iOS 14.6. 

Repro app: https://8w.8thwall.app/performance-test-video-texture
Repro code: https://codesandbox.io/s/video-texture-performance-test-hmybu

This sample code uses a Three.js THREE.VideoTexture as the texture for a material on a cube. Result:
- iPhone 12 Pro Max iOS 15.0: The render loop takes ~60 ms and has constant hangs with both WebGL 1 and WebGL 2, and with "WebGL via Metal" = true / false. 
- iPhone 12 Pro iOS 14.6: The render loops takes 4.2 ms with no hangs with both WebGL 1 and WebGL 2.
As noted above I have tested with WebGL 1 / 2 and "WebGL via Metal" = true / false (on iOS 15), and no combination fixes the issue on iOS 15 or causes it to repro on iOS 14.6.

Attached is a video of the repro app. Please let me know if I can provide more info, thank you.

Comment 1 Paris Morgan 2021-06-16 10:36:31 PDT

Created attachment 431563 [details]
Video of the repro app running on an Phone 12 Pro with iOS 14.6

Comment 2 Kenneth Russell 2021-06-16 12:46:34 PDT

Kimmo, would you be able to triage this since it's happening on hardware? May be related to Bug 226830, though that one seems specific to video elements created from getUserMedia.

Comment 3 Kenneth Russell 2021-06-16 12:48:01 PDT

Note that this is a key code path that was carefully ensured to retain performance in the ANGLE/GLES code path. Not sure whether the GPU process has been deployed in Safari on iOS 15, or whether this might be a regression in ANGLE/Metal.

Comment 4 Kenneth Russell 2021-06-16 12:49:33 PDT

Actually, this may be a duplicate of Bug 223740.

Comment 5 Kimmo Kinnunen 2021-06-23 04:50:37 PDT

Created attachment 432031 [details]
Patch

Comment 6 Radar WebKit Bug Importer 2021-06-23 10:35:41 PDT

<rdar://problem/79673064>

Comment 7 Eric Carlson 2021-06-23 12:07:37 PDT

Comment on attachment 432031 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=432031&action=review

This looks great to me, thanks!

> Source/WebKit/Scripts/webkit/tests/TestWithCVPixelBufferMessageReceiver.cpp:2
> + * Copyright (C) 2010-2021 Apple Inc. All rights reserved.

Probably only need `2021` for a new file.

> Source/WebKit/Scripts/webkit/tests/TestWithCVPixelBufferMessages.h:2
> + * Copyright (C) 2010-2021 Apple Inc. All rights reserved.

Ditto

> Source/WebKit/Scripts/webkit/tests/TestWithCVPixelBufferMessagesReplies.h:2
> + * Copyright (C) 2010-2021 Apple Inc. All rights reserved.

Ditto

Comment 8 Kenneth Russell 2021-06-23 16:44:53 PDT

Amazing diagnosis and fix Kimmo!

Comment 9 Kimmo Kinnunen 2021-06-24 05:37:21 PDT

Created attachment 432153 [details]
Patch for landing

Comment 10 EWS 2021-06-24 07:38:25 PDT

Committed r279221 (239107@main): <https://commits.webkit.org/239107@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 432153 [details].

Comment 11 Kimmo Kinnunen 2021-06-24 11:04:51 PDT

Thanks for the report and the test case!