Bug 215908 - REGRESSION: Textures Fail to Render in WebGL from HLS Stream [iOS 14]
Summary: REGRESSION: Textures Fail to Render in WebGL from HLS Stream [iOS 14]
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebGL (show other bugs)
Version: Other
Hardware: iPhone / iPad Other
: P1 Critical
Assignee: Kimmo Kinnunen
URL:
Keywords: InRadar
: 216250 (view as bug list)
Depends on:
Blocks: 216573 216574 216915
  Show dependency treegraph
 
Reported: 2020-08-27 18:46 PDT by rigel
Modified: 2020-10-19 12:17 PDT (History)
22 users (show)

See Also:


Attachments
Image comparing iOS 14 Beta 6 to iOS 13.6.1 (400.85 KB, image/png)
2020-08-27 18:46 PDT, rigel
no flags Details
Patch (3.13 KB, patch)
2020-09-15 11:33 PDT, Kimmo Kinnunen
no flags Details | Formatted Diff | Diff
Patch (4.81 KB, patch)
2020-09-21 12:01 PDT, Kimmo Kinnunen
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description rigel 2020-08-27 18:46:56 PDT
Created attachment 407441 [details]
Image comparing iOS 14 Beta 6 to iOS 13.6.1

Overview: As recently as iOS 14 Developer Beta 6, Safari no longer renders textures from HLS streams in WebGL. This is critical for streaming video within popular WebGL frameworks such as three.js, A-Frame, babylon.js etc. 

*Examples to Reproduce Issue*

three.js example: Visit https://8w.8thwall.app/hls-threejs on an iOS/iPadOS device running iOS 14 Developer Beta 6 or later. The rotating cube has a video texture applied from an HLS stream. The video in the top left corner is using the same HLS stream but rendering on top of the page outside WebGL.

babylon.js example: Visit https://www.babylonjs-playground.com/#9FSDC7#49 on iOS/iPadOS device running iOS 14 Developer Beta 6 or later. The plane has a video texture applied from an HLS stream.

Both examples work correctly on: 
iOS 13.6.1 Safari (iPhone 11 Pro Max)
iPadOS 13.6.1 Safari (iPad Pro 11", 2018)
Safari Technology Preview Release 112 [Safari 14.0, WebKit 15610.1.25.5.1] (MacBook Pro 15-inch, 2017; 10.15.6) 

However, both examples fail to produce a texture on iOS 14 Developer Beta 6 Safari (iPhone XR).

Here is an unlisted video comparing iOS 14 Beta 6 to iOS 13.6.1: https://youtu.be/jzjoHtADnmU
Comment 1 Taeheon Kim 2020-08-27 22:18:59 PDT
I can confirm that I also see this issue on iOS 14 Beta 6. Interestingly, this issue can also be seen on iOS 13.6 simulator but not on device.

Here's a fairly simple WebGL sample I've put together that has the issue: https://bit.ly/2EF2a6b
Comment 2 Radar WebKit Bug Importer 2020-08-29 11:32:31 PDT
<rdar://problem/68000962>
Comment 3 Alexey Proskuryakov 2020-09-01 08:54:31 PDT
Thank you for the report! Could you please clarify if this worked in previous betas?
Comment 4 rigel 2020-09-01 09:00:30 PDT
I believe this was working prior to iOS 14 Developer Beta 4.
Comment 5 Kimmo Kinnunen 2020-09-15 11:33:35 PDT
Created attachment 408839 [details]
Patch
Comment 6 EWS Watchlist 2020-09-15 11:34:23 PDT
Note that there are important steps to take when updating ANGLE. See https://trac.webkit.org/wiki/UpdatingANGLE
Comment 7 rpr_ag 2020-09-15 19:19:37 PDT
I also was able to reproduce this issue on iOS 14 Beta 6 with multiple WebGL applications that use HLS streaming

I tested the same projects on iOS 13.6 and they functioned properly.

We have many live projects which utilize HLS video playback within a WebGL context. These projects will break when iOS 14 goes fully live, so it would be very helpful if this issue were patched before that point.
Comment 8 Dean Jackson 2020-09-16 12:48:05 PDT
I'm just compiling this for iOS hardware to confirm, but I think this is good to go.
Comment 9 Dean Jackson 2020-09-16 15:37:23 PDT
Huh. Weirdly this works for me without the patch on ToT.
Comment 10 Dean Jackson 2020-09-16 15:45:20 PDT
It's a shame we don't have a test.
Comment 11 Dean Jackson 2020-09-16 15:48:10 PDT
But does not work with the patch applied! I might have screwed up but I'll need to investigate this.
Comment 12 Dean Jackson 2020-09-16 15:58:40 PDT
This was using https://8w.8thwall.app/hls-threejs.
Comment 13 Kimmo Kinnunen 2020-09-21 12:01:33 PDT
Created attachment 409291 [details]
Patch
Comment 14 Dean Jackson 2020-09-23 11:21:00 PDT
Comment on attachment 409291 [details]
Patch

We should be able to make a test case for this. There are some HLS tests in http/tests/media/hls.
Comment 15 EWS 2020-09-23 23:43:40 PDT
kkinnunen@apple.com does not have committer permissions according to https://svn.webkit.org/repository/webkit/trunk/Tools/Scripts/webkitpy/common/config/contributors.json.

Rejecting attachment 409291 [details] from commit queue.
Comment 16 Kimmo Kinnunen 2020-09-23 23:57:12 PDT
> We should be able to make a test case for this. There are some HLS tests in http/tests/media/hls.

We don't know if we need add a test that would trigger any of the bugs in here.

Primarily the problem isn't whether there is a test case.
* _All_ gpu video image conversions failed with simulator, yet no test failed. I did file a bug about this.
* SW video conversion codepath failed on simulator and on device. I did file a bug about that. When this gets worked on, we need to come back here and understand why the sw codepath failed and use the content here to add a test case if existing video/hls tests wouldn't trigger the problem.

Further more, even if simulator tests would've worked, all video tests would have failed on device on gpu codepath. I think we have a bug about improving this situation too.
Comment 17 EWS 2020-09-24 00:01:41 PDT
Committed r267520: <https://trac.webkit.org/changeset/267520>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 409291 [details].
Comment 18 Kenneth Russell 2020-09-25 16:32:50 PDT
Great diagnosis and fix Kimmo. Filed https://bugs.chromium.org/p/angleproject/issues/detail?id=5104 to track upstreaming of the ANGLE changes.
Comment 19 Kimmo Kinnunen 2020-09-29 23:32:32 PDT
*** Bug 216250 has been marked as a duplicate of this bug. ***
Comment 20 Dustin Kerstein 2020-09-30 05:15:48 PDT
Good to hear this fix also addresses bug 216250. Do you happen to know if it's possible this fix will make it into iOS 14.2? Thanks.
Comment 21 Chris Dumez 2020-09-30 07:09:08 PDT
(In reply to Dustin Kerstein from comment #20)
> Good to hear this fix also addresses bug 216250. Do you happen to know if
> it's possible this fix will make it into iOS 14.2? Thanks.

Apple does not comment on when a particular fix will be released to the public.
Comment 22 Dustin Kerstein 2020-09-30 07:17:41 PDT
(In reply to Chris Dumez from comment #21)
> Apple does not comment on when a particular fix will be released to the
> public.

Ok, thanks for the info. Do you think it'd be possible to increase the importance of this bug to P1? Based on the rubric described on https://webkit.org/bug-prioritization it seems the behavior described in 215908 and 216250 would qualify as P1.
Comment 23 Andrii Barvynko 2020-10-02 06:07:31 PDT
Hello. Faced with same issue on pixi.js. Could somebody clarify please, fix is still not available in the latest iOS 14.0.1, right? From what I can see examples from babylon and tree.js are not working as well, so, I'm wondering why is it marked as resolved and fixed?
Comment 24 Dustin Kerstein 2020-10-19 12:17:13 PDT
FYI, I believe this fix has been pulled into iOS 14.2 beta 3 (18B5072f). Just tested in a 1st generation iPad Pro and the https://8w.8thwall.app/hls-threejs test works correctly, and the performance noted in bug 216250 has significantly improved.