Bug 234659

Summary: REGRESSION (Safari 15.2): HLS/m3u8 videos no longer render in Canvas via drawImage()
Product: WebKit Reporter: AndrewK <me>
Component: CanvasAssignee: Nobody <webkit-unassigned>
Status: RESOLVED CONFIGURATION CHANGED    
Severity: Normal CC: dino, heycam, jer.noble, sabouhallawa, simon.fraser, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: Safari 15   
Hardware: All   
OS: All   

Description AndrewK 2021-12-23 23:58:54 PST
Overview:
Calling CanvasRenderingContext2D.drawImage() with a HTMLVideoElement that references a HLS/m3u8 streaming video, no longer renders in the Canvas, when the Experimental Feature "GPU Process: Canvas Rendering" is enabled. Which is enabled by default for Safari 15.2 on both MacOS and iOS/iPadOS.

On older versions of Safari (MacOS Safari 15.1 or iOS 15.1), or when this option is disabled, this video plays fine.

Steps to Reproduce: 
JSFiddle: https://jsfiddle.net/92hz5r8d/
Open the above link in Safari 15.2 (with "GPU Process: Canvas Rendering" enabled) and press play on the video in the Results panel.

Actual Results: 
The video image is not redrawn in the canvas element, only plays the video in the HTML Video element.

Expected Results:
The video should be rendered in the canvas via the drawImage() call as well.

Affected Platforms:
All versions of Safari 15.2 (tested on iOS 15.2, iPadOS 15.2, macOS Intel 10.15.7, macOS Intel BigSur 11), where "GPU Process: Canvas Rendering" is enabled by default.

Additional Information:
Switching the video source to an MP4, renders the image fine in the Canvas. 
Disabling "GPU Process: Canvas Rendering" (in Safari 15.2), also renders the video image fine.
Default behaviour worked fine in all previous Safari versions (<= 15.1), even with "GPU Process: Canvas Rendering" enabled (tested on iOS 15.1.1).

This functionality is integral to our client application, and in frameworks like the Phaser game framework.
Comment 1 Radar WebKit Bug Importer 2021-12-26 11:52:19 PST
<rdar://problem/86920338>
Comment 2 Cameron McCormack (:heycam) 2022-01-10 15:22:58 PST
Thanks for filing the bug, Andrew.  I can reproduce the problem in Safari 15.2.  I think this is fixed in trunk and when I try the JSFiddle in Safari Technology Preview 136 the drawImage call is working properly.  Could you confirm that it works in STP for you too?
Comment 3 AndrewK 2022-01-10 22:16:34 PST
(In reply to Cameron McCormack (:heycam) from comment #2)
> Thanks for filing the bug, Andrew.  I can reproduce the problem in Safari
> 15.2.  I think this is fixed in trunk and when I try the JSFiddle in Safari
> Technology Preview 136 the drawImage call is working properly.  Could you
> confirm that it works in STP for you too?

Hi Cameron, thanks for the reply. I managed to get Safari Technical Preview 137 (Safari 15.4) running on macOS Big Sur 11.6. 

I was able to see the first frame drawn in the canvas, but playing the video does not update the frame in the canvas (stays at 00:00 frame). 

Disabling "GPU Process: Canvas Rendering" in STP restores the old behaviour where every requestAnimationFrame call, the video frames are updated in the canvas. 

So the functionality has improved, but still incomplete in the latest STP. Are you seeing the same behaviour?
Comment 4 Cameron McCormack (:heycam) 2022-01-12 16:14:33 PST
(In reply to AndrewK from comment #3)
> So the functionality has improved, but still incomplete in the latest STP.
> Are you seeing the same behaviour?

I was testing STP 136 on Monterey before, but I just tried now with STP 137 on Big Sur 11.6.2 on an M1 MacBook Air, and the video is getting painted correctly to the canvas as the video plays.  And I confirmed that GPU Process Canvas rendering is enabled.
Comment 5 AndrewK 2022-01-12 22:35:53 PST
Thanks for checking. Very likely an issue on my end then, as I was testing via VM on an Intel Mac. I'll try and get access to more devices running Big Sur natively, and get back to you. Thanks Cameron
Comment 6 Cameron McCormack (:heycam) 2022-01-13 16:09:30 PST
Thanks Andrew. I'll close this bug but feel free to re-open if you find it's not just an issue on your end.