Bug 212260 – requestAnimationFrame conflict with WebGL video playback

• Status: NEW
• Component: Media

[Dustin Kerstein]

Created attachment 400040 [details]
Replication video

Check out https://jsfiddle.net/9sfr8dj3 using Safari. 

This one is a little quirky. When the issue happens (about 20% of the time on my 2018 Macbook Pro), texture updates will freeze and the entire tab becomes sluggish (rAF drops to ~5fps on my device). Note that if you are unable to replicate at first using the JSFiddle, just keep hitting the 'Play Again' button until it does replicate. 

Some additional notes:

1. Happens with both video.src as URL video.src as and Blob (try toggling 'useBlob' in the JSFiddle)
2. Using setInterval/setTimeout avoids this issue (try toggling 'useRAF' in the JSFidddle)
3. Having the <video> tag visible on the page avoids this issue (try toggling 'showBothCanvasAndVideoTag' in the JSFidddle)

Let me know if there's any further debug I can get.

[Dustin Kerstein]

FYI, this can also be replicated without Three.js (using WebGL directly) - https://jsfiddle.net/bm67yk9x/

[Dustin Kerstein]

Another note - Sometimes having the Javascript developer console open can make replication harder for some reason, so I'd recommend not opening it for initial replication.

[Radar WebKit Bug Importer]

<rdar://problem/63603790>

[Simon Fraser (smfr)]

I think this is more about media playback.

[Kenneth Russell]

This is reproducible in current Safari 13.1 (15609.1.20.111.8), but appears fixed in Safari Technology Preview Release 106 (Safari 13.2, WebKit 15610.1.12.2).

Bug 205734 fixed a problem which happened intermittently in WebGL video-to-texture uploads, and could either be related, or be the root cause of this issue.

[Dustin Kerstein]

Kenneth, I just gave https://jsfiddle.net/9sfr8dj3 a try in release 106 of the Technical Preview and am able to replicate similar, though slightly different, behavior. Instead of the texture updates entirely freezing, the texture updates slow to the pace of which video.currentTime is updating. But when the issue happens (around 1/5 times for me) the playback speed also slows to a crawl (and still nearly freezes the tab). Are you able to replicate that behavior in the Technical Preview?

[Kenneth Russell]

https://jsfiddle.net/9sfr8dj3 , when "Play Again" is clicked rapidly, occasionally demonstrates some slow updating on my machine.

The pure WebGL example you provided, https://jsfiddle.net/bm67yk9x/ , works pretty reliably in Safari Technology Preview, even clicking "Play Again" rapidly.

I think this may be a library-level problem in Three's VideoTexture implementation. Could you debug into that and see whether it's failing to terminate earlier instances promptly, whether requestAnimationFrame callbacks are piling up somewhere, or similar?

[Dustin Kerstein]

I'm able to replicate the same behavior with the WebGL example + Technical Preview (the newer behavior where the textures update slowly rather than freeze entirely). And here is a slightly modified Three.js example that doesn't use VideoTexture and instead manually marks needsUpdate in the render loop - https://jsfiddle.net/hxvsjnfm - Both examples replicate for me in the Technical Preview.

[Kenneth Russell]

Let's please focus on the test cases that don't involve Three.js. Any browser engineering team needs the smallest possible test, and Three brings in a lot of additional code.

https://jsfiddle.net/bm67yk9x/ , when "Play Again" is clicked rapidly before waiting to see the rendered result, does occasionally demonstrate the slow updating and apparent high CPU consumption in Safari Technology Preview - confirmed.

[Dustin Kerstein]

K, thanks for the feedback. 

Let me know if there's anything else I can get you. FYI, I can also replicate on the WebGL example even when waiting for the video to play several loops (and then hitting Play Again). Ie. I don't have to rapidly click Play Again before it starts displaying frames to replicate.