Another major video texImage2D performance regression has appeared with iOS 15.4 beta. Using my usual test cases from Bug 216250, featuring a 720p video: https://tango-bravo.net/WebGLVideoPerformance/texImage2d_video.html (uses gl.RGB format and internal format) https://tango-bravo.net/WebGLVideoPerformance/texImage2d_video_rgba.html (uses gl.RGBA format and internal format) On an iPhone 12 Pro with iOS 15.4 beta 3, the first one results in ~60ms per frame for texImage2D and the second one is ~40ms. I've confirmed with systrace that the CPU is busy for all that time in the WebKit content process, and running on a performance core. This is around a 20x performance regression (previously typical timings were in the 2-3ms range when CPU is running with high clocks). --- In case it's helpful: I know in the past some of these regressions have been caused by the compressed internal IOSurface formats - I wonder if it might be worth explicitly requesting a non-compressed format for AVPlayerItemVideoOutput / AVCaptureVideoDataOutput? Something like this (ideally I suppose you'd pick the best match for the source video in terms of full / video range): NSDictionary *pixBuffAttributes = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithInt:kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange], (id)kCVPixelBufferPixelFormatTypeKey, nil]; AVPlayerItemVideoOutput* output = [[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:pixBuffAttributes]; I struggled to find much about the compressed IOSurface support, but this WWDC talk is the best reference I've seen: https://developer.apple.com/videos/play/wwdc2021/10047/?time=1843
<rdar://problem/89024095>
This isn't reproducible on an iPod touch 7th gen with 15.4b3, which points towards this being another one specific to the compressed IOSurface formats.
Thanks for the report. It looks like this is due to a change in ANGLE, introduced in r286603. The compressed pixel buffers are failing validation inside IOSurfaceSurfaceMtl::ValidateAttributes().
To clarify, that was the ANGLE roll performed in Bug 220896 in early December 2021, which resolved 1+ year of divergent work. Can this bug be reproduced on macOS, perhaps on the M1 GPU? Which incoming compressed pixel format is failing validation? Was an entry dropped in ANGLE's format tables during the upstreaming process?
Created attachment 452332 [details] Patch
Note that there are important steps to take when updating ANGLE. See https://trac.webkit.org/wiki/UpdatingANGLE
Committed r290011 (247397@main): <https://commits.webkit.org/247397@main> All reviewed patches have been landed. Closing bug and clearing flags on attachment 452332 [details].
Regression test to prevent his kind of mistake from being reintroduced in future?
Oh, I see: coming in bug 236759
(In reply to Kenneth Russell from comment #4) > To clarify, that was the ANGLE roll performed in Bug 220896 in early > December 2021, which resolved 1+ year of divergent work. > > Can this bug be reproduced on macOS, perhaps on the M1 GPU? > > Which incoming compressed pixel format is failing validation? Was an entry > dropped in ANGLE's format tables during the upstreaming process? The format in question was kCVPixelFormatType_Lossless_420YpCbCr8BiPlanarVideoRange, documented (kind of) here: <https://developer.apple.com/documentation/corevideo/3746862-anonymous/kcvpixelformattype_lossless_420ypcbcr8biplanarvideorange>. The compression means that each "element" in a given plane is a block of multiple pixels. While each pixel within that element conforms correctly to the validation (two-components, each component occupies 1 byte, etc.), the element as a whole does not, and so the return value of IOSurfaceGetBytesPerElementOfPlane() is much, much larger than ANGLE expects.
Are there any known workarounds for this issue in case this makes it out in the official iOS 15.4? From the original post it seems to affect both RGB and RGBA textures, but if there are any ways to avoid this perf regression, it'd be good to post here for folks who may encounter it.
(In reply to Kenneth Russell from comment #4) > Can this bug be reproduced on macOS, perhaps on the M1 GPU? It does reproduce on my M1 Pro MBP in the latest Safari Tech Preview 140. Weirdly there it is reporting 60 FPS still and upload timings around 13ms but the video is noticeably jankier that those timings suggest. (In reply to Dustin Kerstein from comment #11) > Are there any known workarounds for this issue in case this makes it out in > the official iOS 15.4? From the original post it seems to affect both RGB > and RGBA textures, but if there are any ways to avoid this perf regression, > it'd be good to post here for folks who may encounter it. This is basically the same bug that was with us in iOS 14 up until 14.6 and I don't know of any workarounds from that time. It's a small patch so I'm guessing the chances are good it will be fixed by 15.4 final, but Apple of course will not comment on such a thing as they enjoy giving us all sleepless nights waiting for the release to drop to a few hundred million users :)
I've just checked with the 15.4 RC build on iPhone 12 Pro, and the performance regression has been resolved there, so I can sleep a little more easily waiting for the final 15.4 to drop now. On the Mac side, the latest STP 141 (from March 3rd) still suffers from this issue on M1 systems.
Created attachment 454688 [details] STP 141 on MacBook Pro with M1 Pro Attached a video of the RGBA test page in STP 141 on my M1 Pro MBP. You can see it claims to be hitting 60 FPS consistently, but the upload time is high. The actual playback does look much jankier than usual, so there is something else slightly odd happening - possibly just the decode progress being somewhat starved of CPU time by the software fallback upload path?