Bug 222822 – WebGL2: Video upload to SRGB8_ALPHA8 does not match upload to SRGB8

• Status: NEW
• Component: WebGL

[Brendan Duncan]

Created attachment 422412 [details]
An html+video example to test the SRGB8_ALPHA8 format for video textures

I attached a test example to demonstrate the issue. 
The test works on Chrome, and I have yet to have it fail on iOS 14.5 beta 2 Safari, but on macOS Safari Tech Preview 121 (Safari 14.2, WebKit 16612.1.4.3) the test fails about 1/10 times of refreshing the page. This is better than Firefox, which fails every time.

I iterate on the tests by refreshing the page and visually looking at the results.

The example loads a video and has 3 canvases with WebGL2 contexts. It draws the video frame into each of these canvases with different texture formats.
The first video uses the gl.SRGB8 internal format; the second uses the gl.SRGB8_ALPHA8 format; and the third uses the unmodified gl.RGBA format to use as a test basis.

Both of the SRGB8* formats should be linearizing the texture color. But when the test fails, the SRGB8_ALPHA8 version is not linearized, it's staying in the original sRGB color.

The previous version of the Safari Tech Preview, the failure rate was about 50% of the time of refreshing the page, so great progress, but there still seems to be something not quite right, maybe a race condition somewhere or a caching issue...

This (and Firefox not working) is keeping Unity from being able to use SRGB8_ALPHA8 for video textures in linear color space, we have to keep them as SRGB8 only.

[Kenneth Russell]

It'd be great to know whether this is fixed with ANGLE's Metal backend.

Maybe it's a common bug in the GPU-accelerated video-to-texture upload code, where it's not correctly setting all needed OpenGL state.

[Kimmo Kinnunen]

Thanks for the report!
The test has some state problems:
1) The test should wait until video has some data before uploading it to the texture. I get some "no video" errors.

2) The test sets up 3 shader programs and 3 textures and 3 active requestAnimationFrame callbacks. However, the rAF callbacks themselves use just the last bound texture and last bound program.

On the top of my head, I would suggest following test structure (note, I haven't thought this through so it could have some bugs but I'm sure you get the general idea):

0) Set up just one shader program
1) Wait for the video update event
2) Upload the textures from the video update event callback
3) Draw all the textures to the correct individual canvases from the video update event callback
3.1) When uploading and drawing, remember to bind the expected texture.

This would simplify when I check for the bug.

Also please mention what kind of computer you have, you can access it from "Apple menu" -> "About this Mac".

[Brendan Duncan]

Created attachment 422590 [details]
Revised RGB8_ALPHA8 tests

Three variants of the test:

srgb8_alpha8_oneFrame.html: Draws a single frame of the video, SRGB8_ALPHA8 format. Chrome displays in linear color (darkened); Safari displays in sRGB (original color).

srgb8_alpha8_video.html: Draws the video as its playing in SRGB8_ALPHA8, using ontimeupdate. ontimeupdate playback is not smooth, just demonstrating incorrect color on video.

srgb8_alpha8_compare.html: Two canvases, one drawing the video frame as SRGB8, and the second using SRGB8_ALPHA8. The SRGB8 version is correct, the SRGB8_ALPHA8 version is incorrect.

[Brendan Duncan]

Thanks for looking into it, Kimmo. I uploaded a revised version of the test, with a couple variants, whatever you need.

I put it up on my personal server if you need a quick way to test on a https server:
https://loki3d.com/test/srgb8_alpha8/srgb8_alpha8_oneFrame.html
https://loki3d.com/test/srgb8_alpha8/srgb8_alpha8_video.html
https://loki3d.com/test/srgb8_alpha8/srgb8_alpha8_compare.html

I wrapped the start in the video play promise to avoid the video not ready error, which seems to be supported on the Safari versions I tested.

One thing I noticed in this version of the test is that both MacOS and iOS always fail now. That leads me to believe the success cases I was seeing before was actually the different GL contexts getting mixed up. That seems like a bug of it's own, the different GL contexts shouldn't be affecting each other. 

My Mac details:
macOS Big Sur 11.2.2
MacBook Pro (15-inch, 2017)
Processor: 3.1 GHz Quad-Core Intel Core i7
Memory: 16 GB 2133 MHz LPDDR3
Graphics: Intel HD Graphics 630 1536 MB

[Radar WebKit Bug Importer]

<rdar://problem/75376942>

[Kimmo Kinnunen]

Thanks for the clarification.
I can repro this with MacBookPro16,1 both Intel UHD 630 and AMD Radeon Pro 5500M
I cannot repro this with iMacPro1,1

[Kimmo Kinnunen]

Originally in this bug, the report was two distinct bugs:
Symptom 1. gl.SRGB8 and gl.SRGB8_ALPHA8 video uploads look different
Symptom 2. Symptom 1. happens only sometimes.

Since the report, WebKit has now fixed the symptom 2.

The symptom 1 comes from a bug where GPU fast path does not agree to the CPU slow path. SRGB8 is the slow path, as that's not color-renderable in WebGL.

The GPU fast path renders the source video from the YUV texture to the target texture SRGB8_ALPHA8. The destination gl.SRGB8 does not affect the rendering shader. This means that sampling the destination texture produces always the same result, regardless of the destination target.

The CPU slow path reads the source video to (a few) CPU buffers, mangles the data and does real glTexImage2D. The destination gl.SRGB8 does not affect the mangling, and as such it will have the same data in glTexImage2D for SRGB8 as for RGB8. This means that sampling the destination texture produces different result for different texture targets.

So:
The GPU fast path is correct in the sense that it always produces a texture that samples similar to as how the video looks without WebGL.

The CPU slow path might be correct if WebGL would define the behaviour this way.

Naturally WebKit should produce the same result for both SRBG8 and SRGB8_ALPHA8.

A slight problem is that WebGL doesn't specify what should happen with the video to texture upload to SRGB8 or SRGB8_ALPHA8 texture.

[Kimmo Kinnunen]

Created attachment 461426 [details]
Simplified test case

[Kimmo Kinnunen]

Spec issue:
https://github.com/KhronosGroup/WebGL/issues/3472

[Kimmo Kinnunen]

*** Bug 259768 has been marked as a duplicate of this bug. ***