Bug 164540 – HLS WebGL video textures produces faulty colour rendering in IOS 10

• Status: RESOLVED
• Resolution: CONFIGURATION CHANGED
• Component: WebGL

[Daniel Rossi]

Regarding #163866

I am separating this into a different ticket but they might be related to the FlipY bug. 

I was able to successfully test the HLS webGL rendering on an Iphone after the FlipY work arounds. The simulator was causing similar colour faults, but when using the RGBA format flag, the output I see on the Iphone is the same regardless which RGB format is used. 

The colours are rendering incorrect. Using this code

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, gl.RGB, gl.UNSIGNED_BYTE, video);

If there is a quick fix work around rather waiting months for fixes that could be a start. 

So such HLS webgl output is only partially working on IOS 10 not IOS 9. Many devices are stuck on 9 now like my Ipad 3. I need to find a work around for 9.

http://dev.electroteque.org/webgl/threejs-hls.html
http://dev.electroteque.org/webgl/webgl.html

Thanks.

[Daniel Rossi]

Regarding this FlipY bug  ticket even 

https://bugs.webkit.org/show_bug.cgi?id=163866

[Radar WebKit Bug Importer]

<rdar://problem/29177493>

[Radar WebKit Bug Importer]

<rdar://problem/29177494>

[Daniel Rossi]

According to this comment. 

On their IDevice only the RGBA flag would get a render for them but was producing the same colourspace issue with HLS. On the IDevice I tested RGB flag was ok and also producing the colourspace problem. 

Within the simulator the RGB flag would render with green vertical bars through it and only the RGBA flag would work. 

I'm hoping there is an actual combination of flags between the colour format and the byte type that will make it work ? 

https://github.com/mrdoob/three.js/issues/9754#issuecomment-260238925

[Daniel Rossi]

Is there going to be any way to fix it ? 

IOS is pretty much locked out of Live WebVR with HLS right now. 

IOS9 seems buggered up and all older devices stuck on IOS 9 unable to get the IOS10 update are stuffed. 

I tried all possible colour format flag, and any byte flag available and no good. Only unsigned byte type and RGB RGBA colour format is possible with this. 

If it can be fixed with how the HLS is encoded and packaged that would be good to know. I can't see what the difference is between an mp4 file and a HLS packaged version of it, same encoding.

This bug is very annoying and is putting iOS completely out of VR live streaming.

Is there any timeline regarding this bug? Please fix it.

[Daniel Rossi]

Someone was lucky to figure out what it was. 

Webkit is delivering textures from HLS video with rgb colours switched around. The red and blue is switched. 

I believe they ran a test by switching the colours in the source file. This solution isn't good enough because it will be impossible to work around with live streams. 

please see commend

https://github.com/mrdoob/three.js/issues/9754#issuecomment-267840695

I believe you can do something like this to swap the red and blue colours but it doesn't work in that raw webgl demo. It needs other stuff changed

gl.shaderSource(ps, "precision mediump float;uniform sampler2D sm;varying vec2 tx;void main(){gl_FragColor=texture2D(sm,txt).bgra;}");


So 

texture2D(sm,txt)

is collecting the texture colours

and

bgra swaps the red and blue colour channel order.

[Daniel Rossi]

There was a program error simply copying and pasting. 

Using this code above. I can confirm what is being rendered is what is visible in IOS 10 so it should hopefully invert the red and blur colours that are swapped in the shader ? 

http://dev.electroteque.org/webgl/webglbgra.html

Is this easily fixable ? Is anyone out there ?

[Daniel Rossi]

Confirming rotating the colour channel order , it gets inverted back on IOS 10. I had to add this colour format to display properly in the simulator for now. 

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, video);

[Daniel Rossi]

please see fix here. Close this if you wish as I have no doubt it will ever be bothered to be fixed. 

This works around this particular bug. IOS is inverting the rgba colours on video textures for HLS. They need to be pre-inverted and it will switch them around. 

https://github.com/mrdoob/three.js/issues/9754#issuecomment-268197088

[Daniel Rossi]

IOS9 still refuses to render HLS video textures.

[Jon Lee]

(In reply to comment #10)
> please see fix here. Close this if you wish as I have no doubt it will ever
> be bothered to be fixed. 

Thanks for the pointer and for doing a lot of the investigation. But statements like this last one aren't productive to the project.

[Daniel Rossi]

A response would be appreciated. I spent months with others figuring that out. As I mentioned when it can be bothered to be fixed. I said nothing wrong. 

You may close if you wish and people can see the work around. Thanks.

[Daniel Rossi]

This is now invalid for IOS 11. Work around fix is for IOS 10 only. HLS completely broken for IOS 11 now. No possible work around yet.

[Justin Fan]

Configuration change in iOS 11.3 fixed this issue. Tested this and is all working now.