REGRESSION: [ Mac ] webgl/2.0.0/conformance2/glsl3/no-attribute-vertex-shader.html is failing

[Truitt Savell]

webgl/2.0.0/conformance2/glsl3/no-attribute-vertex-shader.html

Description:
This test began failing after https://trac.webkit.org/changeset/254014/webkit
This looks like just a rebase

History:
https://results.webkit.org/?suite=layout-tests&test=webgl%2F2.0.0%2Fconformance2%2Fglsl3%2Fno-attribute-vertex-shader.html

Diff:
--- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/webgl/2.0.0/conformance2/glsl3/no-attribute-vertex-shader-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/webgl/2.0.0/conformance2/glsl3/no-attribute-vertex-shader-actual.txt
@@ -1,7 +1,5 @@
 This test runs the WebGL Test listed below in an iframe and reports PASS or FAIL.
 
 Test: ../../resources/webgl_test_files/conformance2/glsl3/no-attribute-vertex-shader.html
-[ 1: FAIL ] should be green at (0, 0) expected: 0,255,0,255 was 0,0,0,0
-[ 2: PASS ] successfullyParsed is true
-[ FAIL ] 1 failures reported
+[ PASS ] All tests passed

[Radar WebKit Bug Importer]

<rdar://problem/58309241>

[Alexey Proskuryakov]

The original patch didn't touch results for this test, but the revert did. Looks like it wasn't actually a revert?

[Dean Jackson]

I'm now very confused. I did revert this test to what it was before the ANGLE default. There clearly is a bug in Apple's OpenGL where, on some hardware, gl_VertexID 0 doesn't exist.

This is also mentioned in the ANGLE code:

// This flag works around a inconsistent behavior in Mac AMD driver where gl_VertexID doesn't
// include base vertex value. It replaces gl_VertexID with (gl_VertexID + angle_BaseVertex)
// when angle_BaseVertex is available.
const ShCompileOptions SH_ADD_BASE_VERTEX_TO_VERTEX_ID = UINT64_C(1) << 48;

So it makes sense that this would fail on hardware without ANGLE (which is now the expected result). And maybe this bot doesn't have the bug, so passes the test (not expected).

But what doesn't make sense is how this only just started happening on the bots. I didn't change anything in TestExpectations about this.

[Dean Jackson]

You can test this if you change the vertex shader code to:

if (gl_VertexID == 0) {
    gl_Position = vec4(-1., 1., 0, 1);
} else if (gl_VertexID == 1) {
    gl_Position = vec4(-1., -1., 0, 1);
} else if (gl_VertexID == 2) {
    gl_Position = vec4(1., -1., 0, 1);
} else {
    gl_Position = vec4(-1, 1, 0, 1);
}

That should give you a single green triangle.

Change the else value to vec4(0), and you should still get a triangle, but you don't on some mac hardware.

[Dean Jackson]

I don't think it is worth investigating this. We're moving to ANGLE where it doesn't reproduce.

[Dean Jackson]

Expectations updated in r254035

[Alexey Proskuryakov]

The confusion is because matching -expected.txt is not always the same as passing a test.

> I'm now very confused. I did revert this test to what it was before the ANGLE default.

This is not accurate. Here is the full history.

1. Before ANGLE was enabled, the test was passing on most hardware, and had a passing -expected.txt checked in. It was failing on several models though:

iMacPro1,1
MacPro7,1
MacBookPro10,1
MacBookPro11,3
MacBookPro9,1

These are the oldest and newest AMD based devices. 

2. When ANGLE got enabled, the test started to pass everywhere, and you didn't change -expected.txt.

3. When ANGLE got rolled back, you also landed -expected.txt with failing results.

Did you generate your new results on an iMac Pro?

> I don't think it is worth investigating this. We're moving to ANGLE where it doesn't reproduce.

This is correct. Thank you for updating TestExpectations. It would be even better to roll back -expected.txt to a PASS, as that's the IG behavior.

[Justin Fan]

Two things: 

The current expectations are 1 FAIL, 1 PASS. This matches the results on iMac Pro/Catalina WK2/ANGLE.

On Mac mini 8,1/Catalina WK2/ANGLE, the results fully PASS. Thus, the bug likely lies with AMD drivers or ANGLE on the iMac Pro's AMD GPU.

[Kenneth Russell]

Note that Chromium doesn't have suppressions for this test - see:
https://source.chromium.org/chromium/chromium/src/+/master:content/test/gpu/gpu_tests/test_expectations/webgl2_conformance_expectations.txt;l=1?q=webgl2_conformance_expectations.txt

but we aren't testing on an iMac Pro. The AMD GPUs in our testing fleet are those in MacBook Pros with AMD GPUs, and Mac Pros.

[Alexey Proskuryakov]

I think that there may still be some confusion here. As described in my latest comment, this test passed for us on all hardware when ANGLE was enabled. The reason why it got flagged was due to incorrect -expected.txt landed with one of the commits.

So there is nothing to do here, except for us to update -expected.txt when enabling ANGLE again.