Bug 205667 – REGRESSION (r253926): webgl/2.0.0/conformance2/buffers/get-buffer-sub-data.html is a flaky failure

• Component: WebGL
• Status: RESOLVED FIXED

[Alexey Proskuryakov]

webgl/2.0.0/conformance2/buffers/get-buffer-sub-data.html became a flaky failure after the switch to ANGLE.

https://results.webkit.org/?suite=layout-tests&test=webgl%2F2.0.0%2Fconformance2%2Fbuffers%2Fget-buffer-sub-data.html

@@ -29,11 +29,11 @@
 [ 26: PASS ] getError was expected value: INVALID_OPERATION : after evaluating: gl.getBufferSubData(gl.ELEMENT_ARRAY_BUFFER, 0, retArray)
 [ 27: FAIL ] getError expected: INVALID_VALUE. Was INVALID_OPERATION : after evaluating: gl.getBufferSubData(gl.ARRAY_BUFFER, -1, retArray)
 [ 28: PASS ] getError was expected value: NO_ERROR : after evaluating: gl.getBufferSubData(gl.ARRAY_BUFFER, 0, retArray)
-[ 29: PASS ] areArraysEqual(retArray, [0, 0, 0, 0, 0, 0, 0, 0, 0]) is true
+[ 29: FAIL ] areArraysEqual(retArray, [0, 0, 0, 0, 0, 0, 0, 0, 0]) should be true. Was false.
 [ 30: PASS ] getError was expected value: NO_ERROR : after evaluating: gl.getBufferSubData(gl.ARRAY_BUFFER, 0, retArray)
 [ 31: PASS ] areArraysEqual(retArray, floatArray) is true
 [ 32: PASS ] getError was expected value: NO_ERROR : after evaluating: gl.getBufferSubData(gl.ARRAY_BUFFER, 0, retArray)
 [ 33: PASS ] areArraysEqual(retArray, floatArray) is true
 [ 34: PASS ] successfullyParsed is true
-[ FAIL ] 7 failures reported
+[ FAIL ] 8 failures reported

[Radar WebKit Bug Importer]

<rdar://problem/58257175>

[Kenneth Russell]

I'll investigate this locally.

[Kenneth Russell]

Can't reproduce with a built (release) WebKit inside Safari. Will try to reproduce inside the layout test runner.

[James Darpinian]

Can't reproduce on my MacBook Pro or on a MacBookAir7,2 with Intel HD Graphics 6000 and Catalina 10.15.2. I tried running the test standalone and also with the layout test runner along with other tests.

[Justin Fan]

I have access to a Catalina mac mini 8,1 for now, but first pass (1000 repeats) hasn't revealed anything yet.

[Kenneth Russell]

(In reply to Justin Fan from comment #5)
> I have access to a Catalina mac mini 8,1 for now, but first pass (1000
> repeats) hasn't revealed anything yet.

Thanks for trying. Just to confirm, you're testing with the current version of ANGLE in WebKit, and with USE_ANGLE=1?

If running this test in isolation doesn't repro, what about running the entire set of WebGL conformance tests within the layout tests?

[Justin Fan]

The last thing I did was run 25 iterations of a giant list of tests that last led up to the flaky failure on one of our bots, and yes, this was with USE_ANGLE=1 :(

[Kenneth Russell]

Is there any way to tell whether the [ Pass Failure ] expectation for this test can be safely removed?

It's likely that the fix for Bug 208724 addressed this flakiness.

[Alexey Proskuryakov]

Indeed, the last failure was right before that fix. The expectation can be removed.

[Alexey Proskuryakov]

History on the bots: https://results.webkit.org/?suite=layout-tests&test=webgl%2F2.0.0%2Fconformance2%2Fbuffers%2Fget-buffer-sub-data.html&recent=false

[Kenneth Russell]

Created attachment 399935 [details]
Patch

[Kenneth Russell]

Thanks for confirming. I wasn't sure whether the [ Pass Failure ] expectation would mask failures as passes in the bots' history. (Do they show up as warnings?)

[Alexey Proskuryakov]

Comment on attachment 399935 [details]
Patch

They show up as actual failures in the history, unless you check "Filter expected results" at the right.

Thank you for following through on this!

[EWS]

Committed r261997: <https://trac.webkit.org/changeset/261997>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 399935 [details].