Bug 205667 - REGRESSION (r253926): webgl/2.0.0/conformance2/buffers/get-buffer-sub-data.html is a flaky failure
Summary: REGRESSION (r253926): webgl/2.0.0/conformance2/buffers/get-buffer-sub-data.ht...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebGL (show other bugs)
Version: WebKit Local Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Kenneth Russell
URL:
Keywords: InRadar
Depends on: 208724
Blocks:
  Show dependency treegraph
 
Reported: 2019-12-31 14:55 PST by Alexey Proskuryakov
Modified: 2020-05-21 08:26 PDT (History)
5 users (show)

See Also:


Attachments
Patch (1.56 KB, patch)
2020-05-20 23:08 PDT, Kenneth Russell
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Proskuryakov 2019-12-31 14:55:02 PST
webgl/2.0.0/conformance2/buffers/get-buffer-sub-data.html became a flaky failure after the switch to ANGLE.

https://results.webkit.org/?suite=layout-tests&test=webgl%2F2.0.0%2Fconformance2%2Fbuffers%2Fget-buffer-sub-data.html

@@ -29,11 +29,11 @@
 [ 26: PASS ] getError was expected value: INVALID_OPERATION : after evaluating: gl.getBufferSubData(gl.ELEMENT_ARRAY_BUFFER, 0, retArray)
 [ 27: FAIL ] getError expected: INVALID_VALUE. Was INVALID_OPERATION : after evaluating: gl.getBufferSubData(gl.ARRAY_BUFFER, -1, retArray)
 [ 28: PASS ] getError was expected value: NO_ERROR : after evaluating: gl.getBufferSubData(gl.ARRAY_BUFFER, 0, retArray)
-[ 29: PASS ] areArraysEqual(retArray, [0, 0, 0, 0, 0, 0, 0, 0, 0]) is true
+[ 29: FAIL ] areArraysEqual(retArray, [0, 0, 0, 0, 0, 0, 0, 0, 0]) should be true. Was false.
 [ 30: PASS ] getError was expected value: NO_ERROR : after evaluating: gl.getBufferSubData(gl.ARRAY_BUFFER, 0, retArray)
 [ 31: PASS ] areArraysEqual(retArray, floatArray) is true
 [ 32: PASS ] getError was expected value: NO_ERROR : after evaluating: gl.getBufferSubData(gl.ARRAY_BUFFER, 0, retArray)
 [ 33: PASS ] areArraysEqual(retArray, floatArray) is true
 [ 34: PASS ] successfullyParsed is true
-[ FAIL ] 7 failures reported
+[ FAIL ] 8 failures reported
Comment 1 Radar WebKit Bug Importer 2019-12-31 14:55:13 PST
<rdar://problem/58257175>
Comment 2 Kenneth Russell 2020-01-03 14:11:48 PST
I'll investigate this locally.
Comment 3 Kenneth Russell 2020-01-03 18:20:16 PST
Can't reproduce with a built (release) WebKit inside Safari. Will try to reproduce inside the layout test runner.
Comment 4 James Darpinian 2020-01-10 17:13:09 PST
Can't reproduce on my MacBook Pro or on a MacBookAir7,2 with Intel HD Graphics 6000 and Catalina 10.15.2. I tried running the test standalone and also with the layout test runner along with other tests.
Comment 5 Justin Fan 2020-01-28 18:52:59 PST
I have access to a Catalina mac mini 8,1 for now, but first pass (1000 repeats) hasn't revealed anything yet.
Comment 6 Kenneth Russell 2020-01-28 23:04:40 PST
(In reply to Justin Fan from comment #5)
> I have access to a Catalina mac mini 8,1 for now, but first pass (1000
> repeats) hasn't revealed anything yet.

Thanks for trying. Just to confirm, you're testing with the current version of ANGLE in WebKit, and with USE_ANGLE=1?

If running this test in isolation doesn't repro, what about running the entire set of WebGL conformance tests within the layout tests?
Comment 7 Justin Fan 2020-01-29 17:48:40 PST
The last thing I did was run 25 iterations of a giant list of tests that last led up to the flaky failure on one of our bots, and yes, this was with USE_ANGLE=1 :(
Comment 8 Kenneth Russell 2020-05-20 17:48:55 PDT
Is there any way to tell whether the [ Pass Failure ] expectation for this test can be safely removed?

It's likely that the fix for Bug 208724 addressed this flakiness.
Comment 9 Alexey Proskuryakov 2020-05-20 18:46:20 PDT
Indeed, the last failure was right before that fix. The expectation can be removed.
Comment 11 Kenneth Russell 2020-05-20 23:08:39 PDT
Created attachment 399935 [details]
Patch
Comment 12 Kenneth Russell 2020-05-20 23:10:06 PDT
Thanks for confirming. I wasn't sure whether the [ Pass Failure ] expectation would mask failures as passes in the bots' history. (Do they show up as warnings?)
Comment 13 Alexey Proskuryakov 2020-05-21 08:22:37 PDT
Comment on attachment 399935 [details]
Patch

They show up as actual failures in the history, unless you check "Filter expected results" at the right.

Thank you for following through on this!
Comment 14 EWS 2020-05-21 08:26:52 PDT
Committed r261997: <https://trac.webkit.org/changeset/261997>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 399935 [details].