Bug 207267 – [ Mac wk2 ] webgpu/whlsl/ensure-proper-variable-lifetime.html is flaky ImageOnlyFailure.

• Status: NEW
• Component: WebGPU

[Jason Lawrence]

webgpu/whlsl/ensure-proper-variable-lifetime.html

Description:
This test is flaky failing on Mac wk2. There are other webgpu tests with similar results which I will relate.

History:
https://results.webkit.org/?suite=layout-tests&test=webgpu%2Fwhlsl%2Fensure-proper-variable-lifetime.html&limit=50000

Diff:
https://build.webkit.org/results/Apple-Catalina-Debug-WK2-Tests/r255810%20(2131)/webgpu/whlsl/ensure-proper-variable-lifetime-diff.png

[Radar WebKit Bug Importer]

<rdar://problem/59187793>

[Jason Lawrence]

Created attachment 389806 [details]
Update Test Expectations

[Truitt Savell]

Comment on attachment 389806 [details]
Update Test Expectations

Clearing flags on attachment: 389806

Committed r255811: <https://trac.webkit.org/changeset/255811>

[Jason Lawrence]

I am able to reproduce this issue with the following command:
run-webkit-tests xxx webgpu/whlsl/ensure-proper-variable-lifetime.html --iterations 5000 -f --exit-after-n-failures 5 -g --force

[106/5000] webgpu/whlsl/ensure-proper-variable-lifetime.html failed unexpectedly (reference mismatch)
[212/5000] webgpu/whlsl/ensure-proper-variable-lifetime.html failed unexpectedly (reference mismatch)
[259/5000] webgpu/whlsl/ensure-proper-variable-lifetime.html failed unexpectedly (reference mismatch)
[290/5000] webgpu/whlsl/ensure-proper-variable-lifetime.html failed unexpectedly (reference mismatch)
[617/5000] webgpu/whlsl/ensure-proper-variable-lifetime.html failed unexpectedly (reference mismatch)
Exiting early after 5 failures. 614 tests run.


609 tests ran as expected, 5 didn't (4386 didn't run):


Regressions: Unexpected image-only failures (1)
  webgpu/whlsl/ensure-proper-variable-lifetime.html [ ImageOnlyFailure ]

[Jason Lawrence]

I was able to reproduce the issue on 255856.

[Karl Rackler]

Description:
webgpu/whlsl/ensure-proper-variable-lifetime.html

This is a flaky imageonlyfailure.

The first image failure that I see this on the dashboard was 10/152019 at r251120, but it was not a commit that seemed relevant that would have caused this.

It appears the original test expectations were set to failure instead of imageonlyfailure.  Modifying/correcting text expectation in https://trac.webkit.org/changeset/266564/webkit

I attempted to reproduce the imageonlyfailure using the previously used reproduction steps, but was unable to reproduce using those or other combinations I tried at this time.

The test was expecting a blue square outline with white background and center, and received a white square outline with black background and center.
History:
https://results.webkit.org/?suite=layout-tests&test=webgpu%2Fwhlsl%2Fensure-proper-variable-lifetime.html&limit=50000

Diff:
https://build.webkit.org/results/Apple-Catalina-Debug-WK2-Tests/r266535%20(6417)/webgpu/whlsl/ensure-proper-variable-lifetime-diff.png