Bug 239822

Summary: [ macOS Debug wk2 ] fast/css/variables/test-suite/168.html is a flaky image failure
Product: WebKit Reporter: Karl Rackler <rackler>
Component: New BugsAssignee: Karl Rackler <rackler>
Status: NEW ---    
Severity: Normal CC: dino, simon.fraser, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   

Description Karl Rackler 2022-04-27 13:27:38 PDT
Description:
fast/css/variables/test-suite/168.html

The first failure that I saw on the dashboard for Big Sur was 2/23/2022 at r290359, but this does not appear relevant to causing this issue.  The first failure that I saw on the dashboard for Monterey was 3/24/2022 at r291848, but this modified WTFString and does not appear relevant to causing this issue.  This test seems to have been flaky since the start. 

At present, I have been unable to reproduce this issue.

History:
https://results.webkit.org/?suite=layout-tests&test=fast%2Fcss%2Fvariables%2Ftest-suite%2F168.html&platform=mac&flavor=wk2&architecture=x86_64&limit=50000&style=debug

Image Diff:
https://build.webkit.org/results/Apple-Monterey-Debug-WK2-Tests/250041@main%20(1595)/fast/css/variables/test-suite/168-diffs.html
Comment 1 Radar WebKit Bug Importer 2022-04-27 13:27:53 PDT
<rdar://problem/92416836>
Comment 2 Karl Rackler 2022-04-27 13:28:31 PDT
I have marked this test as a flaky image failure while this issue is investigated.
Comment 3 Karl Rackler 2022-04-27 13:38:27 PDT
Pull request: https://github.com/WebKit/WebKit/pull/418
Comment 4 EWS 2022-04-27 13:42:02 PDT
Test gardening commit r293532 (250061@main): <https://commits.webkit.org/250061@main>

Reviewed commits have been landed. Closing PR #418 and removing active labels.
Comment 5 Karl Rackler 2022-05-11 08:10:48 PDT
It looks like the ‘failure’ in this case is the reference image is entirely black. The failing run also fails a lot of other tests in the same way, with one of the two images being black (not always the reference image).

I have unmarked this test as a flaky image failure while this issue is investigated so we do not lose test coverage.
Comment 6 Karl Rackler 2022-05-11 08:13:52 PDT
Pull request: https://github.com/WebKit/WebKit/pull/589
Comment 7 EWS 2022-05-11 08:17:50 PDT
Test gardening commit r294053 (250459@main): <https://commits.webkit.org/250459@main>

Reviewed commits have been landed. Closing PR #589 and removing active labels.
Comment 8 Karl Rackler 2022-05-11 08:51:08 PDT
Pull request: https://github.com/WebKit/WebKit/pull/591
Comment 9 EWS 2022-05-11 08:55:20 PDT
Test gardening commit r294054 (250460@main): <https://commits.webkit.org/250460@main>

Reviewed commits have been landed. Closing PR #591 and removing active labels.
Comment 10 Karl Rackler 2022-05-11 08:58:07 PDT
(In reply to Karl Rackler from comment #8)
> Pull request: https://github.com/WebKit/WebKit/pull/591

Please disregard, for another bug.