Bug 230488 - [ BigSur wk2 arm64 ] imported/w3c/web-platform-tests/infrastructure/reftest-wait.html is a flaky image failure
Summary: [ BigSur wk2 arm64 ] imported/w3c/web-platform-tests/infrastructure/reftest-w...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Mac (Apple Silicon) Unspecified
: P2 Normal
Assignee: Nobody
Keywords: InRadar
Depends on:
Reported: 2021-09-20 10:39 PDT by ayumi_kojima
Modified: 2021-09-22 11:05 PDT (History)
8 users (show)

See Also:

Image diff (2.72 KB, image/png)
2021-09-20 10:40 PDT, ayumi_kojima
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description ayumi_kojima 2021-09-20 10:39:49 PDT

Is flaky image failing on BigSur wk2 arm64.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Finfrastructure%2Freftest-wait.html

Result page: https://build.webkit.org/results/Apple-BigSur-Release-AppleSilicon-WK2-Tests/r282745%20(4008)/results.html
Comment 1 ayumi_kojima 2021-09-20 10:40:53 PDT
Created attachment 438694 [details]
Image diff
Comment 2 Radar WebKit Bug Importer 2021-09-20 10:41:23 PDT
Comment 3 ayumi_kojima 2021-09-20 10:46:45 PDT
Marked test expectations: https://trac.webkit.org/changeset/282759/webkit
Comment 4 Alexey Proskuryakov 2021-09-20 13:38:13 PDT
This test just checks if basic WPT infrastructure works. So if it's flaky, then all reftest-wait tests are.

So this is particularly important to look into.
Comment 5 Sam Sneddon [:gsnedders] 2021-09-21 06:34:01 PDT
It appears like it's just some pixels (not even around the edge of the square) with a very slightly different colour. Any idea, Simon?
Comment 6 ayumi_kojima 2021-09-21 13:28:17 PDT
I was able to reproduce the image failure locally on BigSur arm64 at TOT and the oldest revision available (r268271) using run-webkit-tests --force --iterations 100 --exit-after-n-failures 1 imported/w3c/web-platform-tests/infrastructure/reftest-wait.html -f
Comment 7 Simon Fraser (smfr) 2021-09-21 13:31:50 PDT
I'm not sure we even support reftest-wait.
Comment 8 Sam Sneddon [:gsnedders] 2021-09-22 10:28:14 PDT
Darin added it in https://bugs.webkit.org/show_bug.cgi?id=186045
Comment 9 Simon Fraser (smfr) 2021-09-22 11:05:47 PDT
The diff is pixel fuzz, so it's one of those bugs where some underlying graphics state is different, which might be something WebKit is doing, or something an underlying framework is doing.

It would be nice to know what sequence of tests reliably cause this pixel fuzz.