248544 2022-11-30 11:52:40 -0800 REGRESSION (256665@main): [ Ventura ] fast/images/avif-as-image.html is a consistent image failure 2023-01-17 12:05:19 -0800 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=247704 https://bugs.webkit.org/show_bug.cgi?id=250045 InRadar P2 Normal --- 1 rackler sabouhallawa ryanhaddad sabouhallawa webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1915660 0 rackler 2022-11-30 11:52:40 -0800 Description: fast/images/avif-as-image.html There was an AVIF tests garden at 255947@main where I first saw the history of the image failure for Ventura. I observed later in the history at 255984@main that AVIF image decoding using libavif and dav1d is enabled, and that is where I see the test begin to pass on Monterey and BigSur, but fail for Ventura. Still, there was a test expectation in place for this. Finally, there was work to AVIF image animates only one loop at 256665@main where the test expectation was removed, which seemed to cause the image failure on the flakiness dashboard to appear again. History: https://results.webkit.org/?suite=layout-tests&test=fast%2Fimages%2Favif-as-image.html&platform=ios&platform=mac&limit=50000 Image Diff: https://build.webkit.org/results/Apple-Ventura-Release-WK2-Tests/257176@main%20(385)/fast/images/avif-as-image-diffs.html 1915661 1 webkit-bug-importer 2022-11-30 11:53:06 -0800 <rdar://problem/102823196> 1915686 2 rackler 2022-11-30 13:24:59 -0800 This issue can be reproduced using the command: run-webkit-tests --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --iterations 50 -f --no-retry fast/images/avif-as-image.html Result: Regressions: Unexpected image-only failures (1) fast/images/avif-as-image.html [ ImageOnlyFailure ] 1915688 3 rackler 2022-11-30 13:28:10 -0800 I have marked this test as a consistent image failure while this issue is investigated. 1915694 4 ews-feeder 2022-11-30 13:38:47 -0800 Test gardening commit 257195@main (60a1d6da7b93): <https://commits.webkit.org/257195@main> Reviewed commits have been landed. Closing PR #6985 and removing active labels. 1917368 5 sabouhallawa 2022-12-07 14:33:50 -0800 Pull request: https://github.com/WebKit/WebKit/pull/7287 1918532 6 ews-feeder 2022-12-12 10:44:03 -0800 Committed 257738@main (c7d2aef57843): <https://commits.webkit.org/257738@main> Reviewed commits have been landed. Closing PR #7287 and removing active labels. 1923539 7 rackler 2023-01-06 10:13:19 -0800 Re-opening. Looking at the history, landing 257738@main did not seem to resolve the issue. 1923694 8 sabouhallawa 2023-01-06 17:25:17 -0800 The TestExpectations have these entries for this test: LayoutTests/TestExpectations:fast/images/avif-as-image.html [ ImageOnlyFailure ] LayoutTests/platform/win/TestExpectations:fast/images/avif-as-image.html [ Skip ] LayoutTests/platform/mac/TestExpectations:[ Ventura+ ] fast/images/avif-as-image.html [ Pass ] What I wanted to say by these entries is: this test is expected to fail on all platforms and on all macOS versions except on macOS Ventura. So I am not sure why iOS says it fails even though the expectations say: "yes we expect it to fail on iOS". Also I do not know why it runs on macOS Monetary and BigSur arm64. Should I add entries in mac-ventura/TestExpectations, mac-monterey/TestExpectations and mac-ventura/TestExpectations for this file? 1923711 9 rackler 2023-01-06 18:08:49 -0800 (In reply to Said Abou-Hallawa from comment #8) > The TestExpectations have these entries for this test: > > LayoutTests/TestExpectations:fast/images/avif-as-image.html [ > ImageOnlyFailure ] > LayoutTests/platform/win/TestExpectations:fast/images/avif-as-image.html [ > Skip ] > LayoutTests/platform/mac/TestExpectations:[ Ventura+ ] > fast/images/avif-as-image.html [ Pass ] > > What I wanted to say by these entries is: this test is expected to fail on > all platforms and on all macOS versions except on macOS Ventura. > > So I am not sure why iOS says it fails even though the expectations say: > "yes we expect it to fail on iOS". Also I do not know why it runs on macOS > Monetary and BigSur arm64. Should I add entries in > mac-ventura/TestExpectations, mac-monterey/TestExpectations and > mac-ventura/TestExpectations for this file? You currently have the test expectations set up such that everything is passing on WebKit Results Database if you turn on "Filter expected results" except for mac-ventura. This is because the test expectation for Mac-ventura was removed by 257738@main. I am looking at the latest run, and the expected image is a lime square, and the actual image is a black square. https://build.webkit.org/results/Apple-Ventura-Release-AppleSilicon-WK2-Tests/258574@main%20(917)/fast/images/avif-as-image-diffs.html I'm adding the test expectation back in place while this issue is investigated. 1923712 10 ews-feeder 2023-01-06 18:16:15 -0800 Test gardening commit 258590@main (eda93cc0bc7f): <https://commits.webkit.org/258590@main> Reviewed commits have been landed. Closing PR #8342 and removing active labels. 1925388 11 sabouhallawa 2023-01-13 10:58:08 -0800 Pull request: https://github.com/WebKit/WebKit/pull/8630 1925692 12 ews-feeder 2023-01-15 11:13:24 -0800 Committed 258934@main (9bffbcdd799e): <https://commits.webkit.org/258934@main> Reviewed commits have been landed. Closing PR #8630 and removing active labels. 1926059 13 sabouhallawa 2023-01-17 12:05:19 -0800 *** Bug 250045 has been marked as a duplicate of this bug. ***