284898 2024-12-18 13:17:11 -0800 REGRESSION (272231@main): [ macOS wk1 ] fonts tests following font-cache-memory-pressure-crash.html are a flaky failure 2026-01-05 11:44:35 -0800 1 1 1 Unclassified WebKit Text WebKit Nightly Build Unspecified Unspecified RESOLVED WONTFIX https://bugs.webkit.org/show_bug.cgi?id=273212 InRadar P2 Normal --- 1 rackler gsnedders ap bfulgham fantasai.bugs jbedard repstein vitor.roriz webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 2082761 0 rackler 2024-12-18 13:17:11 -0800 Description: fonts/font-weight-invalid-crash.html fonts/ligature.html The test fonts/font-cache-memory-pressure-crash.html , was introduced at https://commits.webkit.org/272231@main. This implementation corresponded with the onset of inconsistent test results, transitioning from consistent passes to frequent flaky failures. This issue can be bisected to <commit> using the command: run-webkit-tests --debug -1 --no-retry --force --iterations=50 --clobber-old-results --child-processes 1 --test-list testlist-font-weight-invalid-crash.txt run-webkit-tests --debug -1 --no-retry --force --iterations=50 --clobber-old-results --child-processes 1 --test-list testlist-ligature.txt History: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=fonts%2Ffont-weight-invalid-crash.html&test=fonts%2Fligature.html&flavor=wk1&recent=false Diff: fonts/font-weight-invalid-crash.html -PASS - +Test passes if it doesn't crash. +ｕｎｄｅｆｉｎｅｄ Image Diff: fonts/ligature.html On view layout test results it shows 'images diff (nan%)' and both links open to 404 Not Found 2082762 1 webkit-bug-importer 2024-12-18 13:17:23 -0800 <rdar://problem/141704928> 2082770 2 ews-feeder 2024-12-18 13:30:27 -0800 Test gardening commit 288036@main (2d6e67c75819): <https://commits.webkit.org/288036@main> Reviewed commits have been landed. Closing PR #38155 and removing active labels. 2082774 3 473598 rackler 2024-12-18 13:33:30 -0800 Created attachment 473598 testlist-font-weight-invalid-crash 2082775 4 473599 rackler 2024-12-18 13:34:01 -0800 Created attachment 473599 testlist-ligature 2085202 5 rackler 2025-01-06 14:26:04 -0800 Reproduction: This issue can be bisected to 272231@main using the command: run-webkit-tests --debug -1 --no-retry --force --iterations=50 --clobber-old-results --child-processes 1 --test-list testlist-font-weight-invalid-crash.txt run-webkit-tests --debug -1 --no-retry --force --iterations=50 --clobber-old-results --child-processes 1 --test-list testlist-ligature.txt Results: fonts/ligature.html failed unexpectedly (reference didn't generate pixel results.) Unexpected flakiness: text-only failures (1) fonts/font-weight-invalid-crash.html [ Failure Pass ] Unexpected flakiness: image-only failures (1) fonts/ligature.html [ ImageOnlyFailure Pass ] 2090075 6 gsnedders 2025-01-27 18:15:40 -0800 (In reply to Karl Rackler from comment #0) > Diff: > fonts/font-weight-invalid-crash.html > -PASS > - > +Test passes if it doesn't crash. > +ｕｎｄｅｆｉｎｅｄ This unexpected output is that of fonts/font-cache-memory-pressure-crash.html; the new test added in 272231@main, and fonts/font-weight-invalid-crash.html is the immediately following test. This feels very much like DRT is giving unexpected output and/or the driver is returning the wrong data. > Image Diff: > fonts/ligature.html > On view layout test results it shows 'images diff (nan%)' and both links > open to 404 Not Found Especially with the 404, this seems like a no-output-image case (which was fixed in bug 273212). To be clear, as to ordering, we're running: Tests to skip (1) fonts/font-fallback-prefers-pictographs.html ['SKIP'] LayoutTests/platform/mac-wk1/TestExpectations:2913 webkit.org/b/273279 fonts/font-fallback-prefers-pictographs.html [ Skip ] Tests to run (14) fonts/cursive.html ['PASS'] fonts/default.html ['PASS'] fonts/fantasy.html ['PASS'] fonts/font-cache-crash.html ['PASS'] fonts/font-cache-memory-pressure-crash.html ['PASS'] fonts/font-weight-invalid-crash.html ['PASS', 'FAIL'] LayoutTests/platform/mac-wk1/TestExpectations:3012 webkit.org/b/284898 [ Ventura+ ] fonts/font-weight-invalid-crash.html [ Pass Failure ] fonts/ligature.html ['PASS', 'FAIL', 'IMAGE'] LayoutTests/platform/mac-wk1/TestExpectations:3013 webkit.org/b/284898 [ Ventura+ ] fonts/ligature.html [ Pass Failure ImageOnlyFailure ] fonts/monospace.html ['PASS'] fonts/sans-serif.html ['PASS'] fonts/serif.html ['PASS'] fonts/unicode-character-font-crash.html ['PASS'] fonts/use-typo-metrics-1.html ['PASS'] fonts/use-typo-metrics-2.html ['PASS'] fonts/valid-standard-font.html ['PASS'] So these are three adjacent tests. 2091545 7 gsnedders 2025-02-03 11:34:37 -0800 I wrote this on Slack a week ago, but: > I think someone needs to go check what DRT is actually passing to the driver next, probably looking at the value of what test ends up with in run_test in Tools/Scripts/webkitpy/port/test.py, checking what output we're getting for what test. 2141064 8 gsnedders 2025-09-04 16:12:56 -0700 *** Bug 273279 has been marked as a duplicate of this bug. *** 2141069 9 gsnedders 2025-09-04 16:18:27 -0700 https://github.com/WebKit/WebKit/pull/49978 (from bug 298004) shows this up again, with whatever follows fonts/font-cache-memory-pressure-crash.html crashing with "test sequence mismatch" (including itself, if run with --iterations=N). Let's re-garden these so we skip the test causing the flakiness, not mark all those that follow as flaky. 2141074 10 gsnedders 2025-09-04 16:31:28 -0700 Pull request: https://github.com/WebKit/WebKit/pull/50329 2141321 11 ews-feeder 2025-09-05 14:32:59 -0700 Test gardening commit 299644@main (8e9862298f6f): <https://commits.webkit.org/299644@main> Reviewed commits have been landed. Closing PR #50329 and removing active labels. 2169436 12 jbedard 2026-01-05 11:44:35 -0800 Closing bug to match associated radar 473598 2024-12-18 13:33:30 -0800 2024-12-18 13:33:30 -0800 testlist-font-weight-invalid-crash testlist-font-weight-invalid-crash.txt text/plain 80 rackler Zm9udHMvZm9udC1jYWNoZS1tZW1vcnktcHJlc3N1cmUtY3Jhc2guaHRtbApmb250cy9mb250LXdl aWdodC1pbnZhbGlkLWNyYXNoLmh0bWw= 473599 2024-12-18 13:34:01 -0800 2024-12-18 13:34:01 -0800 testlist-ligature testlist-ligature.txt text/plain 100 rackler Zm9udHMvZm9udC1jYWNoZS1tZW1vcnktcHJlc3N1cmUtY3Jhc2guaHRtbApmb250cy9mb250LXdl aWdodC1pbnZhbGlkLWNyYXNoLmh0bWwKZm9udHMvbGlnYXR1cmUuaHRtbA==