Bug 261306
| Summary: | REGRESSION(267516@main): [ macOS ] 25 tests are constant failures following Sonoma OpenSource expectations migration. | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Ben Schwartz <ben_schwartz> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | RESOLVED FIXED | ||
| Severity: | Normal | CC: | Morningstar, webkit-bot-watchers-bugzilla, webkit-bug-importer |
| Priority: | P2 | Keywords: | InRadar |
| Version: | WebKit Nightly Build | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| See Also: |
https://bugs.webkit.org/show_bug.cgi?id=259373 https://bugs.webkit.org/show_bug.cgi?id=263982 https://bugs.webkit.org/show_bug.cgi?id=263987 https://bugs.webkit.org/show_bug.cgi?id=267439 |
||
Ben Schwartz
The following tests are consistent/constant failures on macOS Monterey (both wk1 and 2):
http/tests/appcache/fail-on-update-2.html
The following tests are constant failures on macOS Monterey+, wk1:
fast/scrolling/mac/scrollbars/scrollbar-state.html
fast/scrolling/mac/scrollbars/overlay-scrollbar-state.html
The following tests are constant failures on macOS Ventura, wk1:
editing/caret/insert-paragraph-does-not-paint-stale-carets.html
editing/input/editable-container-with-word-wrap-normal.html
editing/inserting/caret-position.html
editing/mac/input/caret-primary-bidi.html
editing/mac/input/firstrectforcharacterrange-plain.html
editing/mac/input/firstrectforcharacterrange-styled.html
editing/pasteboard/interchange-newline-2.html
editing/selection/5825350-2.html
editing/selection/block-cursor-overtype-mode.html
editing/selection/caret-in-empty-inline-1.html
editing/selection/caret-in-empty-inline-2.html
editing/selection/internal-caret-rect.html
editing/selection/mixed-editability-10.html
editing/selection/simple-line-layout-caret-is-gone.html
fast/forms/auto-fill-button/input-credit-card-auto-fill-button.html
imported/w3c/web-platform-tests/mathml/relations/css-styling/display-2.html
media/media-can-play-mpeg4-video.html
imported/w3c/web-platform-tests/xhr/open-method-case-sensitive.htm
media/video-source-type-params.html
The following tests are constant failures on macOS Ventura (x86_64 only), wk2.
scrollingcoordinator/mac/fixed-backgrounds/fixed-background-in-overflow-in-iframe.html (x86_64 only)
HISTORY:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=editing%2Fcaret%2Finsert-paragraph-does-not-paint-stale-carets.html&test=editing%2Finput%2Feditable-container-with-word-wrap-normal.html&test=editing%2Finserting%2Fcaret-position.html&test=editing%2Fmac%2Finput%2Fcaret-primary-bidi.html&test=editing%2Fmac%2Finput%2Ffirstrectforcharacterrange-plain.html&test=editing%2Fmac%2Finput%2Ffirstrectforcharacterrange-styled.html&test=editing%2Fpasteboard%2Finterchange-newline-2.html&test=editing%2Fselection%2F5825350-2.html&test=editing%2Fselection%2Fblock-cursor-overtype-mode.html&test=editing%2Fselection%2Fcaret-in-empty-inline-1.html&test=editing%2Fselection%2Fcaret-in-empty-inline-2.html&test=editing%2Fselection%2Finternal-caret-rect.html&test=editing%2Fselection%2Fmixed-editability-10.html&test=editing%2Fselection%2Fsimple-line-layout-caret-is-gone.html&test=fast%2Fforms%2Fauto-fill-button%2Finput-credit-card-auto-fill-button.html&test=fast%2Fscrolling%2Fmac%2Fscrollbars%2Foverlay-scrollbar-state.html&test=fast%2Fscrolling%2Fmac%2Fscrollbars%2Fscrollbar-state.html&test=http%2Ftests%2Fappcache%2Ffail-on-update-2.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fmathml%2Frelations%2Fcss-styling%2Fdisplay-2.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fxhr%2Fopen-method-case-sensitive.htm&test=media%2Fmedia-can-play-mpeg4-video.html&test=media%2Fvideo-source-type-params.html&test=scrollingcoordinator%2Fmac%2Ffixed-backgrounds%2Ffixed-background-in-overflow-in-iframe.html&platform=mac
REPRODUCTION:
All tests can be reproduced on the reported platform by using the following command:
run-webkit-tests -1 --clobber-old-results --verbose --test-list run_tests.txt
...where run_tests.txt contains the affected tests listed above.
REGRESSION:
According to the flakiness dashboard and bisection workflow, these tests have a clear regression point at 267516@main.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Radar WebKit Bug Importer
<rdar://problem/115146287>
Ben Schwartz
Adding http/tests/appcache/remove-cache.html, which is a consistent/constant failure on Monterey wk1.
EWS
Test gardening commit 267764@main (1c1d8e1af449): <https://commits.webkit.org/267764@main>
Reviewed commits have been landed. Closing PR #17569 and removing active labels.
EWS
Test gardening commit 267798@main (af7d03822467): <https://commits.webkit.org/267798@main>
Reviewed commits have been landed. Closing PR #17596 and removing active labels.
Ben Schwartz
Adding imported/w3c/web-platform-tests/content-security-policy/reporting-api/report-to-directive-allowed-in-meta.https.sub.html, which is a constant failure on macOS but is missing exceptions for Monterey wk2.
EWS
Test gardening commit 268540@main (0564f1d21bab): <https://commits.webkit.org/268540@main>
Reviewed commits have been landed. Closing PR #18309 and removing active labels.
Ryan Haddad
It looks like all of these except for scrollingcoordinator/mac/fixed-backgrounds/fixed-background-in-overflow-in-iframe.html were fixed via rebaselines.
http/tests/appcache/fail-on-update-2.html is a flaky timeout, but it appears that we already have at least one bug tracking this.
Before closing this, we need to make sure we have removed any failure expectations added as part of this bug.
Ben Schwartz
Issues have been resolved on most tests, but persist on the following.
This test is consistently timing out on macOS:
http/tests/appcache/fail-on-update-2.html
http/tests/appcache/remove-cache.html (Monterey wk1 only: when not timing out, failing)
This test is constantly failing on macOS:
imported/w3c/web-platform-tests/content-security-policy/reporting-api/report-to-directive-allowed-in-meta.https.sub.html
I will remove the expectations for the other tests.
EWS
Test gardening commit 269991@main (4b5edf812039): <https://commits.webkit.org/269991@main>
Reviewed commits have been landed. Closing PR #19754 and removing active labels.
Ben Schwartz
Filed new bugs for the tests still experiencing issues. Closing, as verified issues with other tests resolved.