Bug 239871 - [ macOS wk1 ] Four imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower tests are flaky image failures
Summary: [ macOS wk1 ] Four imported/w3c/web-platform-tests/css/css-text/text-transfor...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Karl Rackler
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2022-04-28 13:56 PDT by Karl Rackler
Modified: 2022-08-25 16:25 PDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Karl Rackler 2022-04-28 13:56:00 PDT
Description:
imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-003.html

The first failure that I saw on the dashboard for Big Sur was 2/15/2022 at r289847, and covers EWS rebase and is not relevant to causing this issue.  The first failure that I saw on the dashboard for Monterey was 3/17/2022 at r291422 but is modifying WebGPU/buffer and does not appear to be causing this issue.   This test seems to have been flaky since the start. 

History:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-text%2Ftext-transform%2Ftext-transform-upperlower-003.html&platform=mac&flavor=wk1&limit=50000

Image Diff:
https://build.webkit.org/results/Apple-Monterey-Debug-WK1-Tests/250087@main%20(1970)/imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-003-diffs.html
Comment 1 Radar WebKit Bug Importer 2022-04-28 13:56:20 PDT
<rdar://problem/92485701>
Comment 2 Karl Rackler 2022-04-28 14:11:49 PDT
REPRODUCTION STEPS
I can reproduce this on ToT (r293579).

Command: 
run-webkit-tests --root -1 --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --pixel-tests --tolerance 0.0364 --iterations 100 -f --no-retry

Result:
Unexpected flakiness: image-only failures (1)
  imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-003.html [ Pass ImageOnlyFailure ]
Comment 3 Karl Rackler 2022-04-28 14:26:14 PDT
I have marked this test as a flaky image failure while this issue is investigated.
Comment 4 Karl Rackler 2022-04-28 14:29:17 PDT
Pull request: https://github.com/WebKit/WebKit/pull/429
Comment 5 EWS 2022-04-28 15:48:22 PDT
Test gardening commit r293594 (250103@main): <https://commits.webkit.org/250103@main>

Reviewed commits have been landed. Closing PR #429 and removing active labels.
Comment 6 Karl Rackler 2022-08-25 16:18:40 PDT
Description:
imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-003.html
ADDING: 
imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-007.html
imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-015.html
imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-102.html

After a series of consistent passes, the test imported/w3c/web-platform-tests/css/css-text/text-transform/text-transform-upperlower-003.html begins to fail at approximately 249091@main.

The other three tests after a series of consistent passes, begin to fail at approximately245577@main.  

History:
https://results.webkit.org/?limit=50000&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-text%2Ftext-transform%2Ftext-transform-upperlower-007.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-text%2Ftext-transform%2Ftext-transform-upperlower-015.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-text%2Ftext-transform%2Ftext-transform-upperlower-102.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-text%2Ftext-transform%2Ftext-transform-upperlower-003.html&platform=mac&flavor=wk1&recent=false

I have marked these tests as flaky image failures while this issue is investigated.
Comment 7 EWS 2022-08-25 16:25:50 PDT
Test gardening commit 253798@main (a3ccc9ae0d48): <https://commits.webkit.org/253798@main>

Reviewed commits have been landed. Closing PR #3681 and removing active labels.