Bug 296017 – [ iOS Release, macOS Release ] 7x imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image (layout-tests) are flakey image failures

• Status: NEW
• Component: New Bugs

[Bryan Medina]

imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-021.html

is a flakey image failure on iOS Release and macOS wk2 Release.

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-shapes%2Fshape-outside%2Fshape-image%2Fshape-image-021.html

IMAGE DIFF URL:
https://build.webkit.org/results/Apple-Sequoia-Release-AppleSilicon-WK2-Tests/297403@main%20(3716)/imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-021-diffs.html

Image diffs will be attached to bug, it is off by 0.79%. There are expected to be 3 green squares, but there are actually 1 red square and 1 green square with different shades of green.

[Radar WebKit Bug Importer]

<rdar://problem/155918731>

[Bryan Medina]

Created attachment 476033 [details]
Image diffs for failing test

[Bryan Medina]

I was able to locally reproduce this image failure on my device running macOS Sequoia Release ToT using the command:

run-webkit-tests imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-021.html --child-processes=1 -f --iterations=500

[EWS]

Test gardening commit 297428@main (62290e1e895b): <https://commits.webkit.org/297428@main>

Reviewed commits have been landed. Closing PR #48092 and removing active labels.

[Bryan Medina]

*** Bug 296101 has been marked as a duplicate of this bug. ***

[Bryan Medina]

I also found these 2 other tests that were impacted in the same manner (flakey image failures):

imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-022.html
&
imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-006.html

NOTICE: 
imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-022.html

also impacts macOS Release as well. Expectations will be set accordingly.

HISTORY:
https://results.webkit.org/?limit=10000&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-shapes%2Fshape-outside%2Fshape-image%2Fshape-image-022.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-shapes%2Fshape-outside%2Fshape-image%2Fshape-image-006.html

IMAGE DIFF URLS:
https://build.webkit.org/results/Apple-iOS-18-Simulator-Release-WK2-Tests/297721@main%20(2710)/imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-022-diffs.html
&
https://ews-build.s3-us-west-2.amazonaws.com/iOS-18-Simulator-WPT-WK2-Tests-EWS/1f9d60fd-25729-clean-tree/imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-006-diffs.html

Image diffs will be attached to bug above.

[Bryan Medina]

Created attachment 476153 [details]
Diffs for shape-image-022

[Bryan Medina]

Created attachment 476154 [details]
Diffs for shape-image-006

[Bryan Medina]

For clarity, the 4 tests stated in the title are referring to the 3 in this bug + the duplicate of this bug.

[EWS]

Test gardening commit 297763@main (8ef0076c7b46): <https://commits.webkit.org/297763@main>

Reviewed commits have been landed. Closing PR #48393 and removing active labels.

[Bryan Medina]

I found another test that is failing in the same way on iOS Release, macOS Release:
imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-007.html 

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-shapes%2Fshape-outside%2Fshape-image%2Fshape-image-007.html

Image diff will be attached once again

[Bryan Medina]

Created attachment 476311 [details]
Diffs for shape image 007

[EWS]

Test gardening commit 298406@main (f8619286cd80): <https://commits.webkit.org/298406@main>

Reviewed commits have been landed. Closing PR #49128 and removing active labels.

[Bryan Medina]

Another:
imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-026.html

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-shapes%2Fshape-outside%2Fshape-image%2Fshape-image-026.html

Diffs will be attached.

[Bryan Medina]

Created attachment 476363 [details]
Diffs for shape image-026

[EWS]

Test gardening commit 298540@main (4a578ead5451): <https://commits.webkit.org/298540@main>

Reviewed commits have been landed. Closing PR #49247 and removing active labels.

[Bryan Medina]

Another:
imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-014.html

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-shapes%2Fshape-outside%2Fshape-image%2Fshape-image-014.html

Diffs will be attached.

[Bryan Medina]

Created attachment 476458 [details]
Diffs for shape-image-014

[EWS]

Test gardening commit 298920@main (53d0734e3749): <https://commits.webkit.org/298920@main>

Reviewed commits have been landed. Closing PR #49616 and removing active labels.

[Bryan Medina]

Another:
imported/w3c/web-platform-tests/css/css-shapes/shape-outside/shape-image/shape-image-003.html 

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-shapes%2Fshape-outside%2Fshape-image%2Fshape-image-003.html

[Bryan Medina]

Created attachment 476466 [details]
Diffs for shape-image-003

[EWS]

Test gardening commit 298969@main (55678189e062): <https://commits.webkit.org/298969@main>

Reviewed commits have been landed. Closing PR #49665 and removing active labels.