Bug 231685 – [ iOS ] Some ImageDiffs are appearing as blank images instead of expected or actual outputs

• Status: NEW
• Component: CSS

css3/color-filters/color-filter-backgrounds-borders.html
css3/color-filters/color-filter-brightness.html
css3/color-filters/color-filter-filter-list.html
css3/color-filters/color-filter-gradients.html
css3/color-filters/color-filter-hue-rotate.html
css3/color-filters/color-filter-inherits.html
css3/color-filters/color-filter-saturate.html
css3/color-filters/color-filter-sepia.html
css3/color-filters/color-filter-text-decoration-shadow.html

Are flaky image failures.

History: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=css3%2Fcolor-filters%2Fcolor-filter-backgrounds-borders.html&test=css3%2Fcolor-filters%2Fcolor-filter-brightness.html&test=css3%2Fcolor-filters%2Fcolor-filter-filter-list.html&test=css3%2Fcolor-filters%2Fcolor-filter-gradients.html&test=css3%2Fcolor-filters%2Fcolor-filter-hue-rotate.html&test=css3%2Fcolor-filters%2Fcolor-filter-inherits.html&test=css3%2Fcolor-filters%2Fcolor-filter-saturate.html&test=css3%2Fcolor-filters%2Fcolor-filter-sepia.html&test=css3%2Fcolor-filters%2Fcolor-filter-text-decoration-shadow.html

Result page: https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284088%20(150)/results.html#

I am not sure if these are real failures since when those tests fail, it always fail together with many other tests (e.g. https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r283463%20(47)/results.html)

Also, the actual or expected image is white with a grey frame that's causing the failure.(See the attached images)

Seems like the same issue as Bug 227647.

Created attachment 441103 [details]
Expected color-filter-backgrounds-borders.html

Created attachment 441104 [details]
color-filter-backgrounds-borders.html actual

Created attachment 441105 [details]
Diff color-filter-backgrounds-borders.html

Created attachment 441106 [details]
Expected color-filter-filter-list.html

Created attachment 441108 [details]
Actual color-filter-filter-list.html

[Radar WebKit Bug Importer]

<rdar://problem/84206453>

It looks like the other tests failed in https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284088%20(150)/results.html# are also failed because of the white with grey frame screen.

compositing/overflow/overflow-change-reposition-descendants.html
fast/attachment/attachment-border-should-stay-inside-attachment.html
fast/forms/hidpi-fieldset-on-subpixel-position-when-legend-is-present.html
imported/blink/fast/text/mixed-direction-bidi-wrap.html
mathml/presentation/attributes-background-color.html
mathml/presentation/attributes-display.html
mathml/presentation/attributes-mathvariant.html
mathml/presentation/mstyle-css-attributes.html

[Jonathan Bedard]

The reference file in these cases is an html file, no? This probably means that the reference file is messing up it's rendering, not that ImageDiff is reporting things incorrectly

[Eric Hutchison]

*** Bug 232054 has been marked as a duplicate of this bug. ***

[Eric Hutchison]

*** Bug 232030 has been marked as a duplicate of this bug. ***

[Eric Hutchison]

*** Bug 232022 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

I think the failure here is just a blank image, right? The gray frame is in the host document.

[Simon Fraser (smfr)]

"when those tests fail, it always fail together with many other tests": looks like a GPU Process crash.

[Jonathan Bedard]

(In reply to Simon Fraser (smfr) from comment #13)
> "when those tests fail, it always fail together with many other tests":
> looks like a GPU Process crash.

Is the GPU Process shared between multiple test runners?

[Simon Fraser (smfr)]

There is one GPU process per WKWebView, so no (I hope).

(In reply to Simon Fraser (smfr) from comment #12)
> I think the failure here is just a blank image, right? The gray frame is in
> the host document.

Updated the title.

I was not able to reproduce the image failure locally on iOS 15 at r284088, which image failures happened on the bot, using run-webkit-tests --exit-after-n-crashes-or-timeouts 50 --exit-after-n-failures 500 --release --iphone-simulator --no-build --no-show-results --no-new-test-results --clobber-old-results --debug-rwt-logging --no-retry-failures --child-processes 8

Tried with a test list as well, but doesn't seem to be able to reproduce the image failure. The test run exits early after 50+ timeouts for some reasons.

Created attachment 442083 [details]
test list

[Brent Fulgham]

Ryan Haddad confirmed that this is no longer an issue as of 5/25/2022.