Bug 231685

Summary:

[ iOS ] Some ImageDiffs are appearing as blank images instead of expected or actual outputs

Product:

WebKit

Reporter:

ayumi_kojima

Component:

CSS

Assignee:

Nobody <webkit-unassigned>

Status:

NEW ---

Severity:

Normal

CC:

ap, bfulgham, ehutchison, jbedard, simon.fraser, webkit-bot-watchers-bugzilla, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

iPhone / iPad

OS:

Unspecified

See Also:

https://bugs.webkit.org/show_bug.cgi?id=227647

Attachments:

Description	Flags
Expected color-filter-backgrounds-borders.html	none
color-filter-backgrounds-borders.html actual	none
Diff color-filter-backgrounds-borders.html	none
Expected color-filter-filter-list.html	none
Actual color-filter-filter-list.html	none
test list	none

Description ayumi_kojima 2021-10-13 10:49:45 PDT

css3/color-filters/color-filter-backgrounds-borders.html
css3/color-filters/color-filter-brightness.html
css3/color-filters/color-filter-filter-list.html
css3/color-filters/color-filter-gradients.html
css3/color-filters/color-filter-hue-rotate.html
css3/color-filters/color-filter-inherits.html
css3/color-filters/color-filter-saturate.html
css3/color-filters/color-filter-sepia.html
css3/color-filters/color-filter-text-decoration-shadow.html

Are flaky image failures.

History: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=css3%2Fcolor-filters%2Fcolor-filter-backgrounds-borders.html&test=css3%2Fcolor-filters%2Fcolor-filter-brightness.html&test=css3%2Fcolor-filters%2Fcolor-filter-filter-list.html&test=css3%2Fcolor-filters%2Fcolor-filter-gradients.html&test=css3%2Fcolor-filters%2Fcolor-filter-hue-rotate.html&test=css3%2Fcolor-filters%2Fcolor-filter-inherits.html&test=css3%2Fcolor-filters%2Fcolor-filter-saturate.html&test=css3%2Fcolor-filters%2Fcolor-filter-sepia.html&test=css3%2Fcolor-filters%2Fcolor-filter-text-decoration-shadow.html

Result page: https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284088%20(150)/results.html#

I am not sure if these are real failures since when those tests fail, it always fail together with many other tests (e.g. https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r283463%20(47)/results.html)

Also, the actual or expected image is white with a grey frame that's causing the failure.(See the attached images)

Seems like the same issue as Bug 227647.

Comment 1 ayumi_kojima 2021-10-13 10:51:52 PDT

Created attachment 441103 [details]
Expected color-filter-backgrounds-borders.html

Comment 2 ayumi_kojima 2021-10-13 10:52:10 PDT

Created attachment 441104 [details]
color-filter-backgrounds-borders.html actual

Comment 3 ayumi_kojima 2021-10-13 10:52:38 PDT

Created attachment 441105 [details]
Diff color-filter-backgrounds-borders.html

Comment 4 ayumi_kojima 2021-10-13 10:53:08 PDT

Created attachment 441106 [details]
Expected color-filter-filter-list.html

Comment 5 ayumi_kojima 2021-10-13 10:53:55 PDT

Created attachment 441108 [details]
Actual color-filter-filter-list.html

Comment 6 Radar WebKit Bug Importer 2021-10-13 10:55:51 PDT

<rdar://problem/84206453>

Comment 7 ayumi_kojima 2021-10-13 11:21:39 PDT

It looks like the other tests failed in https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284088%20(150)/results.html# are also failed because of the white with grey frame screen.

compositing/overflow/overflow-change-reposition-descendants.html
fast/attachment/attachment-border-should-stay-inside-attachment.html
fast/forms/hidpi-fieldset-on-subpixel-position-when-legend-is-present.html
imported/blink/fast/text/mixed-direction-bidi-wrap.html
mathml/presentation/attributes-background-color.html
mathml/presentation/attributes-display.html
mathml/presentation/attributes-mathvariant.html
mathml/presentation/mstyle-css-attributes.html

Comment 8 Jonathan Bedard 2021-10-15 08:40:44 PDT

The reference file in these cases is an html file, no? This probably means that the reference file is messing up it's rendering, not that ImageDiff is reporting things incorrectly

Comment 9 Eric Hutchison 2021-10-20 16:38:01 PDT

*** Bug 232054 has been marked as a duplicate of this bug. ***

Comment 10 Eric Hutchison 2021-10-20 16:38:36 PDT

*** Bug 232030 has been marked as a duplicate of this bug. ***

Comment 11 Eric Hutchison 2021-10-20 16:39:04 PDT

*** Bug 232022 has been marked as a duplicate of this bug. ***

Comment 12 Simon Fraser (smfr) 2021-10-20 18:56:06 PDT

I think the failure here is just a blank image, right? The gray frame is in the host document.

Comment 13 Simon Fraser (smfr) 2021-10-20 18:56:47 PDT

"when those tests fail, it always fail together with many other tests": looks like a GPU Process crash.

Comment 14 Jonathan Bedard 2021-10-21 08:26:43 PDT

(In reply to Simon Fraser (smfr) from comment #13)
> "when those tests fail, it always fail together with many other tests":
> looks like a GPU Process crash.

Is the GPU Process shared between multiple test runners?

Comment 15 Simon Fraser (smfr) 2021-10-21 09:17:41 PDT

There is one GPU process per WKWebView, so no (I hope).

Comment 16 ayumi_kojima 2021-10-21 10:33:02 PDT

(In reply to Simon Fraser (smfr) from comment #12)
> I think the failure here is just a blank image, right? The gray frame is in
> the host document.

Updated the title.

Comment 17 ayumi_kojima 2021-10-21 13:52:51 PDT

I was not able to reproduce the image failure locally on iOS 15 at r284088, which image failures happened on the bot, using run-webkit-tests --exit-after-n-crashes-or-timeouts 50 --exit-after-n-failures 500 --release --iphone-simulator --no-build --no-show-results --no-new-test-results --clobber-old-results --debug-rwt-logging --no-retry-failures --child-processes 8

Comment 18 ayumi_kojima 2021-10-21 16:36:15 PDT

Tried with a test list as well, but doesn't seem to be able to reproduce the image failure. The test run exits early after 50+ timeouts for some reasons.

Comment 19 ayumi_kojima 2021-10-21 16:36:48 PDT

Created attachment 442083 [details]
test list

Comment 20 Brent Fulgham 2022-06-23 14:40:01 PDT

Ryan Haddad confirmed that this is no longer an issue as of 5/25/2022.