Bug 231685 - [ iOS ] Some ImageDiffs are appearing as blank images instead of expected or actual outputs
Summary: [ iOS ] Some ImageDiffs are appearing as blank images instead of expected or ...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: CSS (show other bugs)
Version: WebKit Nightly Build
Hardware: iPhone / iPad Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
: 232022 232030 232054 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-10-13 10:49 PDT by ayumi_kojima
Modified: 2022-06-23 14:40 PDT (History)
7 users (show)

See Also:


Attachments
Expected color-filter-backgrounds-borders.html (188.07 KB, image/png)
2021-10-13 10:51 PDT, ayumi_kojima
no flags Details
color-filter-backgrounds-borders.html actual (138.51 KB, image/png)
2021-10-13 10:52 PDT, ayumi_kojima
no flags Details
Diff color-filter-backgrounds-borders.html (182.48 KB, image/png)
2021-10-13 10:52 PDT, ayumi_kojima
no flags Details
Expected color-filter-filter-list.html (136.12 KB, image/png)
2021-10-13 10:53 PDT, ayumi_kojima
no flags Details
Actual color-filter-filter-list.html (173.68 KB, image/png)
2021-10-13 10:53 PDT, ayumi_kojima
no flags Details
test list (464.01 KB, text/plain)
2021-10-21 16:36 PDT, ayumi_kojima
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description ayumi_kojima 2021-10-13 10:49:45 PDT
css3/color-filters/color-filter-backgrounds-borders.html
css3/color-filters/color-filter-brightness.html
css3/color-filters/color-filter-filter-list.html
css3/color-filters/color-filter-gradients.html
css3/color-filters/color-filter-hue-rotate.html
css3/color-filters/color-filter-inherits.html
css3/color-filters/color-filter-saturate.html
css3/color-filters/color-filter-sepia.html
css3/color-filters/color-filter-text-decoration-shadow.html

Are flaky image failures.

History: https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=css3%2Fcolor-filters%2Fcolor-filter-backgrounds-borders.html&test=css3%2Fcolor-filters%2Fcolor-filter-brightness.html&test=css3%2Fcolor-filters%2Fcolor-filter-filter-list.html&test=css3%2Fcolor-filters%2Fcolor-filter-gradients.html&test=css3%2Fcolor-filters%2Fcolor-filter-hue-rotate.html&test=css3%2Fcolor-filters%2Fcolor-filter-inherits.html&test=css3%2Fcolor-filters%2Fcolor-filter-saturate.html&test=css3%2Fcolor-filters%2Fcolor-filter-sepia.html&test=css3%2Fcolor-filters%2Fcolor-filter-text-decoration-shadow.html

Result page: https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284088%20(150)/results.html#

I am not sure if these are real failures since when those tests fail, it always fail together with many other tests (e.g. https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r283463%20(47)/results.html)

Also, the actual or expected image is white with a grey frame that's causing the failure.(See the attached images)

Seems like the same issue as Bug 227647.
Comment 1 ayumi_kojima 2021-10-13 10:51:52 PDT
Created attachment 441103 [details]
Expected color-filter-backgrounds-borders.html
Comment 2 ayumi_kojima 2021-10-13 10:52:10 PDT
Created attachment 441104 [details]
color-filter-backgrounds-borders.html actual
Comment 3 ayumi_kojima 2021-10-13 10:52:38 PDT
Created attachment 441105 [details]
Diff color-filter-backgrounds-borders.html
Comment 4 ayumi_kojima 2021-10-13 10:53:08 PDT
Created attachment 441106 [details]
Expected color-filter-filter-list.html
Comment 5 ayumi_kojima 2021-10-13 10:53:55 PDT
Created attachment 441108 [details]
Actual color-filter-filter-list.html
Comment 6 Radar WebKit Bug Importer 2021-10-13 10:55:51 PDT
<rdar://problem/84206453>
Comment 7 ayumi_kojima 2021-10-13 11:21:39 PDT
It looks like the other tests failed in https://build.webkit.org/results/Apple-iOS-15-Simulator-Release-WK2-Tests/r284088%20(150)/results.html# are also failed because of the white with grey frame screen.

compositing/overflow/overflow-change-reposition-descendants.html
fast/attachment/attachment-border-should-stay-inside-attachment.html
fast/forms/hidpi-fieldset-on-subpixel-position-when-legend-is-present.html
imported/blink/fast/text/mixed-direction-bidi-wrap.html
mathml/presentation/attributes-background-color.html
mathml/presentation/attributes-display.html
mathml/presentation/attributes-mathvariant.html
mathml/presentation/mstyle-css-attributes.html
Comment 8 Jonathan Bedard 2021-10-15 08:40:44 PDT
The reference file in these cases is an html file, no? This probably means that the reference file is messing up it's rendering, not that ImageDiff is reporting things incorrectly
Comment 9 Eric Hutchison 2021-10-20 16:38:01 PDT
*** Bug 232054 has been marked as a duplicate of this bug. ***
Comment 10 Eric Hutchison 2021-10-20 16:38:36 PDT
*** Bug 232030 has been marked as a duplicate of this bug. ***
Comment 11 Eric Hutchison 2021-10-20 16:39:04 PDT
*** Bug 232022 has been marked as a duplicate of this bug. ***
Comment 12 Simon Fraser (smfr) 2021-10-20 18:56:06 PDT
I think the failure here is just a blank image, right? The gray frame is in the host document.
Comment 13 Simon Fraser (smfr) 2021-10-20 18:56:47 PDT
"when those tests fail, it always fail together with many other tests": looks like a GPU Process crash.
Comment 14 Jonathan Bedard 2021-10-21 08:26:43 PDT
(In reply to Simon Fraser (smfr) from comment #13)
> "when those tests fail, it always fail together with many other tests":
> looks like a GPU Process crash.

Is the GPU Process shared between multiple test runners?
Comment 15 Simon Fraser (smfr) 2021-10-21 09:17:41 PDT
There is one GPU process per WKWebView, so no (I hope).
Comment 16 ayumi_kojima 2021-10-21 10:33:02 PDT
(In reply to Simon Fraser (smfr) from comment #12)
> I think the failure here is just a blank image, right? The gray frame is in
> the host document.

Updated the title.
Comment 17 ayumi_kojima 2021-10-21 13:52:51 PDT
I was not able to reproduce the image failure locally on iOS 15 at r284088, which image failures happened on the bot, using run-webkit-tests --exit-after-n-crashes-or-timeouts 50 --exit-after-n-failures 500 --release --iphone-simulator --no-build --no-show-results --no-new-test-results --clobber-old-results --debug-rwt-logging --no-retry-failures --child-processes 8
Comment 18 ayumi_kojima 2021-10-21 16:36:15 PDT
Tried with a test list as well, but doesn't seem to be able to reproduce the image failure. The test run exits early after 50+ timeouts for some reasons.
Comment 19 ayumi_kojima 2021-10-21 16:36:48 PDT
Created attachment 442083 [details]
test list
Comment 20 Brent Fulgham 2022-06-23 14:40:01 PDT
Ryan Haddad confirmed that this is no longer an issue as of 5/25/2022.