Bug 227467 - fast/css/sticky/sticky-left.html is a flaky image failure
Summary: fast/css/sticky/sticky-left.html is a flaky image failure
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Karl Rackler
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-06-28 15:46 PDT by Eric Hutchison
Modified: 2022-04-19 13:57 PDT (History)
7 users (show)

See Also:


Attachments
Patch (1.44 KB, patch)
2021-06-29 14:14 PDT, Eric Hutchison
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Hutchison 2021-06-28 15:46:25 PDT
fast/css/sticky/sticky-left.html

Failure on Big Sur wk2 ARM64.

First appearance 6/3/21 r278415.

History: https://results.webkit.org/?suite=layout-tests&test=fast%2Fcss%2Fsticky%2Fsticky-left.html

Unable to access diff due to 404 error.
Comment 1 Radar WebKit Bug Importer 2021-06-28 15:46:54 PDT
<rdar://problem/79885861>
Comment 2 Eric Hutchison 2021-06-28 18:27:55 PDT
Reproduced test results on build r278417.

run-webkit-tests --root 278417 fast/css/sticky/sticky-left.html --iterations 500 --exit-after-n-failures 2
Comment 3 Ryan Haddad 2021-06-28 20:35:16 PDT
13:44:16.831 21360 worker/4 Test marked as failed, ImageDiff crashed because of no data while reading stdout for the server process.
13:44:16.831 21360 worker/4   fast/css/sticky/sticky-left.html : ImageDiff produced stderr output:
Error: test and reference images have different sizes. Test image is 800x700, reference image is 800x600
ImageDiff crashed
13:44:16.831 21360 worker/4 Test marked as failed, ImageDiff crashed because of a broken pipe when writing to stdin of the server process.
13:44:16.831 21360 worker/4 ref test mismatch did not produce an image diff.
13:44:16.832 21360 [10756/59269] fast/css/sticky/sticky-left.html failed unexpectedly (reference mismatch)
13:44:16.832 21360 worker/4 fast/css/sticky/sticky-left.html failed:
13:44:16.832 21360 worker/4 reference mismatch
Comment 4 Eric Hutchison 2021-06-29 11:25:35 PDT
Bisected test failure to range of 227412 (results not reproduced) to 227417 (reproduced results).
Comment 5 Truitt Savell 2021-06-29 12:03:59 PDT
(In reply to Eric Hutchison from comment #4)
> Bisected test failure to range of 227412 (results not reproduced) to 227417
> (reproduced results).

So I revisited the bisection with Eric. It seems that the initial reproduction case was actually reproducing a crash in the test. We have not been able to reproduce this crashing image diff issue that the bot is seeing. But we found that the crash is only happening on the release builds and started on r278417. a debug build of r278417 does not seem to crash. 

I am uncertain how a version bump in r278417 could have anything to do with this, but our archived builds seem to point to this revision as starting some kind of problem.
Comment 6 Eric Hutchison 2021-06-29 14:14:05 PDT
Created attachment 432530 [details]
Patch
Comment 7 Truitt Savell 2021-06-29 14:17:22 PDT
Comment on attachment 432530 [details]
Patch

Clearing flags on attachment: 432530

Committed r279386 (239251@main): <https://commits.webkit.org/239251@main>
Comment 8 Truitt Savell 2021-06-29 14:17:24 PDT
All reviewed patches have been landed.  Closing bug.
Comment 9 Truitt Savell 2021-06-29 14:50:45 PDT
reopen bug
Comment 10 Alexey Proskuryakov 2021-06-29 14:59:12 PDT
Probably has something to do with this test, which runs just before sticky-left.html

LayoutTests/fast/css/sticky/sticky-left-percentage.html:        window.resizeTo(800, 700)
Comment 11 Alexey Proskuryakov 2021-06-29 15:04:56 PDT
Or perhaps even more relevant is sticky-left-percentage-expected.html, which would be the one loaded right before. Note that it performs window.resizeTo(800, 700) from an inline script, before onload. I wonder if this confuses IPC.
Comment 12 Karl Rackler 2022-04-19 09:50:13 PDT
Pull request: https://github.com/WebKit/WebKit/pull/316
Comment 13 Karl Rackler 2022-04-19 09:56:55 PDT
Adding fast/css/sticky/sticky-left.html is a flaky image failure on [ Monterey Release wk2 arm64 ] as well.

Description:
fast/css/sticky/sticky-left.html

The first failure that I see on the dashboard was 3/18/2022 at r291490 , but this is not relevant to causing this issue.  This test seems to have been flaky since the start on Monterey.

History:
https://results.webkit.org/?suite=layout-tests&test=fast%2Fcss%2Fsticky%2Fsticky-left.html&platform=mac&version_name=Monterey&architecture=arm64&flavor=wk2&style=release&limit=50000

Image Diff:
https://build.webkit.org/results/Apple-Monterey-Release-AppleSilicon-WK2-Tests/r249750@main%20(2764)/fast/css/sticky/sticky-left-diffs.html

I have marked this test as flaky image failure while this issue is investigated.
Comment 14 EWS 2022-04-19 11:54:58 PDT
Committed r293029 (249768@main): <https://commits.webkit.org/249768@main>

Reviewed commits have been landed. Closing PR #316 and removing active labels.
Comment 15 Ryan Haddad 2022-04-19 13:57:11 PDT
Reopening since the landed change was just updating test expectations.