Bug 204409 - [EWS] iOS layout-tests fails with 30+ failures on re-run and on clean-tree
Summary: [EWS] iOS layout-tests fails with 30+ failures on re-run and on clean-tree
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Jonathan Bedard
Keywords: InRadar
Depends on:
Reported: 2019-11-20 05:14 PST by Aakash Jain
Modified: 2019-11-20 12:35 PST (History)
6 users (show)

See Also:

Patch (4.87 KB, patch)
2019-11-20 11:13 PST, Jonathan Bedard
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Aakash Jain 2019-11-20 05:14:37 PST
If the layout-test finds any test failure, we re-run the tests (to rule-out flakiness) and do clean-tree run (to check pre-existing failures).

on iOS, recently the re-runs and clean-tree runs have frequently started failing with 30+ failures. 


This behavior is somewhat flaky. For e.g. in https://ews-build.webkit.org/#/builders/24/builds/5261 run-layout-tests-without-patch worked, and in https://ews-build.webkit.org/#/builders/24/builds/5254 re-run-layout-tests worked.
Comment 1 Aakash Jain 2019-11-20 05:15:04 PST
This issue stalls iOS WK2 queue whenever there is any failure on trunk.
Comment 2 Jonathan Bedard 2019-11-20 08:59:53 PST
I don't understand how clean-tree results are somehow more flakey than the test runs with the patch.
Comment 3 Aakash Jain 2019-11-20 09:06:50 PST
This seems like another fallout from Catalina update on iOS bots. We updated the bots around Oct 30, and we started seeing 30+ layout-test failure in https://bugs.webkit.org/show_bug.cgi?id=203792
That issue was fixed on Nov 10. I believe this issue has been happening since then.
Comment 4 Jonathan Bedard 2019-11-20 10:27:14 PST
I bet that running kill-old-processes between runs would help. As Alexey pointed out, this is probably a bug of some sort, so I just landed <https://trac.webkit.org/r252691> to help us figure out what the problem is.
Comment 5 Jonathan Bedard 2019-11-20 11:13:58 PST
Created attachment 383974 [details]
Comment 6 Jonathan Bedard 2019-11-20 11:56:00 PST
Comment on attachment 383974 [details]

Now that we have some logging from <https://trac.webkit.org/r252691>, it seems to be simulator and https. I'll work on fixes for those problems independently, but as I mention offline to Aakash:

"I think the kill-old-processes stuff needs to land. If nothing else, it's possible that a patch which fails to clean up resources appropriately could cause failures in the clean run and cause us to mark the patch as 'good' even though it is demonstrably not"
Comment 7 Jonathan Bedard 2019-11-20 12:14:28 PST
Committed r252700: <https://trac.webkit.org/changeset/252700>
Comment 8 Jonathan Bedard 2019-11-20 12:15:09 PST
Comment on attachment 383974 [details]

Landed manually, removing flags.
Comment 9 Radar WebKit Bug Importer 2019-11-20 12:15:19 PST
Comment 10 Radar WebKit Bug Importer 2019-11-20 12:15:25 PST