Bug 204409 – [EWS] iOS layout-tests fails with 30+ failures on re-run and on clean-tree

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Aakash Jain]

If the layout-test finds any test failure, we re-run the tests (to rule-out flakiness) and do clean-tree run (to check pre-existing failures).

on iOS, recently the re-runs and clean-tree runs have frequently started failing with 30+ failures. 

e.g.:
https://ews-build.webkit.org/#/builders/24/builds/5262
https://ews-build.webkit.org/#/builders/24/builds/5260
https://ews-build.webkit.org/#/builders/24/builds/5259
https://ews-build.webkit.org/#/builders/24/builds/5256
https://ews-build.webkit.org/#/builders/24/builds/5251
https://ews-build.webkit.org/#/builders/24/builds/5077
https://ews-build.webkit.org/#/builders/24/builds/4981
https://ews-build.webkit.org/#/builders/24/builds/4779


This behavior is somewhat flaky. For e.g. in https://ews-build.webkit.org/#/builders/24/builds/5261 run-layout-tests-without-patch worked, and in https://ews-build.webkit.org/#/builders/24/builds/5254 re-run-layout-tests worked.

[Aakash Jain]

This issue stalls iOS WK2 queue whenever there is any failure on trunk.

[Jonathan Bedard]

I don't understand how clean-tree results are somehow more flakey than the test runs with the patch.

[Aakash Jain]

This seems like another fallout from Catalina update on iOS bots. We updated the bots around Oct 30, and we started seeing 30+ layout-test failure in https://bugs.webkit.org/show_bug.cgi?id=203792
That issue was fixed on Nov 10. I believe this issue has been happening since then.

[Jonathan Bedard]

I bet that running kill-old-processes between runs would help. As Alexey pointed out, this is probably a bug of some sort, so I just landed <https://trac.webkit.org/r252691> to help us figure out what the problem is.

[Jonathan Bedard]

Created attachment 383974 [details]
Patch

[Jonathan Bedard]

Comment on attachment 383974 [details]
Patch

Now that we have some logging from <https://trac.webkit.org/r252691>, it seems to be simulator and https. I'll work on fixes for those problems independently, but as I mention offline to Aakash:

"I think the kill-old-processes stuff needs to land. If nothing else, it's possible that a patch which fails to clean up resources appropriately could cause failures in the clean run and cause us to mark the patch as 'good' even though it is demonstrably not"

[Jonathan Bedard]

Committed r252700: <https://trac.webkit.org/changeset/252700>

[Jonathan Bedard]

Comment on attachment 383974 [details]
Patch

Landed manually, removing flags.

[Radar WebKit Bug Importer]

<rdar://problem/57368402>

[Radar WebKit Bug Importer]

<rdar://problem/57368404>