Bug 203732 – Flaky API Test TestWebKitAPI.WebKit.ApplicationCacheDirectories

• Component: Tools / Tests
• Status: RESOLVED DUPLICATE

[Aakash Jain]

TestWebKitAPI.WebKit.ApplicationCacheDirectories seems flaky on iOS. In https://ews-build.webkit.org/#/builders/9/builds/11269, the test failed in run-api-tests step. However, in the immediately next retry step (re-run-api-tests), it passed.

Also, this test failed on clean tree in https://ews-build.webkit.org/#/builders/9/builds/11291/steps/13/logs/json

[Aakash Jain]

This test was added 3 months back in https://trac.webkit.org/changeset/247686/webkit

[Radar WebKit Bug Importer]

<rdar://problem/56814641>

[Aakash Jain]

This test flaked in https://ews-build.webkit.org/#/builders/9/builds/11281 as well.

[Aakash Jain]

Flaked again in https://ews-build.webkit.org/#/builders/9/builds/11296 and https://ews-build.webkit.org/#/builders/9/builds/11297

[Aakash Jain]

This test has become too much flaky recently and slowing down EWS.

This failure/flakiness started yesterday, and was first seen in: https://ews-build.webkit.org/#/builders/9/builds/11260

More examples of flakiness/failures:
https://ews-build.webkit.org/#/builders/9/builds/11307
https://ews-build.webkit.org/#/builders/9/builds/11314
https://ews-build.webkit.org/#/builders/9/builds/11318
https://ews-build.webkit.org/#/builders/9/builds/11320

[Alexey Proskuryakov]

There isn't much in this test to tie out on, unless TCPServer is broken...

[Alexey Proskuryakov]

Looks like another one that isn't failing in post-commit testing.

[Aakash Jain]

(In reply to Alexey Proskuryakov from comment #7)
> Looks like another one that isn't failing in post-commit testing.
Yes, also this test isn't failing on EWS bots which are on Catalina (ews156 and ews157), it is only failing/flaky on bots which are on Mojave (ews158 and ews159).

[Alex Christensen]

Do we know when this flakiness started?
I don't have a Mojave machine with me, so this won't be investigated this week.  Could we disable this test on Mojave for now?

[Aakash Jain]

Committed r252404: <https://trac.webkit.org/changeset/252404>

[Aakash Jain]

(In reply to Alex Christensen from comment #9)
> Do we know when this flakiness started?
First failure was noticed in https://ews-build.webkit.org/#/builders/9/builds/11260 on Oct 31st.

> I don't have a Mojave machine with me, so this won't be investigated this
> week.  Could we disable this test on Mojave for now?
I don't think we have a way to disable API tests on specific platforms. I have disabled this test for now.

This is still flaking frequently and causing false-positives on EWS:
https://ews-build.webkit.org/#/builders/9/builds/11936
https://ews-build.webkit.org/#/builders/9/builds/11928
https://ews-build.webkit.org/#/builders/9/builds/11884
https://ews-build.webkit.org/#/builders/9/builds/11876

[Alex Christensen]

*** This bug has been marked as a duplicate of bug 203734 ***