API Test TestWebKitAPI.WebKit.DefaultQuota is failing consistently on macOS Release build. e.g.: https://build.webkit.org/builders/Apple%20High%20Sierra%20Release%20WK2%20%28Tests%29/builds/13637/steps/run-api-tests/logs/stdio https://ews-build.webkit.org/#/builders/3/builds/10036
This test was modified recently in https://trac.webkit.org/changeset/250728/webkit
That was a week ago. Has it been failing consistently since then? A sampling of the latest API test failures shows it is not failing consistently.
Pretty much always fails on OpenSource High Sierra: <https://results.webkit.org/?suite=api-tests&test=TestWebKitAPI.WebKit.DefaultQuota> (unfortunately, we just brought up the database with the Catalina testers, so we don't have more results....)
Actually it's not consistently failing, it's very flaky. For e.g.: in https://ews-build.webkit.org/#/builders/3/builds/10056, the test failed in run-api-tests step, but in immediate retry (re-run-api-tests) it passed. Similar thing happened in: https://ews-build.webkit.org/#/builders/3/builds/10054 https://ews-build.webkit.org/#/builders/3/builds/10048 https://ews-build.webkit.org/#/builders/3/builds/10018 https://ews-build.webkit.org/#/builders/3/builds/10005 Also, in following patches, it was flaky in a manner, which caused EWS to wrongly blame the patch being tested: https://ews-build.webkit.org/#/builders/3/builds/10037 https://ews-build.webkit.org/#/builders/3/builds/10019 https://ews-build.webkit.org/#/builders/3/builds/10014 https://ews-build.webkit.org/#/builders/3/builds/10006
The first failure seems to be in https://ews-build.webkit.org/#/builders/3/builds/9629 followed by https://ews-build.webkit.org/#/builders/3/builds/9643 and https://ews-build.webkit.org/#/builders/3/builds/9729. All these patches belongs to Bug 202590. Bug 202590 has red EWS bubble for api-mac, indicating this failure, and that patch was committed. Seems like that commit broke this test.
(In reply to Aakash Jain from comment #5) > The first failure seems to be in > https://ews-build.webkit.org/#/builders/3/builds/9629 > followed by https://ews-build.webkit.org/#/builders/3/builds/9643 and > https://ews-build.webkit.org/#/builders/3/builds/9729. All these patches > belongs to Bug 202590. > > Bug 202590 has red EWS bubble for api-mac, indicating this failure, and that > patch was committed. Seems like that commit broke this test. I will take a look but Bug 202590 is a PageCache patch and should have nothing to do with quotas.
The test passes for me locally.
+Youenn who wrote this test because I doubt this can be related to my PageCache change. Looking at this, I have no idea how it could be related.
I was able to reproduce the timeout one time. I'll try running the test in a loop for make it more reproducible.
(In reply to Chris Dumez from comment #9) > I was able to reproduce the timeout one time. I'll try running the test in a > loop for make it more reproducible. So, then it reproduces, with some extra logging I see: doTest(10) BEGIN (0) doTest(10) END (0) doTest(10) BEGIN (1) doTest(10) END (1) doTest(10) BEGIN (2) doTest(10) END (2) doTest(10) BEGIN (3) doTest(10) END (3) doTest(10) BEGIN (4) doTest(10) END (4) doTest(10) BEGIN (5) doTest(10) END (5) doTest(10) BEGIN (6) doTest(10) END (6) doTest(10) BEGIN (7) So the 7 first calls to doTest() caused us to receive a message back from the page, but the 8th one did not.
(In reply to Chris Dumez from comment #10) > (In reply to Chris Dumez from comment #9) > > I was able to reproduce the timeout one time. I'll try running the test in a > > loop for make it more reproducible. > > So, then it reproduces, with some extra logging I see: > doTest(10) BEGIN (0) > doTest(10) END (0) > doTest(10) BEGIN (1) > doTest(10) END (1) > doTest(10) BEGIN (2) > doTest(10) END (2) > doTest(10) BEGIN (3) > doTest(10) END (3) > doTest(10) BEGIN (4) > doTest(10) END (4) > doTest(10) BEGIN (5) > doTest(10) END (5) > doTest(10) BEGIN (6) > doTest(10) END (6) > doTest(10) BEGIN (7) > > > So the 7 first calls to doTest() caused us to receive a message back from > the page, but the 8th one did not. I am making good progress debugging this. I should be able to fix this shortly.
Created attachment 380798 [details] Patch
Comment on attachment 380798 [details] Patch Clearing flags on attachment: 380798 Committed r251031: <https://trac.webkit.org/changeset/251031>
All reviewed patches have been landed. Closing bug.
<rdar://problem/56313729>
This test flaked again 2 days ago. In https://ews-build.webkit.org/#/builders/3/builds/11094, it timed out in run-api-tests step. However, in the immediately next retry step (re-run-api-tests), it passed.