TestWebKitAPI.WKWebView.LocalStorageProcessSuspends seems flaky. In https://ews-build.webkit.org/#/builders/9/builds/4903, the test Failed in run-api-tests step. However, in the immediately next retry step (re-run-api-tests), it passed. Same thing happened in https://ews-build.webkit.org/#/builders/9/builds/4901 In https://ews-build.webkit.org/#/builders/9/builds/4784, it just failed consistently.
It can also be seen failing on a trunk bot here: https://build.webkit.org/builders/Apple%20Mojave%20Debug%20WK2%20%28Tests%29/builds/3730/steps/run-api-tests/logs/stdio
<rdar://problem/53501721>
This is indeed quite flaky. The bots are often red because of this failure. TestWebKitAPI.WKWebView.LocalStorageProcessSuspends /Volumes/Data/slave/ios-simulator-12-release/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/LocalStoragePersistence.mm:156 Expected equality of these values: @"value" Which is: "value" result Which is: "newValue"
This test is creating issue in EWS infrastrucutre. It is extremely flaky and various times flakes in a manner that misleads EWS into blaming the patch being tested (this test failed consistently twice with the patch, and pass without the patch). e.g.: https://ews-build.webkit.org/#/builders/9/builds/5177 https://ews-build.webkit.org/#/builders/9/builds/5145 https://ews-build.webkit.org/#/builders/9/builds/5140 https://ews-build.webkit.org/#/builders/9/builds/5109
(In reply to Aakash Jain from comment #4) > This test is creating issue in EWS infrastrucutre. It is extremely flaky and > various times flakes in a manner that misleads EWS into blaming the patch > being tested (this test failed consistently twice with the patch, and pass > without the patch). > > e.g.: > https://ews-build.webkit.org/#/builders/9/builds/5177 > https://ews-build.webkit.org/#/builders/9/builds/5145 > https://ews-build.webkit.org/#/builders/9/builds/5140 > https://ews-build.webkit.org/#/builders/9/builds/5109 Do you mean this API test always passes without those patches being tested but fails with them? It sounds like some states are kept between test runs.. Does this happen since last week? If this starts to happen recently, it may help narrow down the blamed changes.
FWIW this isn't specific to iOS. Based on history, this may have started after: Speed up StorageManager::getValues() https://trac.webkit.org/changeset/247486/webkit
(In reply to Ryan Haddad from comment #6) > FWIW this isn't specific to iOS. > > Based on history, this may have started after: > > Speed up StorageManager::getValues() > https://trac.webkit.org/changeset/247486/webkit Ok, so probably making it a WorkQueueMessageReceiver again.
> Do you mean this API test always passes without those patches being tested > but fails with them? Not always. It flakes in various different manners. This particular manner is particularly bad for EWS (as it incorrectly blames a patch). In following examples, the test is clearly flaky; it failed in first run (run-api-tests step), but in immediate retry step (in re-run-api-tests) this test passed: https://ews-build.webkit.org/#/builders/9/builds/5199 https://ews-build.webkit.org/#/builders/9/builds/5183 https://ews-build.webkit.org/#/builders/9/builds/5179 https://ews-build.webkit.org/#/builders/9/builds/5176 https://ews-build.webkit.org/#/builders/9/builds/5170 https://ews-build.webkit.org/#/builders/9/builds/5163 https://ews-build.webkit.org/#/builders/9/builds/5157 https://ews-build.webkit.org/#/builders/9/builds/5151 https://ews-build.webkit.org/#/builders/9/builds/5123 https://ews-build.webkit.org/#/builders/9/builds/5120 https://ews-build.webkit.org/#/builders/9/builds/5119 https://ews-build.webkit.org/#/builders/9/builds/5114 https://ews-build.webkit.org/#/builders/9/builds/5103 https://ews-build.webkit.org/#/builders/9/builds/5102 > > Does this happen since last week? If this starts to happen recently, it may > help narrow down the blamed changes. Seems to be happening atleast since July 16: https://ews-build.webkit.org/#/builders/9/builds/4550 https://ews-build.webkit.org/#/builders/9/builds/4547 https://ews-build.webkit.org/#/builders/9/builds/4545
Created attachment 375227 [details] Patch
Comment on attachment 375227 [details] Patch Clearing flags on attachment: 375227 Committed r248047: <https://trac.webkit.org/changeset/248047>
All reviewed patches have been landed. Closing bug.