On macOS Mojave, in a Release build of r244241, the API test WKAttachmentTests.AddAttachmentToConnectedImageElement failed on 6 out of 10 runs of the WKAttachmentTests API test suite. It also sometimes fails on api-mac EWS bots in a way that makes it appear like a seemingly-unrelated patch regressed the test (e.g., bug #177484). Steps to reproduce: 1. Create a Mac Release build of r244241. 2. run-api-tests WKAttachmentTests. Expected results: All 31 tests pass. Actual results: On 6 out of 10 runs, WKAttachmentTests.AddAttachmentToConnectedImageElement failed. TestWebKitAPI.WKAttachmentTests.AddAttachmentToConnectedImageElement _RegisterApplication(), FAILED TO establish the default connection to the WindowServer, _CGSDefaultConnection() is NULL. 2019-04-14 08:23:20.508 TestWebKitAPI[59497:767481] Expected inserted attachments: ( ) to match ( "<_WKAttachment 0x7fbcc7d01340 id='2414efeb-bbf1-4650-83ce-a167b0a11484'>" ). /Volumes/Data/Projects/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm:150 Value of: insertedAttachmentsMatch Actual: false Expected: true
<rdar://problem/49886096>
This flaky test is slowing down EWS (since EWS has to re-run API tests few times to confirm that the failure is not because of the patch being tested). e.g.: https://ews-build.webkit.org/#/builders/3/builds/535 https://ews-build.webkit.org/#/builders/3/builds/530 https://ews-build.webkit.org/#/builders/3/builds/528 https://ews-build.webkit.org/#/builders/3/builds/503 https://ews-build.webkit.org/#/builders/3/builds/500 https://ews-build.webkit.org/#/builders/3/builds/467 https://ews-build.webkit.org/#/builders/3/builds/465 https://ews-build.webkit.org/#/builders/3/builds/460 https://ews-build.webkit.org/#/builders/3/builds/541 https://ews-build.webkit.org/#/builders/3/builds/527 https://ews-build.webkit.org/#/builders/3/builds/526 https://ews-build.webkit.org/#/builders/3/builds/518 https://ews-build.webkit.org/#/builders/3/builds/515 https://ews-build.webkit.org/#/builders/3/builds/509 https://ews-build.webkit.org/#/builders/3/builds/499 https://ews-build.webkit.org/#/builders/3/builds/497 https://ews-build.webkit.org/#/builders/3/builds/494 https://ews-build.webkit.org/#/builders/3/builds/493 https://ews-build.webkit.org/#/builders/3/builds/490 https://ews-build.webkit.org/#/builders/3/builds/480 https://ews-build.webkit.org/#/builders/3/builds/479 https://ews-build.webkit.org/#/builders/3/builds/477 https://ews-build.webkit.org/#/builders/3/builds/476 https://ews-build.webkit.org/#/builders/3/builds/474 https://ews-build.webkit.org/#/builders/3/builds/473 https://ews-build.webkit.org/#/builders/3/builds/470 In following builds, the test was so flaky that EWS incorrectly thought that the failure was introduced by the patch being tested (this test failed twice with the patch, and didn't fail on clean tree): https://ews-build.webkit.org/#/builders/3/builds/542 https://ews-build.webkit.org/#/builders/3/builds/546 https://ews-build.webkit.org/#/builders/3/builds/512 https://ews-build.webkit.org/#/builders/3/builds/495 https://ews-build.webkit.org/#/builders/3/builds/492 https://ews-build.webkit.org/#/builders/3/builds/488 https://ews-build.webkit.org/#/builders/3/builds/484 https://ews-build.webkit.org/#/builders/3/builds/471 https://ews-build.webkit.org/#/builders/3/builds/468 https://ews-build.webkit.org/#/builders/3/builds/464 https://ews-build.webkit.org/#/builders/3/builds/454 https://ews-build.webkit.org/#/builders/3/builds/440 https://ews-build.webkit.org/#/builders/3/builds/439 https://ews-build.webkit.org/#/builders/3/builds/429 https://ews-build.webkit.org/#/builders/3/builds/426 https://ews-build.webkit.org/#/builders/3/builds/414 Observation: This test seems to be failing/flaky since https://ews-build.webkit.org/#/builders/3/builds/414
I can't seem to hit this when running the test as-is, but this is reasonably easy to reproduce by running it in a loop like so: --- a/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm +++ b/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKAttachmentTests.mm @@ -1348,6 +1348,8 @@ TEST(WKAttachmentTests, ChangeFileWrapperForPastedImage) TEST(WKAttachmentTests, AddAttachmentToConnectedImageElement) { + for (int i = 0; i < 50; i++) { + NSLog(@"<WK>: Running iteration: %tu", i); auto webView = webViewForTestingAttachments(); [webView _synchronouslyExecuteEditCommand:@"InsertHTML" argument:@"<img></img>"]; @@ -1370,6 +1372,7 @@ TEST(WKAttachmentTests, AddAttachmentToConnectedImageElement) [attachment synchronouslySetFileWrapper:firstImage.get() newContentType:nil error:nil]; [webView waitForImageElementSizeToBecome:CGSizeMake(215, 174)]; + } }
Created attachment 367408 [details] Patch
Created attachment 367409 [details] Add radar link to ChangeLog
Created attachment 367410 [details] Also reenable the test
Comment on attachment 367410 [details] Also reenable the test Clearing flags on attachment: 367410 Committed r244251: <https://trac.webkit.org/changeset/244251>
All reviewed patches have been landed. Closing bug.