Bug 196909 - Disable Flaky API Test WKAttachmentTests.AddAttachmentToConnectedImageElement
Summary: Disable Flaky API Test WKAttachmentTests.AddAttachmentToConnectedImageElement
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Aakash Jain
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-04-14 16:14 PDT by Aakash Jain
Modified: 2019-04-14 17:03 PDT (History)
7 users (show)

See Also:


Attachments
Patch (1.34 KB, patch)
2019-04-14 16:18 PDT, Aakash Jain
wenson_hsieh: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Aakash Jain 2019-04-14 16:14:58 PDT
API Test WKAttachmentTests.AddAttachmentToConnectedImageElement is very flaky. It makes EWS for API Test very inefficient (since in case of any test failure, EWS has to re-run the tests, and then re-run again on clean tree).

We should disable the test to keep infrastructure happy, while the root-cause is being investigated.
Comment 1 Aakash Jain 2019-04-14 16:15:37 PDT
This flaky test is slowing down EWS (since EWS has to re-run API tests few times to confirm that the failure is not because of the patch being tested).

e.g.:
https://ews-build.webkit.org/#/builders/3/builds/535
https://ews-build.webkit.org/#/builders/3/builds/530
https://ews-build.webkit.org/#/builders/3/builds/528
https://ews-build.webkit.org/#/builders/3/builds/503
https://ews-build.webkit.org/#/builders/3/builds/500
https://ews-build.webkit.org/#/builders/3/builds/467
https://ews-build.webkit.org/#/builders/3/builds/465
https://ews-build.webkit.org/#/builders/3/builds/460

https://ews-build.webkit.org/#/builders/3/builds/541
https://ews-build.webkit.org/#/builders/3/builds/527
https://ews-build.webkit.org/#/builders/3/builds/526
https://ews-build.webkit.org/#/builders/3/builds/518
https://ews-build.webkit.org/#/builders/3/builds/515
https://ews-build.webkit.org/#/builders/3/builds/509
https://ews-build.webkit.org/#/builders/3/builds/499
https://ews-build.webkit.org/#/builders/3/builds/497
https://ews-build.webkit.org/#/builders/3/builds/494
https://ews-build.webkit.org/#/builders/3/builds/493
https://ews-build.webkit.org/#/builders/3/builds/490
https://ews-build.webkit.org/#/builders/3/builds/480
https://ews-build.webkit.org/#/builders/3/builds/479
https://ews-build.webkit.org/#/builders/3/builds/477
https://ews-build.webkit.org/#/builders/3/builds/476
https://ews-build.webkit.org/#/builders/3/builds/474
https://ews-build.webkit.org/#/builders/3/builds/473
https://ews-build.webkit.org/#/builders/3/builds/470



In following builds, the test was so flaky that EWS incorrectly thought that the failure was introduced by the patch being tested (this test failed twice with the patch, and didn't fail on clean tree):
https://ews-build.webkit.org/#/builders/3/builds/542
https://ews-build.webkit.org/#/builders/3/builds/546
https://ews-build.webkit.org/#/builders/3/builds/512
https://ews-build.webkit.org/#/builders/3/builds/495
https://ews-build.webkit.org/#/builders/3/builds/492
https://ews-build.webkit.org/#/builders/3/builds/488
https://ews-build.webkit.org/#/builders/3/builds/484
https://ews-build.webkit.org/#/builders/3/builds/471
https://ews-build.webkit.org/#/builders/3/builds/468
https://ews-build.webkit.org/#/builders/3/builds/464
https://ews-build.webkit.org/#/builders/3/builds/454
https://ews-build.webkit.org/#/builders/3/builds/440
https://ews-build.webkit.org/#/builders/3/builds/439
https://ews-build.webkit.org/#/builders/3/builds/429
https://ews-build.webkit.org/#/builders/3/builds/426
https://ews-build.webkit.org/#/builders/3/builds/414
Comment 2 Aakash Jain 2019-04-14 16:18:25 PDT
Created attachment 367407 [details]
Patch
Comment 3 Wenson Hsieh 2019-04-14 16:20:15 PDT
Seems reasonable for the time being :(

I'll be following up in https://bugs.webkit.org/show_bug.cgi?id=196905.
Comment 4 Aakash Jain 2019-04-14 17:02:08 PDT
Committed r244249: <https://trac.webkit.org/changeset/244249>
Comment 5 Radar WebKit Bug Importer 2019-04-14 17:03:18 PDT
<rdar://problem/49889118>