203652 2019-10-30 18:23:30 -0700 Flaky API Test TestWebKitAPI.WebKit.UploadDirectory 2019-11-04 09:50:44 -0800 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 aakash_jain achristensen achristensen ap cdumez commit-queue webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1585702 0 aakash_jain 2019-10-30 18:23:30 -0700 TestWebKitAPI.WebKit.UploadDirectory seems flaky on macOS. In https://ews-build.webkit.org/#/builders/3/builds/11242, the test failed in run-api-tests step. However, in the immediately next retry step (re-run-api-tests), it passed. Similar thing happened in: https://ews-build.webkit.org/#/builders/3/builds/11243 1585805 1 ap 2019-10-30 23:37:58 -0700 This test started to crash in r251778, then it turned into a failure with r251795, and then it stopped failing without a clear reason a few commits later. I suspect that this is a bad test that can leak state when it crashes. Either way, this is not happening any more. 1585861 2 aakash_jain 2019-10-31 05:41:23 -0700 Still flaky. Failed just now in https://ews-build.webkit.org/#/builders/3/builds/11251/steps/13/logs/json 1586021 3 ap 2019-10-31 12:54:49 -0700 Build 11251 had 788 API test failures with the patch, and then just this one with the patch removed. I think that this is a bad test that leaves state sometimes, which makes it fail the next time. 1586023 4 webkit-bug-importer 2019-10-31 12:56:01 -0700 <rdar://problem/56791786> 1586937 5 382732 achristensen 2019-11-04 03:55:16 -0800 Created attachment 382732 Patch 1586988 6 382732 cdumez 2019-11-04 08:01:50 -0800 Comment on attachment 382732 Patch Can't we remove the directory at the beginning of the test? 1586990 7 382741 achristensen 2019-11-04 08:13:33 -0800 Created attachment 382741 Patch 1586992 8 382741 cdumez 2019-11-04 08:18:17 -0800 Comment on attachment 382741 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=382741&action=review > Tools/TestWebKitAPI/Tests/WebKitCocoa/UploadDirectory.mm:75 > + [fileManager removeItemAtPath:directory.path error:nil]; Then why not keep the assertion? 1587005 9 382741 achristensen 2019-11-04 08:35:54 -0800 Comment on attachment 382741 Patch The point was that after a previous iteration of this test crashes, it leaves behind a file. removeItemAtPath:error: returns different values depending on whether the file existed before or not, so keeping the assertion would be propagating the flakiness. 1587009 10 cdumez 2019-11-04 08:44:01 -0800 (In reply to Alex Christensen from comment #9) > Comment on attachment 382741 [details] > Patch > > The point was that after a previous iteration of this test crashes, it > leaves behind a file. removeItemAtPath:error: returns different values > depending on whether the file existed before or not, so keeping the > assertion would be propagating the flakiness. I don't understand why asserting that the folder is not there after removing it would cause flakiness? Do you expect folder removal to fail for some reason? 1587011 11 achristensen 2019-11-04 08:49:53 -0800 Oh, asserting that the folder is not there after removing it would be fine but I think it would be redundant. I was talking about asserting that removing it succeeded or that the folder is not there before removing, both of which would be flaky. 1587016 12 382741 ap 2019-11-04 09:06:19 -0800 Comment on attachment 382741 Patch I would slightly prefer checking for the error or asserting after deleting. Knowing that deletion succeeded will make it easier to reason about future failures, if any occur. r=me with or without such changes. 1587017 13 382741 achristensen 2019-11-04 09:07:26 -0800 Comment on attachment 382741 Patch Committing as-is. If we see future flakiness we can add such assertions. 1587030 14 382741 commit-queue 2019-11-04 09:50:42 -0800 Comment on attachment 382741 Patch Clearing flags on attachment: 382741 Committed r251996: <https://trac.webkit.org/changeset/251996> 1587031 15 commit-queue 2019-11-04 09:50:44 -0800 All reviewed patches have been landed. Closing bug. 382732 2019-11-04 03:55:16 -0800 2019-11-04 08:09:40 -0800 Patch bug-203652-20191104115515.patch text/plain 1771 achristensen SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI1MTk4MykKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE3IEBACisyMDE5LTExLTA0ICBBbGV4IENocmlzdGVuc2VuICA8YWNocmlzdGVuc2VuQHdl YmtpdC5vcmc+CisKKyAgICAgICAgRmxha3kgQVBJIFRlc3QgVGVzdFdlYktpdEFQSS5XZWJLaXQu VXBsb2FkRGlyZWN0b3J5CisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVn LmNnaT9pZD0yMDM2NTIKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKwor ICAgICAgICBJdCBpcyBub3QgaW1wb3J0YW50IHRoYXQgdGhlIGRpcmVjdG9yeSB0byB1cGxvYWQg ZG9lcyBub3QgZXhpc3QgYmVmb3JlIHRoZSB0ZXN0IHN0YXJ0cy4KKyAgICAgICAgTWFrZSB0aGUg dGVzdCBtb3JlIHJvYnVzdCB0byBzdGF0ZSBsZWZ0IGJlaGluZCBieSBwcmV2aW91cyBpdGVyYXRp b25zIG9mIHRoaXMgdGVzdCBieSByZW1vdmluZyB0aGUgY2hlY2suCisgICAgICAgIFRoZSBpbXBv cnRhbnQgY2hlY2sgaW4gdGhpcyB0ZXN0IGlzIHRoZSB1cGxvYWRlZCBib2R5IHNpemUuCisKKyAg ICAgICAgKiBUZXN0V2ViS2l0QVBJL1Rlc3RzL1dlYktpdENvY29hL1VwbG9hZERpcmVjdG9yeS5t bToKKyAgICAgICAgKFRFU1QpOgorCiAyMDE5LTExLTAzICBUaW0gSG9ydG9uICA8dGltb3RoeV9o b3J0b25AYXBwbGUuY29tPgogCiAgICAgICAgIFdLV2ViVmlldyBjYW4gZ2V0IHN0dWNrIGJsYW5r IChXZWIgQ29udGVudCBwcm9jZXNzIHRoaW5rcyB0aGUgYXBwIGlzIGJhY2tncm91bmRlZCkKSW5k ZXg6IFRvb2xzL1Rlc3RXZWJLaXRBUEkvVGVzdHMvV2ViS2l0Q29jb2EvVXBsb2FkRGlyZWN0b3J5 Lm1tCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT0KLS0tIFRvb2xzL1Rlc3RXZWJLaXRBUEkvVGVzdHMvV2ViS2l0Q29jb2Ev VXBsb2FkRGlyZWN0b3J5Lm1tCShyZXZpc2lvbiAyNTE5ODApCisrKyBUb29scy9UZXN0V2ViS2l0 QVBJL1Rlc3RzL1dlYktpdENvY29hL1VwbG9hZERpcmVjdG9yeS5tbQkod29ya2luZyBjb3B5KQpA QCAtNzIsNyArNzIsNiBAQCBURVNUKFdlYktpdCwgVXBsb2FkRGlyZWN0b3J5KQogICAgIE5TRmls ZU1hbmFnZXIgKmZpbGVNYW5hZ2VyID0gW05TRmlsZU1hbmFnZXIgZGVmYXVsdE1hbmFnZXJdOwog ICAgIE5TRXJyb3IgKmVycm9yID0gbmlsOwogICAgIE5TVVJMICpkaXJlY3RvcnkgPSBbTlNVUkwg ZmlsZVVSTFdpdGhQYXRoOltOU1RlbXBvcmFyeURpcmVjdG9yeSgpIHN0cmluZ0J5QXBwZW5kaW5n UGF0aENvbXBvbmVudDpAIlVwbG9hZERpcmVjdG9yeSJdIGlzRGlyZWN0b3J5OllFU107Ci0gICAg RVhQRUNUX0ZBTFNFKFtmaWxlTWFuYWdlciBmaWxlRXhpc3RzQXRQYXRoOmRpcmVjdG9yeS5wYXRo XSk7CiAgICAgRVhQRUNUX1RSVUUoW2ZpbGVNYW5hZ2VyIGNyZWF0ZURpcmVjdG9yeUF0VVJMOmRp cmVjdG9yeSB3aXRoSW50ZXJtZWRpYXRlRGlyZWN0b3JpZXM6WUVTIGF0dHJpYnV0ZXM6bmlsIGVy cm9yOiZlcnJvcl0pOwogICAgIEVYUEVDVF9GQUxTRShlcnJvcik7CiAgICAgTlNEYXRhICp0ZXN0 RGF0YSA9IFtAInRlc3RkYXRhIiBkYXRhVXNpbmdFbmNvZGluZzpOU1VURjhTdHJpbmdFbmNvZGlu Z107Cg== 382741 2019-11-04 08:13:33 -0800 2019-11-04 09:50:42 -0800 Patch bug-203652-20191104161332.patch text/plain 1833 achristensen SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDI1MTk4OSkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE3IEBACisyMDE5LTExLTA0ICBBbGV4IENocmlzdGVuc2VuICA8YWNocmlzdGVuc2VuQHdl YmtpdC5vcmc+CisKKyAgICAgICAgRmxha3kgQVBJIFRlc3QgVGVzdFdlYktpdEFQSS5XZWJLaXQu VXBsb2FkRGlyZWN0b3J5CisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVn LmNnaT9pZD0yMDM2NTIKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKwor ICAgICAgICBJdCBpcyBub3QgaW1wb3J0YW50IHRoYXQgdGhlIGRpcmVjdG9yeSB0byB1cGxvYWQg ZG9lcyBub3QgZXhpc3QgYmVmb3JlIHRoZSB0ZXN0IHN0YXJ0cy4KKyAgICAgICAgTWFrZSB0aGUg dGVzdCBtb3JlIHJvYnVzdCB0byBzdGF0ZSBsZWZ0IGJlaGluZCBieSBwcmV2aW91cyBpdGVyYXRp b25zIG9mIHRoaXMgdGVzdCBieSByZW1vdmluZyB0aGUgY2hlY2suCisgICAgICAgIFRoZSBpbXBv cnRhbnQgY2hlY2sgaW4gdGhpcyB0ZXN0IGlzIHRoZSB1cGxvYWRlZCBib2R5IHNpemUuCisKKyAg ICAgICAgKiBUZXN0V2ViS2l0QVBJL1Rlc3RzL1dlYktpdENvY29hL1VwbG9hZERpcmVjdG9yeS5t bToKKyAgICAgICAgKFRFU1QpOgorCiAyMDE5LTExLTAzICBUaW0gSG9ydG9uICA8dGltb3RoeV9o b3J0b25AYXBwbGUuY29tPgogCiAgICAgICAgIFdLV2ViVmlldyBjYW4gZ2V0IHN0dWNrIGJsYW5r IChXZWIgQ29udGVudCBwcm9jZXNzIHRoaW5rcyB0aGUgYXBwIGlzIGJhY2tncm91bmRlZCkKSW5k ZXg6IFRvb2xzL1Rlc3RXZWJLaXRBUEkvVGVzdHMvV2ViS2l0Q29jb2EvVXBsb2FkRGlyZWN0b3J5 Lm1tCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT0KLS0tIFRvb2xzL1Rlc3RXZWJLaXRBUEkvVGVzdHMvV2ViS2l0Q29jb2Ev VXBsb2FkRGlyZWN0b3J5Lm1tCShyZXZpc2lvbiAyNTE5ODApCisrKyBUb29scy9UZXN0V2ViS2l0 QVBJL1Rlc3RzL1dlYktpdENvY29hL1VwbG9hZERpcmVjdG9yeS5tbQkod29ya2luZyBjb3B5KQpA QCAtNzIsNyArNzIsNyBAQCBURVNUKFdlYktpdCwgVXBsb2FkRGlyZWN0b3J5KQogICAgIE5TRmls ZU1hbmFnZXIgKmZpbGVNYW5hZ2VyID0gW05TRmlsZU1hbmFnZXIgZGVmYXVsdE1hbmFnZXJdOwog ICAgIE5TRXJyb3IgKmVycm9yID0gbmlsOwogICAgIE5TVVJMICpkaXJlY3RvcnkgPSBbTlNVUkwg ZmlsZVVSTFdpdGhQYXRoOltOU1RlbXBvcmFyeURpcmVjdG9yeSgpIHN0cmluZ0J5QXBwZW5kaW5n UGF0aENvbXBvbmVudDpAIlVwbG9hZERpcmVjdG9yeSJdIGlzRGlyZWN0b3J5OllFU107Ci0gICAg RVhQRUNUX0ZBTFNFKFtmaWxlTWFuYWdlciBmaWxlRXhpc3RzQXRQYXRoOmRpcmVjdG9yeS5wYXRo XSk7CisgICAgW2ZpbGVNYW5hZ2VyIHJlbW92ZUl0ZW1BdFBhdGg6ZGlyZWN0b3J5LnBhdGggZXJy b3I6bmlsXTsKICAgICBFWFBFQ1RfVFJVRShbZmlsZU1hbmFnZXIgY3JlYXRlRGlyZWN0b3J5QXRV Ukw6ZGlyZWN0b3J5IHdpdGhJbnRlcm1lZGlhdGVEaXJlY3RvcmllczpZRVMgYXR0cmlidXRlczpu aWwgZXJyb3I6JmVycm9yXSk7CiAgICAgRVhQRUNUX0ZBTFNFKGVycm9yKTsKICAgICBOU0RhdGEg KnRlc3REYXRhID0gW0AidGVzdGRhdGEiIGRhdGFVc2luZ0VuY29kaW5nOk5TVVRGOFN0cmluZ0Vu Y29kaW5nXTsK