187557 2018-07-11 10:57:33 -0700 REGRESSION: [ MacOS WK1 & iOS ] Layout Tests in media/media-fragments/ are flaky 2024-06-06 09:53:12 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED WORKSFORME https://bugs.webkit.org/show_bug.cgi?id=187522 InRadar P2 Normal --- 1 tsavell webkit-unassigned ben_schwartz commit-queue eric.carlson jer.noble Lawrence.j lforschler realdawei ryanhaddad webkit-bug-importer oldest_to_newest 1441188 0 tsavell 2018-07-11 10:57:33 -0700 The following layout tests are flaky on MacOS WK1 in: media/media-fragments/ Probable cause: All the tests have been flaky for quite some time. They are highly intermittent and all fail in very similar ways. Example of failure output: --- /Volumes/Data/slave/highsierra-debug-tests-wk1/build/layout-test-results/media/media-fragments/TC0078-expected.txt +++ /Volumes/Data/slave/highsierra-debug-tests-wk1/build/layout-test-results/media/media-fragments/TC0078-actual.txt @@ -7,6 +7,6 @@ EXPECTED (video.currentTime == '3') OK RUN(video.play()) EVENT(pause) -EXPECTED (video.currentTime - fragmentEndTime <= '0.5') OK +EXPECTED (video.currentTime - fragmentEndTime <= '0.5'), OBSERVED '0.66' FAIL END OF TEST Flakiness Dashboard: http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Fmedia-fragments%2F 1442016 1 tsavell 2018-07-13 14:29:43 -0700 *** Bug 187522 has been marked as a duplicate of this bug. *** 1442018 2 realdawei 2018-07-13 14:30:37 -0700 *** Bug 187659 has been marked as a duplicate of this bug. *** 1442021 3 tsavell 2018-07-13 14:35:30 -0700 Looking further me and David have found about 12 tests within the media/media-fragments/ are flaky failures for MacOS Debug WK1 media/media-fragments/TC0005.html media/media-fragments/TC0024.html media/media-fragments/TC0037.html media/media-fragments/TC0078.html media/media-fragments/TC0079.html media/media-fragments/TC0085.html media/media-fragments/TC0086.html media/media-fragments/TC0087.html media/media-fragments/TC0088.html media/media-fragments/TC0089.html media/media-fragments/TC0090.html media/media-fragments/TC0091.html 1442025 4 344984 tsavell 2018-07-13 14:44:43 -0700 Created attachment 344984 Update TestExpectations 1442049 5 344984 commit-queue 2018-07-13 15:26:16 -0700 Comment on attachment 344984 Update TestExpectations Clearing flags on attachment: 344984 Committed r233819: <https://trac.webkit.org/changeset/233819> 1442050 6 commit-queue 2018-07-13 15:26:18 -0700 All reviewed patches have been landed. Closing bug. 1442051 7 tsavell 2018-07-13 15:27:26 -0700 Reopening to track issue 1442055 8 344984 commit-queue 2018-07-13 15:28:52 -0700 Comment on attachment 344984 Update TestExpectations Rejecting attachment 344984 from commit-queue. Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-03', 'apply-attachment', '--no-update', '--non-interactive', 344984, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit Logging in as commit-queue@webkit.org... Fetching: https://bugs.webkit.org/attachment.cgi?id=344984&action=edit Fetching: https://bugs.webkit.org/show_bug.cgi?id=187557&ctype=xml&excludefield=attachmentdata Processing 1 patch from 1 bug. Processing patch 344984 from bug 187557. Fetching: https://bugs.webkit.org/attachment.cgi?id=344984 Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit Parsed 2 diffs from patch file(s). patching file LayoutTests/ChangeLog Hunk #1 succeeded at 1 with fuzz 1. patching file LayoutTests/platform/mac-wk1/TestExpectations Hunk #1 FAILED at 567. 1 out of 1 hunk FAILED -- saving rejects to file LayoutTests/platform/mac-wk1/TestExpectations.rej Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit Full output: https://webkit-queues.webkit.org/results/8529954 1442075 9 webkit-bug-importer 2018-07-13 15:53:27 -0700 <rdar://problem/42184800> 1462747 10 tsavell 2018-09-24 10:20:53 -0700 Added test expectations to iOS as these tests are flakey on iOS 12 Committed in: https://trac.webkit.org/changeset/236408/webkit 1462856 11 tsavell 2018-09-24 13:31:52 -0700 Additional tests found to be flakey failures on iOS 12: media/media-fragments/TC0011.html media/media-fragments/TC0053.html media/media-fragments/TC0036.html media/media-fragments/TC0039.html media/media-fragments/TC0072.html media/media-fragments/TC0006.html media/media-fragments/TC0038.html media/media-fragments/TC0052.html media/media-fragments/TC0051.html media/media-fragments/TC0054.html media/media-fragments/TC0055.html media/media-fragments/TC0011.html media/media-fragments/TC0059.html media/media-fragments/TC0035.html 1462859 12 tsavell 2018-09-24 13:41:46 -0700 marked remaining media-fragment/ tests as flakey in https://trac.webkit.org/changeset/236426/webkit 1463610 13 tsavell 2018-09-26 09:13:49 -0700 Marked two more tests as flakey in https://trac.webkit.org/changeset/236504/webkit Tests Marked: media/media-fragments/TC0014.html media/media-fragments/TC0009.html 1640239 14 Lawrence.j 2020-04-10 10:16:38 -0700 I have included media/media-fragments/TC0036.html in the list of flaky failures on Mac wk1. History: https://results.webkit.org/?suite=layout-tests&test=media%2Fmedia-fragments%2FTC0036.html&version_name=Mojave&style=debug&flavor=wk2&limit=50000 Expectations: https://trac.webkit.org/changeset/259881/webkit 2040111 15 ben_schwartz 2024-06-06 09:53:12 -0700 This no longer appears to be occurring on iOS. macOS wk1 has also experienced a major reduction of cases, with one in the past few months on only two of the tests. For this reason, I will consider this infrastructure flake and remove test expectations. Please file a new bug if issues persist. Tracking test expectations removal in bug #274990 (rdar://129080660). 344984 2018-07-13 14:44:43 -0700 2018-07-13 15:36:42 -0700 Update TestExpectations bug-187557-20180713144442.patch text/plain 2233 tsavell U3VidmVyc2lvbiBSZXZpc2lvbjogMjMzODAyCmRpZmYgLS1naXQgYS9MYXlvdXRUZXN0cy9DaGFu Z2VMb2cgYi9MYXlvdXRUZXN0cy9DaGFuZ2VMb2cKaW5kZXggYjRiMzU4YTdlMzc2ZjAyNzdiYzc0 NDdjYjBhOGUyMGM4MGQ2MGJiNy4uOWU1NDA3NDgwZDQyYzU0ZDUwYTYyNDgzOWE0Njk3YmE1YjA5 ZjVmMCAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvQ2hhbmdlTG9nCisrKyBiL0xheW91dFRlc3Rz L0NoYW5nZUxvZwpAQCAtMSwzICsxLDEyIEBACisyMDE4LTA3LTEzICBUcnVpdHQgU2F2ZWxsICA8 dHNhdmVsbEBhcHBsZS5jb20+CisKKyAgICAgICAgWyBNYWNPUyBXSzEgXSBMYXlvdXQgVGVzdHMg aW4gbWVkaWEvbWVkaWEtZnJhZ21lbnRzLyBhcmUgZmxha3kKKyAgICAgICAgaHR0cHM6Ly9idWdz LndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTE4NzU1NworCisgICAgICAgIFVucmV2aWV3ZWQg dGVzdCBnYXJkZW5pbmcuCisKKyAgICAgICAgKiBwbGF0Zm9ybS9tYWMtd2sxL1Rlc3RFeHBlY3Rh dGlvbnM6CisKIDIwMTgtMDctMTMgIFRydWl0dCBTYXZlbGwgIDx0c2F2ZWxsQGFwcGxlLmNvbT4K IAogICAgICAgICBbIE1hY09TIERlYnVnIF0gTGF5b3V0IFRlc3QgaW5zcGVjdG9yL3ZpZXcvYXN5 bmNocm9ub3VzLWxheW91dC5odG1sIGlzIGEgRmxha3kgVGltZW91dApkaWZmIC0tZ2l0IGEvTGF5 b3V0VGVzdHMvcGxhdGZvcm0vbWFjLXdrMS9UZXN0RXhwZWN0YXRpb25zIGIvTGF5b3V0VGVzdHMv cGxhdGZvcm0vbWFjLXdrMS9UZXN0RXhwZWN0YXRpb25zCmluZGV4IDBkMGNmY2UyZmRhODZlNTI4 OTU3MmUzMDk2Y2ZhMjQyNDJiZTJmN2UuLjcyODdjZWVkYmQwYTIzNDFhNTdjZTk2NzY3OWZjMGNj NzQwNDk3NjggMTAwNjQ0Ci0tLSBhL0xheW91dFRlc3RzL3BsYXRmb3JtL21hYy13azEvVGVzdEV4 cGVjdGF0aW9ucworKysgYi9MYXlvdXRUZXN0cy9wbGF0Zm9ybS9tYWMtd2sxL1Rlc3RFeHBlY3Rh dGlvbnMKQEAgLTU2Nyw0ICs1NjcsMTggQEAgd2Via2l0Lm9yZy9iLzE4NTQwNCBodHRwL3Rlc3Rz L3NlY3VyaXR5L2NvbnRlbnRTZWN1cml0eVBvbGljeS9ibG9jay1hbGwtbWl4ZWQtY28KIAogd2Vi a2l0Lm9yZy9iLzE4NjU4MSBbIERlYnVnIF0gZmFzdC9wYXJzZXIveG1sLWVycm9yLWFkb3B0ZWQu eG1sIFsgVGltZW91dCBQYXNzIF0KIAord2Via2l0Lm9yZy9iLzE4NzU1NyBbIERlYnVnIF0gbWVk aWEvbWVkaWEtZnJhZ21lbnRzL1RDMDAwNS5odG1sIFsgUGFzcyBGYWlsdXJlIF0KIHdlYmtpdC5v cmcvYi8xODc1MjIgWyBEZWJ1ZyBdIG1lZGlhL21lZGlhLWZyYWdtZW50cy9UQzAwMTUuaHRtbCBb IFBhc3MgRmFpbHVyZSBdCit3ZWJraXQub3JnL2IvMTg3NTU3IFsgRGVidWcgXSBtZWRpYS9tZWRp YS1mcmFnbWVudHMvVEMwMDI0Lmh0bWwgWyBQYXNzIEZhaWx1cmUgXQord2Via2l0Lm9yZy9iLzE4 NzU1NyBbIERlYnVnIF0gbWVkaWEvbWVkaWEtZnJhZ21lbnRzL1RDMDAzNy5odG1sIFsgUGFzcyBG YWlsdXJlIF0KK3dlYmtpdC5vcmcvYi8xODc1NTcgWyBEZWJ1ZyBdIG1lZGlhL21lZGlhLWZyYWdt ZW50cy9UQzAwNzguaHRtbCBbIFBhc3MgRmFpbHVyZSBdCit3ZWJraXQub3JnL2IvMTg3NTU3IFsg RGVidWcgXSBtZWRpYS9tZWRpYS1mcmFnbWVudHMvVEMwMDc5Lmh0bWwgWyBQYXNzIEZhaWx1cmUg XQord2Via2l0Lm9yZy9iLzE4NzU1NyBbIERlYnVnIF0gbWVkaWEvbWVkaWEtZnJhZ21lbnRzL1RD MDA4NS5odG1sIFsgUGFzcyBGYWlsdXJlIF0KK3dlYmtpdC5vcmcvYi8xODc1NTcgWyBEZWJ1ZyBd IG1lZGlhL21lZGlhLWZyYWdtZW50cy9UQzAwODYuaHRtbCBbIFBhc3MgRmFpbHVyZSBdCit3ZWJr aXQub3JnL2IvMTg3NTU3IFsgRGVidWcgXSBtZWRpYS9tZWRpYS1mcmFnbWVudHMvVEMwMDg3Lmh0 bWwgWyBQYXNzIEZhaWx1cmUgXQord2Via2l0Lm9yZy9iLzE4NzU1NyBbIERlYnVnIF0gbWVkaWEv bWVkaWEtZnJhZ21lbnRzL1RDMDA4OC5odG1sIFsgUGFzcyBGYWlsdXJlIF0KK3dlYmtpdC5vcmcv Yi8xODc1NTcgWyBEZWJ1ZyBdIG1lZGlhL21lZGlhLWZyYWdtZW50cy9UQzAwODkuaHRtbCBbIFBh c3MgRmFpbHVyZSBdCit3ZWJraXQub3JnL2IvMTg3NTU3IFsgRGVidWcgXSBtZWRpYS9tZWRpYS1m cmFnbWVudHMvVEMwMDkwLmh0bWwgWyBQYXNzIEZhaWx1cmUgXQord2Via2l0Lm9yZy9iLzE4NzU1 NyBbIERlYnVnIF0gbWVkaWEvbWVkaWEtZnJhZ21lbnRzL1RDMDA5MS5odG1sIFsgUGFzcyBGYWls dXJlIF0KKworCg==