173978 2017-06-29 11:42:11 -0700 Multiple Layout Tests from media/ tests have become flaky. 2026-01-12 09:09:22 -0800 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=175700 https://bugs.webkit.org/show_bug.cgi?id=176054 https://bugs.webkit.org/show_bug.cgi?id=176122 InRadar P2 Normal --- 1 jlewis3 webkit-unassigned ap graouts ryanhaddad webkit-bug-importer youennf oldest_to_newest 1324014 0 jlewis3 2017-06-29 11:42:11 -0700 Multiple test from LayoutTest/media/ have become flaky after this revision and are failing together: https://trac.webkit.org/changeset/218903/webkit media/video-click-dblckick-standalone.html [ Failure ] media/video-controls-audiotracks-trackmenu.html [ Failure ] media/video-controls-captions-trackmenu-localized.html [ Failure ] media/video-controls-captions-trackmenu-only-captions-descriptions-and-subtitles.html [ Failure ] media/video-controls-captions-trackmenu-sorted.html [ Failure ] media/video-controls-captions-trackmenu.html [ Failure ] media/video-controls-drop-and-restore-timeline.html [ Failure ] media/video-controls-in-media-document.html [ Failure ] media/video-controls-no-display-with-text-track.html [ Failure ] media/video-controls-rendering.html [ Failure ] https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r218928%20(2615)/results.html https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK2%20%28Tests%29/builds/2615 1324015 1 youennf 2017-06-29 11:45:32 -0700 https://trac.webkit.org/changeset/218903/webkit is only modifying werbrtc which is not tested in the media/xx tests. Maybe antoine will know more here. imported/w3c/web-platform-tests/webrtc/simplecall.html is also failing there. Maybe this is a regression there. 1324911 2 ap 2017-07-02 02:11:25 -0700 What's the next step on this? Should we roll back to confirm whether r218903 was the culprit? 1324936 3 youennf 2017-07-02 10:19:22 -0700 r218903 was rolled out and re landed successfully. 1325058 4 ap 2017-07-03 01:13:49 -0700 The first failure on internal bots was with r218901. A couple preceding changes that may be relevant: https://trac.webkit.org/r218892 https://trac.webkit.org/r218891 As there are multiple tests failing in different ways, it seems like it should be possible to figure out what's going wrong from the failures. Antoine, what do these failures imply? https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r218928%20(2615)/results.html 1325059 5 ap 2017-07-03 01:14:49 -0700 <rdar://problem/33103102> 1325609 6 graouts 2017-07-05 10:03:55 -0700 None of the tests listed above use modern media controls, they all have <!-- webkit-test-runner [ enableModernMediaControls=false ] -->. Maybe something else broke that makes DRT/WKTR ignore these flags? 1325614 7 graouts 2017-07-05 10:13:23 -0700 (In reply to Alexey Proskuryakov from comment #4) > As there are multiple tests failing in different ways, it seems like it > should be possible to figure out what's going wrong from the failures. > Antoine, what do these failures imply? The failures are consistent with modern media controls being enabled. Is there a revision that points towards a change in behaviour in dealing with the webkit-test-runner flags? 1325626 8 graouts 2017-07-05 10:23:30 -0700 Furthermore, it seems that when this happens, all the affected tests are running in the same shard. 1325985 9 graouts 2017-07-06 02:47:24 -0700 I haven't been able to reproduce the failures locally over 25 runs with 6 WKTR instances. 1326120 10 ap 2017-07-06 11:51:45 -0700 Did you try running the tests in the same order as the failing bot? 1326419 11 graouts 2017-07-07 01:01:56 -0700 (In reply to Alexey Proskuryakov from comment #10) > Did you try running the tests in the same order as the failing bot? Yes, I ran them in the same order as the initial issue reported, using all the tests ran for worker/0 in https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK2%20%28Tests%29/builds/2615/steps/layout-test/logs/stdio, and I didn't see the failures. I'll try again across multiple runs. 1342456 12 ryanhaddad 2017-08-25 11:35:45 -0700 Examples from today: https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r221186%20(2720)/results.html https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r221185%20(2719)/results.html https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r221190%20(3851)/results.html