Summary: | Multiple Layout Tests from media/ tests have become flaky. | ||
---|---|---|---|
Product: | WebKit | Reporter: | Matt Lewis <jlewis3> |
Component: | New Bugs | Assignee: | Nobody <webkit-unassigned> |
Status: | NEW --- | ||
Severity: | Normal | CC: | ap, graouts, ryanhaddad, webkit-bug-importer, youennf |
Priority: | P2 | Keywords: | InRadar |
Version: | WebKit Nightly Build | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
See Also: |
https://bugs.webkit.org/show_bug.cgi?id=175700 https://bugs.webkit.org/show_bug.cgi?id=176054 https://bugs.webkit.org/show_bug.cgi?id=176122 |
Description
Matt Lewis
2017-06-29 11:42:11 PDT
https://trac.webkit.org/changeset/218903/webkit is only modifying werbrtc which is not tested in the media/xx tests. Maybe antoine will know more here. imported/w3c/web-platform-tests/webrtc/simplecall.html is also failing there. Maybe this is a regression there. What's the next step on this? Should we roll back to confirm whether r218903 was the culprit? The first failure on internal bots was with r218901. A couple preceding changes that may be relevant: https://trac.webkit.org/r218892 https://trac.webkit.org/r218891 As there are multiple tests failing in different ways, it seems like it should be possible to figure out what's going wrong from the failures. Antoine, what do these failures imply? https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r218928%20(2615)/results.html None of the tests listed above use modern media controls, they all have <!-- webkit-test-runner [ enableModernMediaControls=false ] -->. Maybe something else broke that makes DRT/WKTR ignore these flags? (In reply to Alexey Proskuryakov from comment #4) > As there are multiple tests failing in different ways, it seems like it > should be possible to figure out what's going wrong from the failures. > Antoine, what do these failures imply? The failures are consistent with modern media controls being enabled. Is there a revision that points towards a change in behaviour in dealing with the webkit-test-runner flags? Furthermore, it seems that when this happens, all the affected tests are running in the same shard. I haven't been able to reproduce the failures locally over 25 runs with 6 WKTR instances. Did you try running the tests in the same order as the failing bot? (In reply to Alexey Proskuryakov from comment #10) > Did you try running the tests in the same order as the failing bot? Yes, I ran them in the same order as the initial issue reported, using all the tests ran for worker/0 in https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK2%20%28Tests%29/builds/2615/steps/layout-test/logs/stdio, and I didn't see the failures. I'll try again across multiple runs. Examples from today: https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r221186%20(2720)/results.html https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r221185%20(2719)/results.html https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r221190%20(3851)/results.html |