media/navigate-with-pip-should-not-crash.html will intermittently fail. Currently the dashboard states it is only failing on Sierra WK2 Release. https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Fnavigate-with-pip-should-not-crash.html The test first failed on 5/25/2017 with this build: https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK2%20(Tests)/builds/1776 Current build failure: https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r217945%20(2084)/results.html https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK2%20(Tests)/builds/2084 Diff: --- /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/media/navigate-with-pip-should-not-crash-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/media/navigate-with-pip-should-not-crash-actual.txt @@ -5,6 +5,8 @@ Going into Picture-in-Picture EXPECTED (mediaElement.webkitPresentationMode == 'picture-in-picture') OK EXPECTED (mediaElement.paused == 'false') OK +EXPECTED (mediaElement.webkitPresentationMode == 'picture-in-picture'), OBSERVED 'inline' FAIL +EXPECTED (mediaElement.paused == 'false') OK EXPECTED (videoframe.contentDocument.location == 'about:blank') OK END OF TEST
EXPECTED (mediaElement.paused == 'false') OK +EXPECTED (mediaElement.webkitPresentationMode == 'picture-in-picture'), OBSERVED 'inline' FAIL +EXPECTED (mediaElement.paused == 'false') OK Does this mean that we are getting extra callbacks? The only change is that we get extra checks that fail, no existing ones fail.
This test has become far more flaky recently. https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=media%2Fnavigate-with-pip-should-not-crash.html --- /Volumes/Data/slave/highsierra-debug-tests-wk2/build/layout-test-results/media/navigate-with-pip-should-not-crash-expected.txt +++ /Volumes/Data/slave/highsierra-debug-tests-wk2/build/layout-test-results/media/navigate-with-pip-should-not-crash-actual.txt @@ -5,6 +5,10 @@ Going into Picture-in-Picture EXPECTED (mediaElement.webkitPresentationMode == 'picture-in-picture') OK EXPECTED (mediaElement.paused == 'false') OK +EXPECTED (mediaElement.webkitPresentationMode == 'picture-in-picture'), OBSERVED 'inline' FAIL +EXPECTED (mediaElement.paused == 'false') OK +EXPECTED (mediaElement.webkitPresentationMode == 'picture-in-picture'), OBSERVED 'inline' FAIL +EXPECTED (mediaElement.paused == 'false') OK EXPECTED (videoframe.contentDocument.location == 'about:blank') OK END OF TEST
Marked test as flaky in https://trac.webkit.org/r231363