Bug 231084 – Regression(r282374): [ macOS ] imported/w3c/web-platform-tests/html/semantics/embedded-content/the-video-element/resize-during-playback.html is a flaky failure

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

imported/w3c/web-platform-tests/html/semantics/embedded-content/the-video-element/resize-during-playback.html

Is a flaky failure on macOS.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fembedded-content%2Fthe-video-element%2Fresize-during-playback.html

Result page: https://build.webkit.org/results/Apple-BigSur-Debug-WK1-Tests/r283362%20(4465)/results.html

Diff:

--- /Volumes/Data/worker/bigsur-debug-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/the-video-element/resize-during-playback-expected.txt
+++ /Volumes/Data/worker/bigsur-debug-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/the-video-element/resize-during-playback-actual.txt
@@ -1,4 +1,4 @@
 
-FAIL mp4 video assert_equals: Expected resize event, but got ended event instead expected "resize" but got "ended"
+FAIL mp4 video assert_equals: Expected resize event, but got playing event instead expected "resize" but got "playing"
 NOTRUN webm video webm supported

[Radar WebKit Bug Importer]

<rdar://problem/83770133>

Marked test expectations https://trac.webkit.org/changeset/283381/webkit

I was able to reproduce the failure at TOT locally on BigSur using run-webkit-tests --force -f -1 --iterations 50 --exit-after-n-failures 1 --exit-after-n-crashes-or-timeouts 1 --debug imported/w3c/web-platform-tests/html/semantics/embedded-content/the-video-element/resize-during-playback.html --no-timeout

The test passed with r282373 and failed with r282374.

With r282374 ~ r282379 (didn't test newer than r282379), the test failed with different diff than TOT:

--- /Volumes/Data/Builds/BigSur/debug-282374/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/the-video-element/resize-during-playback-expected.txt
+++ /Volumes/Data/Builds/BigSur/debug-282374/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/the-video-element/resize-during-playback-actual.txt
@@ -1,4 +1,7 @@
-
-FAIL mp4 video assert_equals: Expected resize event, but got ended event instead expected "resize" but got "ended"
-NOTRUN webm video webm supported
-
+CONSOLE MESSAGE: ReferenceError: Can't find variable: promise_test
+layer at (0,0) size 800x600
+  RenderView at (0,0) size 800x600
+layer at (0,0) size 800x8
+  RenderBlock {HTML} at (0,0) size 800x8
+    RenderBody {BODY} at (8,8) size 784x0
+      RenderBlock {DIV} at (0,0) size 784x0

[Jean-Yves Avenard [:jya]]

I’ve been unable to reproduce the bug on either MBP 14” (AS) or iMac Pro.
```
run-webkit-tests --debug --no-build --force -1 --repeat 1000 --exit-after-n-failures=1 --no-timeout LayoutTests/imported/w3c/web-platform-tests/html/semantics/embedded-content/the-video-element/resize-during-playback.html
All 1000 tests ran as expected.  
```

Code was modified so that webm is disabled to reproduce the testing conditions where the MediaFormatReader plugin is disabled.
I don’t believe the regression point is valid, particularly because the test will fail due to webm being enabled by default on a local or spade build which is what we are seeing in comment 4.

In comment 4, the error produced is also different.

[Jean-Yves Avenard [:jya]]

This test is failing because we do not support change of resolution mid-stream and we reach the end of the video without firing the "resize" event.

The test calls play() on the video element and wait for both `resize` and `playing` event; each event can only be fired once without causing an error.

in bug 230210 (r282374) we properly fixed stall detections.
If a stall occurs during playback (due to the system being overloaded or whatever), the readyState will move from HAVE_FUTURE_DATA to HAVE_CURRENT_DATA and the `stalled` event will be fired. [1]

When re-buffering completed and the readyState moves again from HAVE_CURRENT_DATA to HAVE_ENOUGH_DATA (or more per spec: we are to `notify about playing` [2] and fire the `playing` event.

The test doesn't handle this case, it assumes that playback will not stall and continue uninterrupted; there's no guarantee this will always be the case. It will happen more likely with WK1 as there's less parallel processing.

in bug 230210 (r282374) we properly fixed stall detections. We have a correct progression in the behaviour simply exposing an existing flaw in the test.

Seeing that the test can't pass as we do not support the feature tested, we should just skip this test.

[1] https://html.spec.whatwg.org/multipage/media.html#ready-states
[2] https://html.spec.whatwg.org/multipage/media.html#notify-about-playingv

[Jean-Yves Avenard [:jya]]

Created attachment 443759 [details]
Patch

[Jean-Yves Avenard [:jya]]

Created attachment 443762 [details]
[fast-cq] Patch

[EWS]

Committed r285554 (244067@main): <https://commits.webkit.org/244067@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 443762 [details].