Bug 194770 – Update WPT tests for embedded content

• Component: New Bugs
• Status: RESOLVED FIXED

[Frédéric Wang (:fredw)]

Import WPT tests for embedded content

[Frédéric Wang (:fredw)]

Created attachment 362269 [details]
Patch

[Frédéric Wang (:fredw)]

Created attachment 362272 [details]
Patch

[Frédéric Wang (:fredw)]

Created attachment 362274 [details]
Patch - trying with only media elements

[Frédéric Wang (:fredw)]

Created attachment 362280 [details]
Patch (applies on top of 362274)

[EWS Watchlist]

Comment on attachment 362274 [details]
Patch - trying with only media elements

Attachment 362274 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/11190055

Number of test failures exceeded the failure limit.

[EWS Watchlist]

Created attachment 362284 [details]
Archive of layout-test-results from ews103 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-highsierra  Platform: Mac OS X 10.13.6

[EWS Watchlist]

Comment on attachment 362274 [details]
Patch - trying with only media elements

Attachment 362274 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11190251

Number of test failures exceeded the failure limit.

[EWS Watchlist]

Created attachment 362285 [details]
Archive of layout-test-results from ews104 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6

[EWS Watchlist]

Comment on attachment 362274 [details]
Patch - trying with only media elements

Attachment 362274 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/11190258

Number of test failures exceeded the failure limit.

[EWS Watchlist]

Created attachment 362286 [details]
Archive of layout-test-results from ews116 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-highsierra  Platform: Mac OS X 10.13.6

[Frédéric Wang (:fredw)]

Created attachment 362292 [details]
Patch

[Frédéric Wang (:fredw)]

Comment on attachment 362292 [details]
Patch

@Youenn: I really don't know what's preventing the patch to apply on all ports but wincairo. I executed the tests on mac-wk2 and ios but I think we'll have to land the patch manually and make update expectations on other ports if necessary.

[Frédéric Wang (:fredw)]

Created attachment 363091 [details]
Patch

[Frédéric Wang (:fredw)]

@Youenn: Any idea why this patch cannot apply on most ports but wincairo? It's the first time I see that. Maybe it's just because the patch is too big and we should split import into smaller steps (trying only media elements in attachment 362274 [details] seemed to help).

[youenn fablet]

I do not think this is the size.
It might be a bad character in a file or something like that.
One way to investigate this is to download the patch and locally apply it using Tools/Scripts/svn-apply.

[youenn fablet]

imported/w3c/web-platform-tests/html/semantics/embedded-content/the-canvas-element/size.attributes.parse.whitespace.html might have some issues.

[Frédéric Wang (:fredw)]

(In reply to youenn fablet from comment #15)
> I do not think this is the size.
> It might be a bad character in a file or something like that.
> One way to investigate this is to download the patch and locally apply it
> using Tools/Scripts/svn-apply.

Ah, thanks for the hint. I generally use git apply, so didn't have the issue :-)

(In reply to youenn fablet from comment #16)
> imported/w3c/web-platform-tests/html/semantics/embedded-content/the-canvas-
> element/size.attributes.parse.whitespace.html might have some issues.

I'll check that later.

[Frédéric Wang (:fredw)]

Created attachment 363742 [details]
Patch

[EWS Watchlist]

Comment on attachment 363742 [details]
Patch

Attachment 363742 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/11397716

New failing tests:
imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_anchor_download_allow_downloads_without_user_activation.sub.tentative.html
imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_anchor_download_block_downloads_without_user_activation.sub.tentative.html

[EWS Watchlist]

Created attachment 363749 [details]
Archive of layout-test-results from ews103 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-highsierra  Platform: Mac OS X 10.13.6

[EWS Watchlist]

Comment on attachment 363742 [details]
Patch

Attachment 363742 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/11397813

New failing tests:
imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_anchor_download_allow_downloads_without_user_activation.sub.tentative.html
imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_anchor_download_block_downloads_without_user_activation.sub.tentative.html

[EWS Watchlist]

Created attachment 363754 [details]
Archive of layout-test-results from ews112 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews112  Port: mac-highsierra  Platform: Mac OS X 10.13.6

[Frédéric Wang (:fredw)]

Created attachment 363774 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 363774 [details]
Patch

Clearing flags on attachment: 363774

Committed r242595: <https://trac.webkit.org/changeset/242595>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/48669509>

[Truitt Savell]

Looks like the test imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay.html

added in https://trac.webkit.org/changeset/242595/webkit

is failing constantly. History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fembedded-content%2Fmedia-elements%2Fready-states%2Fautoplay.html

Diff:
--- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-actual.txt
@@ -1,12 +1,12 @@
 
 FAIL audio.autoplay assert_array_equals: property 1, expected "play" but got "canplaythrough"
 FAIL audio.autoplay and load() assert_array_equals: property 1, expected "play" but got "canplaythrough"
-FAIL audio.autoplay and play() assert_array_equals: property 2, expected "playing" but got "canplaythrough"
+PASS audio.autoplay and play() 
 PASS audio.autoplay and pause() 
 PASS audio.autoplay and internal pause steps 
 FAIL video.autoplay assert_array_equals: property 1, expected "play" but got "canplaythrough"
 FAIL video.autoplay and load() assert_array_equals: property 1, expected "play" but got "canplaythrough"
-FAIL video.autoplay and play() assert_array_equals: property 2, expected "playing" but got "canplaythrough"
+PASS video.autoplay and play() 
 PASS video.autoplay and pause() 
 PASS video.autoplay and internal pause steps

[Truitt Savell]

This test is only effecting Mojave which is why it did not show up on EWS

[Truitt Savell]

This test is also failing: imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/error-codes/error.html

History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fembedded-content%2Fmedia-elements%2Ferror-codes%2Ferror.html

[Truitt Savell]

Filed this to track those two test failures:
https://bugs.webkit.org/show_bug.cgi?id=195466

[Alexey Proskuryakov]

Comment on attachment 363774 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=363774&action=review

> LayoutTests/imported/w3c/web-platform-tests/html/semantics/embedded-content/the-embed-element/embed-represent-nothing-04.html:17
> +  <object type="application/x-shockwave-flash">

This is a bad test, because it depends on Flash. In fact, WebKitTestRunner release asserts when running it on a machine with Flash installed.