Bug 194770 - Update WPT tests for embedded content
Summary: Update WPT tests for embedded content
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Frédéric Wang (:fredw)
URL:
Keywords: InRadar
Depends on: 195331 195358
Blocks:
  Show dependency treegraph
 
Reported: 2019-02-18 01:05 PST by Frédéric Wang (:fredw)
Modified: 2019-03-08 10:14 PST (History)
9 users (show)

See Also:


Attachments
Patch (974.49 KB, patch)
2019-02-18 01:07 PST, Frédéric Wang (:fredw)
no flags Details | Formatted Diff | Diff
Patch (994.94 KB, patch)
2019-02-18 01:39 PST, Frédéric Wang (:fredw)
no flags Details | Formatted Diff | Diff
Patch - trying with only media elements (733.43 KB, patch)
2019-02-18 02:19 PST, Frédéric Wang (:fredw)
no flags Details | Formatted Diff | Diff
Patch (applies on top of 362274) (205.78 KB, patch)
2019-02-18 02:39 PST, Frédéric Wang (:fredw)
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews103 for mac-highsierra (1.56 MB, application/zip)
2019-02-18 03:02 PST, Build Bot
no flags Details
Archive of layout-test-results from ews104 for mac-highsierra-wk2 (1.61 MB, application/zip)
2019-02-18 03:26 PST, Build Bot
no flags Details
Archive of layout-test-results from ews116 for mac-highsierra (1.02 MB, application/zip)
2019-02-18 03:39 PST, Build Bot
no flags Details
Patch (1.33 MB, patch)
2019-02-18 05:44 PST, Frédéric Wang (:fredw)
no flags Details | Formatted Diff | Diff
Patch (1.29 MB, patch)
2019-02-27 08:23 PST, Frédéric Wang (:fredw)
no flags Details | Formatted Diff | Diff
Patch (622.22 KB, patch)
2019-03-06 08:22 PST, Frédéric Wang (:fredw)
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews103 for mac-highsierra (2.44 MB, application/zip)
2019-03-06 09:29 PST, Build Bot
no flags Details
Archive of layout-test-results from ews112 for mac-highsierra (2.28 MB, application/zip)
2019-03-06 10:13 PST, Build Bot
no flags Details
Patch (664.47 KB, patch)
2019-03-06 12:59 PST, Frédéric Wang (:fredw)
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Frédéric Wang (:fredw) 2019-02-18 01:05:33 PST
Import WPT tests for embedded content
Comment 1 Frédéric Wang (:fredw) 2019-02-18 01:07:26 PST
Created attachment 362269 [details]
Patch
Comment 2 Frédéric Wang (:fredw) 2019-02-18 01:39:54 PST
Created attachment 362272 [details]
Patch
Comment 3 Frédéric Wang (:fredw) 2019-02-18 02:19:34 PST
Created attachment 362274 [details]
Patch - trying with only media elements
Comment 4 Frédéric Wang (:fredw) 2019-02-18 02:39:36 PST
Created attachment 362280 [details]
Patch (applies on top of 362274)
Comment 5 Build Bot 2019-02-18 03:02:50 PST
Comment on attachment 362274 [details]
Patch - trying with only media elements

Attachment 362274 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/11190055

Number of test failures exceeded the failure limit.
Comment 6 Build Bot 2019-02-18 03:02:52 PST
Created attachment 362284 [details]
Archive of layout-test-results from ews103 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-highsierra  Platform: Mac OS X 10.13.6
Comment 7 Build Bot 2019-02-18 03:25:59 PST
Comment on attachment 362274 [details]
Patch - trying with only media elements

Attachment 362274 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11190251

Number of test failures exceeded the failure limit.
Comment 8 Build Bot 2019-02-18 03:26:01 PST
Created attachment 362285 [details]
Archive of layout-test-results from ews104 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6
Comment 9 Build Bot 2019-02-18 03:39:48 PST
Comment on attachment 362274 [details]
Patch - trying with only media elements

Attachment 362274 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/11190258

Number of test failures exceeded the failure limit.
Comment 10 Build Bot 2019-02-18 03:39:50 PST
Created attachment 362286 [details]
Archive of layout-test-results from ews116 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-highsierra  Platform: Mac OS X 10.13.6
Comment 11 Frédéric Wang (:fredw) 2019-02-18 05:44:09 PST
Created attachment 362292 [details]
Patch
Comment 12 Frédéric Wang (:fredw) 2019-02-18 07:31:18 PST
Comment on attachment 362292 [details]
Patch

@Youenn: I really don't know what's preventing the patch to apply on all ports but wincairo. I executed the tests on mac-wk2 and ios but I think we'll have to land the patch manually and make update expectations on other ports if necessary.
Comment 13 Frédéric Wang (:fredw) 2019-02-27 08:23:39 PST
Created attachment 363091 [details]
Patch
Comment 14 Frédéric Wang (:fredw) 2019-03-05 06:31:22 PST
@Youenn: Any idea why this patch cannot apply on most ports but wincairo? It's the first time I see that. Maybe it's just because the patch is too big and we should split import into smaller steps (trying only media elements in attachment 362274 [details] seemed to help).
Comment 15 youenn fablet 2019-03-05 07:16:34 PST
I do not think this is the size.
It might be a bad character in a file or something like that.
One way to investigate this is to download the patch and locally apply it using Tools/Scripts/svn-apply.
Comment 16 youenn fablet 2019-03-05 07:26:05 PST
imported/w3c/web-platform-tests/html/semantics/embedded-content/the-canvas-element/size.attributes.parse.whitespace.html might have some issues.
Comment 17 Frédéric Wang (:fredw) 2019-03-05 07:59:45 PST
(In reply to youenn fablet from comment #15)
> I do not think this is the size.
> It might be a bad character in a file or something like that.
> One way to investigate this is to download the patch and locally apply it
> using Tools/Scripts/svn-apply.

Ah, thanks for the hint. I generally use git apply, so didn't have the issue :-)

(In reply to youenn fablet from comment #16)
> imported/w3c/web-platform-tests/html/semantics/embedded-content/the-canvas-
> element/size.attributes.parse.whitespace.html might have some issues.

I'll check that later.
Comment 18 Frédéric Wang (:fredw) 2019-03-06 08:22:35 PST
Created attachment 363742 [details]
Patch
Comment 19 Build Bot 2019-03-06 09:29:15 PST
Comment on attachment 363742 [details]
Patch

Attachment 363742 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/11397716

New failing tests:
imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_anchor_download_allow_downloads_without_user_activation.sub.tentative.html
imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_anchor_download_block_downloads_without_user_activation.sub.tentative.html
Comment 20 Build Bot 2019-03-06 09:29:18 PST
Created attachment 363749 [details]
Archive of layout-test-results from ews103 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-highsierra  Platform: Mac OS X 10.13.6
Comment 21 Build Bot 2019-03-06 10:13:32 PST
Comment on attachment 363742 [details]
Patch

Attachment 363742 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/11397813

New failing tests:
imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_anchor_download_allow_downloads_without_user_activation.sub.tentative.html
imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/iframe_sandbox_anchor_download_block_downloads_without_user_activation.sub.tentative.html
Comment 22 Build Bot 2019-03-06 10:13:33 PST
Created attachment 363754 [details]
Archive of layout-test-results from ews112 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews112  Port: mac-highsierra  Platform: Mac OS X 10.13.6
Comment 23 Frédéric Wang (:fredw) 2019-03-06 12:59:02 PST
Created attachment 363774 [details]
Patch
Comment 24 WebKit Commit Bot 2019-03-07 01:28:53 PST
Comment on attachment 363774 [details]
Patch

Clearing flags on attachment: 363774

Committed r242595: <https://trac.webkit.org/changeset/242595>
Comment 25 WebKit Commit Bot 2019-03-07 01:28:54 PST
All reviewed patches have been landed.  Closing bug.
Comment 26 Radar WebKit Bug Importer 2019-03-07 01:54:36 PST
<rdar://problem/48669509>
Comment 27 Truitt Savell 2019-03-07 09:14:54 PST
Looks like the test imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay.html

added in https://trac.webkit.org/changeset/242595/webkit

is failing constantly. History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fembedded-content%2Fmedia-elements%2Fready-states%2Fautoplay.html

Diff:
--- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay-actual.txt
@@ -1,12 +1,12 @@
 
 FAIL audio.autoplay assert_array_equals: property 1, expected "play" but got "canplaythrough"
 FAIL audio.autoplay and load() assert_array_equals: property 1, expected "play" but got "canplaythrough"
-FAIL audio.autoplay and play() assert_array_equals: property 2, expected "playing" but got "canplaythrough"
+PASS audio.autoplay and play() 
 PASS audio.autoplay and pause() 
 PASS audio.autoplay and internal pause steps 
 FAIL video.autoplay assert_array_equals: property 1, expected "play" but got "canplaythrough"
 FAIL video.autoplay and load() assert_array_equals: property 1, expected "play" but got "canplaythrough"
-FAIL video.autoplay and play() assert_array_equals: property 2, expected "playing" but got "canplaythrough"
+PASS video.autoplay and play() 
 PASS video.autoplay and pause() 
 PASS video.autoplay and internal pause steps
Comment 28 Truitt Savell 2019-03-07 10:31:30 PST
This test is only effecting Mojave which is why it did not show up on EWS
Comment 29 Truitt Savell 2019-03-07 10:50:40 PST
This test is also failing: imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/error-codes/error.html

History:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fembedded-content%2Fmedia-elements%2Ferror-codes%2Ferror.html
Comment 30 Truitt Savell 2019-03-08 10:14:38 PST
Filed this to track those two test failures:
https://bugs.webkit.org/show_bug.cgi?id=195466