Bug 161067 – Import html/browsers web platform tests

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Chris Dumez]

Import html/browsers web platform tests

[Chris Dumez]

Created attachment 286649 [details]
Patch

[Build Bot]

Comment on attachment 286649 [details]
Patch

Attachment 286649 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/1924815

Number of test failures exceeded the failure limit.

[Build Bot]

Created attachment 286654 [details]
Archive of layout-test-results from ews103 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 286649 [details]
Patch

Attachment 286649 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1924827

Number of test failures exceeded the failure limit.

[Build Bot]

Created attachment 286656 [details]
Archive of layout-test-results from ews114 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 286649 [details]
Patch

Attachment 286649 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/1924881

New failing tests:
imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects.sub.html

[Build Bot]

Created attachment 286657 [details]
Archive of layout-test-results from ews104 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 286649 [details]
Patch

Attachment 286649 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/1924867

New failing tests:
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-fragment-scrolling-cross-origin.html
imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects.sub.html
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-navigation-samedoc.html
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-navigation-cross-origin.html
imported/w3c/web-platform-tests/html/browsers/windows/noreferrer-window-name.html

[Build Bot]

Created attachment 286658 [details]
Archive of layout-test-results from ews124 for ios-simulator-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews124  Port: ios-simulator-elcapitan-wk2  Platform: Mac OS X 10.11.5

[Chris Dumez]

Created attachment 286665 [details]
Patch

[Build Bot]

Comment on attachment 286665 [details]
Patch

Attachment 286665 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1925444

New failing tests:
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/unloading-documents/004.html

[Build Bot]

Created attachment 286674 [details]
Archive of layout-test-results from ews114 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 286665 [details]
Patch

Attachment 286665 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/1925474

New failing tests:
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-navigation-cross-origin.html
imported/w3c/web-platform-tests/html/browsers/windows/noreferrer-window-name.html
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-fragment-scrolling-cross-origin.html
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-navigation-samedoc.html

[Build Bot]

Created attachment 286675 [details]
Archive of layout-test-results from ews123 for ios-simulator-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews123  Port: ios-simulator-elcapitan-wk2  Platform: Mac OS X 10.11.5

[Chris Dumez]

Created attachment 286717 [details]
Patch

[youenn fablet]

Comment on attachment 286717 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=286717&action=review

rs=me.

When including a new test suite, it would be nice to update LayoutTests/imported/w3c/resources/ImportExpectations, so that next sync will also pick the changes for these tests.
In this case, it would mean removing line 142 of ImportExpectations.

> LayoutTests/TestExpectations:258
> +imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/001-1.html [ Skip ]

That is a lot of resource files to skip.
I wonder whether we should not move these "expectations" to a different file that would list the "resources" files, not actual tests.
This file could be automatically generated by the test importer script.

> LayoutTests/platform/wk2/TestExpectations:687
> +# Fails with WK2 only, it does not print the latest line.

Is it a test issue or a WTR issue?

[Chris Dumez]

(In reply to comment #16)
> Comment on attachment 286717 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=286717&action=review
> 
> rs=me.
> 
> When including a new test suite, it would be nice to update
> LayoutTests/imported/w3c/resources/ImportExpectations, so that next sync
> will also pick the changes for these tests.
> In this case, it would mean removing line 142 of ImportExpectations.
> 
> > LayoutTests/TestExpectations:258
> > +imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/001-1.html [ Skip ]
> 
> That is a lot of resource files to skip.
> I wonder whether we should not move these "expectations" to a different file
> that would list the "resources" files, not actual tests.
> This file could be automatically generated by the test importer script.
> 
> > LayoutTests/platform/wk2/TestExpectations:687
> > +# Fails with WK2 only, it does not print the latest line.
> 
> Is it a test issue or a WTR issue?

I don't know yet. I need to investigate. The tests started working on WK1 when I started calling testRunner.setCanOpenWindows() but this did not work for WK2 for some reason.

[Chris Dumez]

Created attachment 286729 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 286729 [details]
Patch

Rejecting attachment 286729 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-02', 'validate-changelog', '--check-oops', '--non-interactive', 286729, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

ChangeLog entry in LayoutTests/ChangeLog contains OOPS!.

Full output: http://webkit-queues.webkit.org/results/1928668

[Chris Dumez]

Created attachment 286733 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 286733 [details]
Patch

Clearing flags on attachment: 286733

Committed r204844: <http://trac.webkit.org/changeset/204844>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.