Bug 161067 - Import html/browsers web platform tests
Summary: Import html/browsers web platform tests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Chris Dumez
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-08-22 18:50 PDT by Chris Dumez
Modified: 2016-08-24 10:12 PDT (History)
7 users (show)

See Also:


Attachments
Patch (787.58 KB, patch)
2016-08-22 19:16 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews103 for mac-yosemite (1.03 MB, application/zip)
2016-08-22 20:01 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews114 for mac-yosemite (1.56 MB, application/zip)
2016-08-22 20:10 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews104 for mac-yosemite-wk2 (949.73 KB, application/zip)
2016-08-22 20:20 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews124 for ios-simulator-elcapitan-wk2 (873.67 KB, application/zip)
2016-08-22 20:28 PDT, Build Bot
no flags Details
Patch (767.10 KB, patch)
2016-08-22 21:15 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews114 for mac-yosemite (1.44 MB, application/zip)
2016-08-22 22:26 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews123 for ios-simulator-elcapitan-wk2 (769.75 KB, application/zip)
2016-08-22 22:34 PDT, Build Bot
no flags Details
Patch (772.22 KB, patch)
2016-08-23 09:06 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff
Patch (773.08 KB, patch)
2016-08-23 10:35 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff
Patch (773.07 KB, patch)
2016-08-23 10:52 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Dumez 2016-08-22 18:50:07 PDT
Import html/browsers web platform tests
Comment 1 Chris Dumez 2016-08-22 19:16:17 PDT
Created attachment 286649 [details]
Patch
Comment 2 Build Bot 2016-08-22 20:01:18 PDT
Comment on attachment 286649 [details]
Patch

Attachment 286649 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/1924815

Number of test failures exceeded the failure limit.
Comment 3 Build Bot 2016-08-22 20:01:22 PDT
Created attachment 286654 [details]
Archive of layout-test-results from ews103 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 4 Build Bot 2016-08-22 20:10:03 PDT
Comment on attachment 286649 [details]
Patch

Attachment 286649 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1924827

Number of test failures exceeded the failure limit.
Comment 5 Build Bot 2016-08-22 20:10:06 PDT
Created attachment 286656 [details]
Archive of layout-test-results from ews114 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 6 Build Bot 2016-08-22 20:20:43 PDT
Comment on attachment 286649 [details]
Patch

Attachment 286649 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/1924881

New failing tests:
imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects.sub.html
Comment 7 Build Bot 2016-08-22 20:20:47 PDT
Created attachment 286657 [details]
Archive of layout-test-results from ews104 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5
Comment 8 Build Bot 2016-08-22 20:28:14 PDT
Comment on attachment 286649 [details]
Patch

Attachment 286649 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/1924867

New failing tests:
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-fragment-scrolling-cross-origin.html
imported/w3c/web-platform-tests/html/browsers/origin/cross-origin-objects/cross-origin-objects.sub.html
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-navigation-samedoc.html
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-navigation-cross-origin.html
imported/w3c/web-platform-tests/html/browsers/windows/noreferrer-window-name.html
Comment 9 Build Bot 2016-08-22 20:28:17 PDT
Created attachment 286658 [details]
Archive of layout-test-results from ews124 for ios-simulator-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews124  Port: ios-simulator-elcapitan-wk2  Platform: Mac OS X 10.11.5
Comment 10 Chris Dumez 2016-08-22 21:15:47 PDT
Created attachment 286665 [details]
Patch
Comment 11 Build Bot 2016-08-22 22:26:53 PDT
Comment on attachment 286665 [details]
Patch

Attachment 286665 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1925444

New failing tests:
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/unloading-documents/004.html
Comment 12 Build Bot 2016-08-22 22:26:56 PDT
Created attachment 286674 [details]
Archive of layout-test-results from ews114 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 13 Build Bot 2016-08-22 22:34:12 PDT
Comment on attachment 286665 [details]
Patch

Attachment 286665 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/1925474

New failing tests:
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-navigation-cross-origin.html
imported/w3c/web-platform-tests/html/browsers/windows/noreferrer-window-name.html
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-fragment-scrolling-cross-origin.html
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/persisted-user-state-restoration/scroll-restoration-navigation-samedoc.html
Comment 14 Build Bot 2016-08-22 22:34:15 PDT
Created attachment 286675 [details]
Archive of layout-test-results from ews123 for ios-simulator-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews123  Port: ios-simulator-elcapitan-wk2  Platform: Mac OS X 10.11.5
Comment 15 Chris Dumez 2016-08-23 09:06:13 PDT
Created attachment 286717 [details]
Patch
Comment 16 youenn fablet 2016-08-23 10:15:51 PDT
Comment on attachment 286717 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=286717&action=review

rs=me.

When including a new test suite, it would be nice to update LayoutTests/imported/w3c/resources/ImportExpectations, so that next sync will also pick the changes for these tests.
In this case, it would mean removing line 142 of ImportExpectations.

> LayoutTests/TestExpectations:258
> +imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/001-1.html [ Skip ]

That is a lot of resource files to skip.
I wonder whether we should not move these "expectations" to a different file that would list the "resources" files, not actual tests.
This file could be automatically generated by the test importer script.

> LayoutTests/platform/wk2/TestExpectations:687
> +# Fails with WK2 only, it does not print the latest line.

Is it a test issue or a WTR issue?
Comment 17 Chris Dumez 2016-08-23 10:33:21 PDT
(In reply to comment #16)
> Comment on attachment 286717 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=286717&action=review
> 
> rs=me.
> 
> When including a new test suite, it would be nice to update
> LayoutTests/imported/w3c/resources/ImportExpectations, so that next sync
> will also pick the changes for these tests.
> In this case, it would mean removing line 142 of ImportExpectations.
> 
> > LayoutTests/TestExpectations:258
> > +imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/001-1.html [ Skip ]
> 
> That is a lot of resource files to skip.
> I wonder whether we should not move these "expectations" to a different file
> that would list the "resources" files, not actual tests.
> This file could be automatically generated by the test importer script.
> 
> > LayoutTests/platform/wk2/TestExpectations:687
> > +# Fails with WK2 only, it does not print the latest line.
> 
> Is it a test issue or a WTR issue?

I don't know yet. I need to investigate. The tests started working on WK1 when I started calling testRunner.setCanOpenWindows() but this did not work for WK2 for some reason.
Comment 18 Chris Dumez 2016-08-23 10:35:25 PDT
Created attachment 286729 [details]
Patch
Comment 19 WebKit Commit Bot 2016-08-23 10:49:56 PDT
Comment on attachment 286729 [details]
Patch

Rejecting attachment 286729 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-02', 'validate-changelog', '--check-oops', '--non-interactive', 286729, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

ChangeLog entry in LayoutTests/ChangeLog contains OOPS!.

Full output: http://webkit-queues.webkit.org/results/1928668
Comment 20 Chris Dumez 2016-08-23 10:52:10 PDT
Created attachment 286733 [details]
Patch
Comment 21 WebKit Commit Bot 2016-08-23 11:51:26 PDT
Comment on attachment 286733 [details]
Patch

Clearing flags on attachment: 286733

Committed r204844: <http://trac.webkit.org/changeset/204844>
Comment 22 WebKit Commit Bot 2016-08-23 11:51:34 PDT
All reviewed patches have been landed.  Closing bug.