Bug 178589 – Rebase WPT tests up to a1c0107

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[youenn fablet]

Rebase WPT tests up to a1c0107

[youenn fablet]

Created attachment 324407 [details]
Patch

[Build Bot]

Attachment 324407 [details] did not pass style-queue:


ERROR: /Volumes/Data/StyleQueue/WebKit/LayoutTests/platform/mac/TestExpectations:1784:  Path does not exist.  [test/expectations] [5]
Total errors found: 1 in 363 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[Build Bot]

Comment on attachment 324407 [details]
Patch

Attachment 324407 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/4934220

New failing tests:
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/window-name-after-cross-origin-main-frame-navigation.sub.html

[Build Bot]

Created attachment 324414 [details]
Archive of layout-test-results from ews100 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-elcapitan  Platform: Mac OS X 10.11.6

[Build Bot]

Comment on attachment 324407 [details]
Patch

Attachment 324407 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/4934096

New failing tests:
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/window-name-after-cross-origin-main-frame-navigation.sub.html
imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-ontrack.https.html

[Build Bot]

Created attachment 324415 [details]
Archive of layout-test-results from ews125 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews125  Port: ios-simulator-wk2  Platform: Mac OS X 10.12.6

[Build Bot]

Comment on attachment 324407 [details]
Patch

Attachment 324407 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/4934322

New failing tests:
imported/w3c/web-platform-tests/html/browsers/browsing-the-web/history-traversal/window-name-after-cross-origin-main-frame-navigation.sub.html

[Build Bot]

Created attachment 324416 [details]
Archive of layout-test-results from ews115 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews115  Port: mac-elcapitan  Platform: Mac OS X 10.11.6

[youenn fablet]

Created attachment 324427 [details]
Patch

[Build Bot]

Attachment 324427 [details] did not pass style-queue:


ERROR: /Volumes/Data/StyleQueue/WebKit/LayoutTests/platform/mac/TestExpectations:1784:  Path does not exist.  [test/expectations] [5]
Total errors found: 1 in 363 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[youenn fablet]

Created attachment 324441 [details]
Patch

[Build Bot]

Attachment 324441 [details] did not pass style-queue:


ERROR: /Volumes/Data/StyleQueue/WebKit/LayoutTests/platform/mac/TestExpectations:1784:  Path does not exist.  [test/expectations] [5]
Total errors found: 1 in 363 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[Build Bot]

Comment on attachment 324441 [details]
Patch

Attachment 324441 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/4937064

New failing tests:
imported/w3c/web-platform-tests/webrtc/RTCPeerConnection-ontrack.https.html

[Build Bot]

Created attachment 324450 [details]
Archive of layout-test-results from ews126 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-wk2  Platform: Mac OS X 10.12.6

[WebKit Commit Bot]

Comment on attachment 324441 [details]
Patch

Clearing flags on attachment: 324441

Committed r224156: <https://trac.webkit.org/changeset/224156>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Chris Dumez]

(In reply to WebKit Commit Bot from comment #16)
> All reviewed patches have been landed.  Closing bug.

Some service worker tests have wrong expectations or no expected results file:
https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r224158%20(5354)/results.html

[youenn fablet]

(In reply to Chris Dumez from comment #17)
> (In reply to WebKit Commit Bot from comment #16)
> > All reviewed patches have been landed.  Closing bug.
> 
> Some service worker tests have wrong expectations or no expected results
> file:
> https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/
> r224158%20(5354)/results.html

Ah, probably need to skip these since the folder is no longer skipped as a whole.
I'll update test expectations.

[youenn fablet]

Reopening to attach new patch.

[youenn fablet]

Created attachment 325296 [details]
Patch

[Chris Dumez]

Comment on attachment 325296 [details]
Patch

Only one of them is timing out. Why skip all of them? I am trying to unskip as many tests as possible. This is counter productive.

[youenn fablet]

(In reply to Chris Dumez from comment #21)
> Comment on attachment 325296 [details]
> Patch
> 
> Only one of them is timing out. Why skip all of them? I am trying to unskip
> as many tests as possible. This is counter productive.

Looking at the results, these tests are not yet able to produce something really meaningful. Until we have more service worker infrastructure, running them or not running them does not change a lot, all of these tests basically cover the same missing features.

Unskipping them once we have more infra is surely our first priority.
I am guessing that we will be able to run some timing out tests at about the same time.

[Chris Dumez]

(In reply to youenn fablet from comment #22)
> (In reply to Chris Dumez from comment #21)
> > Comment on attachment 325296 [details]
> > Patch
> > 
> > Only one of them is timing out. Why skip all of them? I am trying to unskip
> > as many tests as possible. This is counter productive.
> 
> Looking at the results, these tests are not yet able to produce something
> really meaningful. Until we have more service worker infrastructure, running
> them or not running them does not change a lot, all of these tests basically
> cover the same missing features.
> 
> Unskipping them once we have more infra is surely our first priority.
> I am guessing that we will be able to run some timing out tests at about the
> same time.

It is much better if we need to rebaseline tests once we implement something rather than looking for skipped tests to unskip :(((

[Chris Dumez]

(In reply to Chris Dumez from comment #23)
> (In reply to youenn fablet from comment #22)
> > (In reply to Chris Dumez from comment #21)
> > > Comment on attachment 325296 [details]
> > > Patch
> > > 
> > > Only one of them is timing out. Why skip all of them? I am trying to unskip
> > > as many tests as possible. This is counter productive.
> > 
> > Looking at the results, these tests are not yet able to produce something
> > really meaningful. Until we have more service worker infrastructure, running
> > them or not running them does not change a lot, all of these tests basically
> > cover the same missing features.
> > 
> > Unskipping them once we have more infra is surely our first priority.
> > I am guessing that we will be able to run some timing out tests at about the
> > same time.
> 
> It is much better if we need to rebaseline tests once we implement something
> rather than looking for skipped tests to unskip :(((

Also please see progress via https://bugs.webkit.org/show_bug.cgi?id=178985

[youenn fablet]

Created attachment 325298 [details]
Patch

[youenn fablet]

I'll put it again as cq+ so that EWS/other bots are fine.
If you have time to improve the patch by refreshing the expectations, please cq- and update the patch. Otherwise, I'll do it tomorrow.

[Chris Dumez]

Comment on attachment 325298 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=325298&action=review

> LayoutTests/ChangeLog:8
> +        * TestExpectations: Skipping som eservice worker tests.

Typo: som eservice

[Chris Dumez]

(In reply to Chris Dumez from comment #27)
> Comment on attachment 325298 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=325298&action=review
> 
> > LayoutTests/ChangeLog:8
> > +        * TestExpectations: Skipping som eservice worker tests.
> 
> Typo: som eservice

Also, as mentioned earlier, I have a task to get SW WPT tests running so please help me by not unnecessarily skipping tests that do not time out.
It is not trivial to figure out which tests can be unskipped after implementing something. However, rebaselining is.

[Chris Dumez]

(In reply to Chris Dumez from comment #28)
> (In reply to Chris Dumez from comment #27)
> > Comment on attachment 325298 [details]
> > Patch
> > 
> > View in context:
> > https://bugs.webkit.org/attachment.cgi?id=325298&action=review
> > 
> > > LayoutTests/ChangeLog:8
> > > +        * TestExpectations: Skipping som eservice worker tests.
> > 
> > Typo: som eservice
> 
> Also, as mentioned earlier, I have a task to get SW WPT tests running so
> please help me by not unnecessarily skipping tests that do not time out.
> It is not trivial to figure out which tests can be unskipped after
> implementing something. However, rebaselining is.

I also would not mind landing baselines if you’d like.

[Chris Dumez]

(In reply to Chris Dumez from comment #29)
> (In reply to Chris Dumez from comment #28)
> > (In reply to Chris Dumez from comment #27)
> > > Comment on attachment 325298 [details]
> > > Patch
> > > 
> > > View in context:
> > > https://bugs.webkit.org/attachment.cgi?id=325298&action=review
> > > 
> > > > LayoutTests/ChangeLog:8
> > > > +        * TestExpectations: Skipping som eservice worker tests.
> > > 
> > > Typo: som eservice
> > 
> > Also, as mentioned earlier, I have a task to get SW WPT tests running so
> > please help me by not unnecessarily skipping tests that do not time out.
> > It is not trivial to figure out which tests can be unskipped after
> > implementing something. However, rebaselining is.
> 
> I also would not mind landing baselines if you’d like.

<https://trac.webkit.org/changeset/224166>

[Radar WebKit Bug Importer]

<rdar://problem/35569084>