RESOLVED FIXED 149645
Update web-platform-tests tools to the latest revision
https://bugs.webkit.org/show_bug.cgi?id=149645
Summary Update web-platform-tests tools to the latest revision
youenn fablet
Reported 2015-09-29 13:43:34 PDT
web-platform-tests tools changed from as a submodule of submodules. This should be reflected in WebKit test infrastructure.
Attachments
Patch (520.19 KB, patch)
2015-09-29 13:54 PDT, youenn fablet
no flags
Archive of layout-test-results from ews100 for mac-mavericks (995.22 KB, application/zip)
2015-09-29 14:11 PDT, Build Bot
no flags
Archive of layout-test-results from ews104 for mac-mavericks-wk2 (783.21 KB, application/zip)
2015-09-29 14:24 PDT, Build Bot
no flags
Rebasing failing test expectations (579.43 KB, patch)
2015-09-30 02:20 PDT, youenn fablet
no flags
Patch (521.40 KB, patch)
2015-10-14 03:15 PDT, youenn fablet
no flags
Archive of layout-test-results from ews103 for mac-mavericks (629.79 KB, application/zip)
2015-10-14 03:49 PDT, Build Bot
no flags
Archive of layout-test-results from ews106 for mac-mavericks-wk2 (665.26 KB, application/zip)
2015-10-14 03:53 PDT, Build Bot
no flags
Cleaning patch (520.70 KB, patch)
2015-10-14 04:14 PDT, youenn fablet
no flags
Patch for landing (519.95 KB, patch)
2015-10-14 04:44 PDT, youenn fablet
no flags
youenn fablet
Comment 1 2015-09-29 13:54:46 PDT
Build Bot
Comment 2 2015-09-29 14:11:51 PDT
Comment on attachment 262103 [details] Patch Attachment 262103 [details] did not pass mac-ews (mac): Output: http://webkit-queues.webkit.org/results/224663 New failing tests: imported/w3c/web-platform-tests/html/dom/interfaces.html imported/w3c/web-platform-tests/dom/interfaces.html
Build Bot
Comment 3 2015-09-29 14:11:54 PDT
Created attachment 262105 [details] Archive of layout-test-results from ews100 for mac-mavericks The attached test failures were seen while running run-webkit-tests on the mac-ews. Bot: ews100 Port: mac-mavericks Platform: Mac OS X 10.9.5
Build Bot
Comment 4 2015-09-29 14:24:19 PDT
Comment on attachment 262103 [details] Patch Attachment 262103 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/224687 New failing tests: imported/w3c/web-platform-tests/html/dom/interfaces.html imported/w3c/web-platform-tests/dom/interfaces.html
Build Bot
Comment 5 2015-09-29 14:24:22 PDT
Created attachment 262108 [details] Archive of layout-test-results from ews104 for mac-mavericks-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews104 Port: mac-mavericks-wk2 Platform: Mac OS X 10.9.5
youenn fablet
Comment 6 2015-09-30 02:20:45 PDT
Created attachment 262148 [details] Rebasing failing test expectations
Chris Dumez
Comment 7 2015-09-30 09:05:35 PDT
Comment on attachment 262148 [details] Rebasing failing test expectations View in context: https://bugs.webkit.org/attachment.cgi?id=262148&action=review > LayoutTests/TestExpectations:245 > +webkit.org/b/149656 imported/w3c/web-platform-tests/html/dom/interfaces.html [ Timeout ] This seems really unfortunate. I wish this patch did not need to skip this test. We are working on improving our DOM / HTML compliance and rely on this test a lot.
Chris Dumez
Comment 8 2015-09-30 09:08:53 PDT
Comment on attachment 262148 [details] Rebasing failing test expectations View in context: https://bugs.webkit.org/attachment.cgi?id=262148&action=review > LayoutTests/imported/w3c/web-platform-tests/dom/interfaces-expected.txt:958 > +FAIL Element interface: operation before([object Object],[object Object]) assert_equals: property has wrong .length expected 0 but got 1 We should file a bug about this. Basically our length is 1 instead of 0 for operations taking a variadic arguments. This should be fixed.
youenn fablet
Comment 9 2015-10-01 00:19:14 PDT
(In reply to comment #7) > Comment on attachment 262148 [details] > Rebasing failing test expectations > > View in context: > https://bugs.webkit.org/attachment.cgi?id=262148&action=review > > > LayoutTests/TestExpectations:245 > > +webkit.org/b/149656 imported/w3c/web-platform-tests/html/dom/interfaces.html [ Timeout ] > > This seems really unfortunate. I wish this patch did not need to skip this > test. We are working on improving our DOM / HTML compliance and rely on this > test a lot. I'll postpone landing this patch until finding a solution for this one then.
Chris Dumez
Comment 10 2015-10-01 09:11:49 PDT
(In reply to comment #9) > (In reply to comment #7) > > Comment on attachment 262148 [details] > > Rebasing failing test expectations > > > > View in context: > > https://bugs.webkit.org/attachment.cgi?id=262148&action=review > > > > > LayoutTests/TestExpectations:245 > > > +webkit.org/b/149656 imported/w3c/web-platform-tests/html/dom/interfaces.html [ Timeout ] > > > > This seems really unfortunate. I wish this patch did not need to skip this > > test. We are working on improving our DOM / HTML compliance and rely on this > > test a lot. > > I'll postpone landing this patch until finding a solution for this one then. FYI, I think I had a similar problem when initially importing this test (would always timeout) and I worked around the issue by commenting out one of the interfaces being tested (Window iirc). Chances are you can do the same here.
Chris Dumez
Comment 11 2015-10-01 11:13:07 PDT
(In reply to comment #8) > Comment on attachment 262148 [details] > Rebasing failing test expectations > > View in context: > https://bugs.webkit.org/attachment.cgi?id=262148&action=review > > > LayoutTests/imported/w3c/web-platform-tests/dom/interfaces-expected.txt:958 > > +FAIL Element interface: operation before([object Object],[object Object]) assert_equals: property has wrong .length expected 0 but got 1 > > We should file a bug about this. Basically our length is 1 instead of 0 for > operations taking a variadic arguments. This should be fixed. I filed <https://bugs.webkit.org/show_bug.cgi?id=149714> and I am working on fixing the bug.
youenn fablet
Comment 12 2015-10-02 03:17:51 PDT
(In reply to comment #11) > (In reply to comment #8) > > Comment on attachment 262148 [details] > > Rebasing failing test expectations > > > > View in context: > > https://bugs.webkit.org/attachment.cgi?id=262148&action=review > > > I filed <https://bugs.webkit.org/show_bug.cgi?id=149714> and I am working on > fixing the bug. Thanks for tackling this issue. That said, I think that we will not be able to always resync and fix bugs at the same time. This calls to two tools IMHO: - Ensuring no regression: we have it with expected files - Ensuring conformance: we do not have any really Doing a quick grep, we have 185 tests out of 936 web-platform-tests that have "FAIL" in their expected files. There is a risk that we forget to fix them. Having a tool to monitor expected files might be useful.
youenn fablet
Comment 13 2015-10-02 03:25:47 PDT
(In reply to comment #10) > (In reply to comment #9) > > (In reply to comment #7) > > > Comment on attachment 262148 [details] > > > Rebasing failing test expectations > > > > > > View in context: > > > https://bugs.webkit.org/attachment.cgi?id=262148&action=review > > > > > > > LayoutTests/TestExpectations:245 > > > > +webkit.org/b/149656 imported/w3c/web-platform-tests/html/dom/interfaces.html [ Timeout ] > > > > > > This seems really unfortunate. I wish this patch did not need to skip this > > > test. We are working on improving our DOM / HTML compliance and rely on this > > > test a lot. > > > > I'll postpone landing this patch until finding a solution for this one then. > > FYI, I think I had a similar problem when initially importing this test > (would always timeout) and I worked around the issue by commenting out one > of the interfaces being tested (Window iirc). Chances are you can do the > same here. It might be good to keep track of these manual changes since they may get lost when resyncing. Maybe by adding a comment so that it appears on the diff when resyncing? Or uploading the changes to wpt directly?
youenn fablet
Comment 14 2015-10-14 03:13:37 PDT
> > FYI, I think I had a similar problem when initially importing this test > > (would always timeout) and I worked around the issue by commenting out one > > of the interfaces being tested (Window iirc). Chances are you can do the > > same here. It seems commenting out showModalDialog method from Window interface is fixing the timeout. Any idea why it is doing so and how to fix that? Not upgrading the testharness.js submodule revision also fixes that issue. This upgrade should be done at the same time we update WebKit testharness.js file (currently we are using WebKit testharness.js file for WPT tests as well).
youenn fablet
Comment 15 2015-10-14 03:15:35 PDT
youenn fablet
Comment 16 2015-10-14 03:17:13 PDT
(In reply to comment #15) > Created attachment 263069 [details] > Patch Compared to r+ patch, the following changes have been made: - Postpone testharness module upgrade to a later patch. - Remove TestExpectations changes since they should not be needed anymore.
Build Bot
Comment 17 2015-10-14 03:49:03 PDT
Comment on attachment 263069 [details] Patch Attachment 263069 [details] did not pass mac-ews (mac): Output: http://webkit-queues.webkit.org/results/285106 New failing tests: streams/readable-stream-controller-error.html
Build Bot
Comment 18 2015-10-14 03:49:09 PDT
Created attachment 263071 [details] Archive of layout-test-results from ews103 for mac-mavericks The attached test failures were seen while running run-webkit-tests on the mac-ews. Bot: ews103 Port: mac-mavericks Platform: Mac OS X 10.9.5
Build Bot
Comment 19 2015-10-14 03:53:49 PDT
Comment on attachment 263069 [details] Patch Attachment 263069 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/285110 New failing tests: streams/readable-stream-controller-error.html
Build Bot
Comment 20 2015-10-14 03:53:53 PDT
Created attachment 263072 [details] Archive of layout-test-results from ews106 for mac-mavericks-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews106 Port: mac-mavericks-wk2 Platform: Mac OS X 10.9.5
youenn fablet
Comment 21 2015-10-14 04:14:21 PDT
Created attachment 263073 [details] Cleaning patch
youenn fablet
Comment 22 2015-10-14 04:44:08 PDT
Created attachment 263074 [details] Patch for landing
WebKit Commit Bot
Comment 23 2015-10-14 05:37:52 PDT
Comment on attachment 263074 [details] Patch for landing Clearing flags on attachment: 263074 Committed r191043: <http://trac.webkit.org/changeset/191043>
WebKit Commit Bot
Comment 24 2015-10-14 05:37:57 PDT
All reviewed patches have been landed. Closing bug.
Note You need to log in before you can comment on or make changes to this bug.