| Summary: | Update web-platform-tests tools to the latest revision | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | youenn fablet <youennf> |
| Component: | Tools / Tests | Assignee: | youenn fablet <youennf> |
| Status: | RESOLVED FIXED | ||
| Severity: | Normal | CC: | benjamin, buildbot, calvaris, cdumez, commit-queue, glenn, lforschler, rniwa |
| Priority: | P2 | ||
| Version: | WebKit Nightly Build | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| See Also: | https://bugs.webkit.org/show_bug.cgi?id=149714 | ||
| Attachments: | |||
|
Description
youenn fablet
2015-09-29 13:43:34 PDT
Created attachment 262103 [details]
Patch
Comment on attachment 262103 [details] Patch Attachment 262103 [details] did not pass mac-ews (mac): Output: http://webkit-queues.webkit.org/results/224663 New failing tests: imported/w3c/web-platform-tests/html/dom/interfaces.html imported/w3c/web-platform-tests/dom/interfaces.html Created attachment 262105 [details]
Archive of layout-test-results from ews100 for mac-mavericks
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100 Port: mac-mavericks Platform: Mac OS X 10.9.5
Comment on attachment 262103 [details] Patch Attachment 262103 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/224687 New failing tests: imported/w3c/web-platform-tests/html/dom/interfaces.html imported/w3c/web-platform-tests/dom/interfaces.html Created attachment 262108 [details]
Archive of layout-test-results from ews104 for mac-mavericks-wk2
The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104 Port: mac-mavericks-wk2 Platform: Mac OS X 10.9.5
Created attachment 262148 [details]
Rebasing failing test expectations
Comment on attachment 262148 [details] Rebasing failing test expectations View in context: https://bugs.webkit.org/attachment.cgi?id=262148&action=review > LayoutTests/TestExpectations:245 > +webkit.org/b/149656 imported/w3c/web-platform-tests/html/dom/interfaces.html [ Timeout ] This seems really unfortunate. I wish this patch did not need to skip this test. We are working on improving our DOM / HTML compliance and rely on this test a lot. Comment on attachment 262148 [details] Rebasing failing test expectations View in context: https://bugs.webkit.org/attachment.cgi?id=262148&action=review > LayoutTests/imported/w3c/web-platform-tests/dom/interfaces-expected.txt:958 > +FAIL Element interface: operation before([object Object],[object Object]) assert_equals: property has wrong .length expected 0 but got 1 We should file a bug about this. Basically our length is 1 instead of 0 for operations taking a variadic arguments. This should be fixed. (In reply to comment #7) > Comment on attachment 262148 [details] > Rebasing failing test expectations > > View in context: > https://bugs.webkit.org/attachment.cgi?id=262148&action=review > > > LayoutTests/TestExpectations:245 > > +webkit.org/b/149656 imported/w3c/web-platform-tests/html/dom/interfaces.html [ Timeout ] > > This seems really unfortunate. I wish this patch did not need to skip this > test. We are working on improving our DOM / HTML compliance and rely on this > test a lot. I'll postpone landing this patch until finding a solution for this one then. (In reply to comment #9) > (In reply to comment #7) > > Comment on attachment 262148 [details] > > Rebasing failing test expectations > > > > View in context: > > https://bugs.webkit.org/attachment.cgi?id=262148&action=review > > > > > LayoutTests/TestExpectations:245 > > > +webkit.org/b/149656 imported/w3c/web-platform-tests/html/dom/interfaces.html [ Timeout ] > > > > This seems really unfortunate. I wish this patch did not need to skip this > > test. We are working on improving our DOM / HTML compliance and rely on this > > test a lot. > > I'll postpone landing this patch until finding a solution for this one then. FYI, I think I had a similar problem when initially importing this test (would always timeout) and I worked around the issue by commenting out one of the interfaces being tested (Window iirc). Chances are you can do the same here. (In reply to comment #8) > Comment on attachment 262148 [details] > Rebasing failing test expectations > > View in context: > https://bugs.webkit.org/attachment.cgi?id=262148&action=review > > > LayoutTests/imported/w3c/web-platform-tests/dom/interfaces-expected.txt:958 > > +FAIL Element interface: operation before([object Object],[object Object]) assert_equals: property has wrong .length expected 0 but got 1 > > We should file a bug about this. Basically our length is 1 instead of 0 for > operations taking a variadic arguments. This should be fixed. I filed <https://bugs.webkit.org/show_bug.cgi?id=149714> and I am working on fixing the bug. (In reply to comment #11) > (In reply to comment #8) > > Comment on attachment 262148 [details] > > Rebasing failing test expectations > > > > View in context: > > https://bugs.webkit.org/attachment.cgi?id=262148&action=review > > > I filed <https://bugs.webkit.org/show_bug.cgi?id=149714> and I am working on > fixing the bug. Thanks for tackling this issue. That said, I think that we will not be able to always resync and fix bugs at the same time. This calls to two tools IMHO: - Ensuring no regression: we have it with expected files - Ensuring conformance: we do not have any really Doing a quick grep, we have 185 tests out of 936 web-platform-tests that have "FAIL" in their expected files. There is a risk that we forget to fix them. Having a tool to monitor expected files might be useful. (In reply to comment #10) > (In reply to comment #9) > > (In reply to comment #7) > > > Comment on attachment 262148 [details] > > > Rebasing failing test expectations > > > > > > View in context: > > > https://bugs.webkit.org/attachment.cgi?id=262148&action=review > > > > > > > LayoutTests/TestExpectations:245 > > > > +webkit.org/b/149656 imported/w3c/web-platform-tests/html/dom/interfaces.html [ Timeout ] > > > > > > This seems really unfortunate. I wish this patch did not need to skip this > > > test. We are working on improving our DOM / HTML compliance and rely on this > > > test a lot. > > > > I'll postpone landing this patch until finding a solution for this one then. > > FYI, I think I had a similar problem when initially importing this test > (would always timeout) and I worked around the issue by commenting out one > of the interfaces being tested (Window iirc). Chances are you can do the > same here. It might be good to keep track of these manual changes since they may get lost when resyncing. Maybe by adding a comment so that it appears on the diff when resyncing? Or uploading the changes to wpt directly? > > FYI, I think I had a similar problem when initially importing this test
> > (would always timeout) and I worked around the issue by commenting out one
> > of the interfaces being tested (Window iirc). Chances are you can do the
> > same here.
It seems commenting out showModalDialog method from Window interface is fixing the timeout. Any idea why it is doing so and how to fix that?
Not upgrading the testharness.js submodule revision also fixes that issue.
This upgrade should be done at the same time we update WebKit testharness.js file (currently we are using WebKit testharness.js file for WPT tests as well).
Created attachment 263069 [details]
Patch
(In reply to comment #15) > Created attachment 263069 [details] > Patch Compared to r+ patch, the following changes have been made: - Postpone testharness module upgrade to a later patch. - Remove TestExpectations changes since they should not be needed anymore. Comment on attachment 263069 [details] Patch Attachment 263069 [details] did not pass mac-ews (mac): Output: http://webkit-queues.webkit.org/results/285106 New failing tests: streams/readable-stream-controller-error.html Created attachment 263071 [details]
Archive of layout-test-results from ews103 for mac-mavericks
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103 Port: mac-mavericks Platform: Mac OS X 10.9.5
Comment on attachment 263069 [details] Patch Attachment 263069 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/285110 New failing tests: streams/readable-stream-controller-error.html Created attachment 263072 [details]
Archive of layout-test-results from ews106 for mac-mavericks-wk2
The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews106 Port: mac-mavericks-wk2 Platform: Mac OS X 10.9.5
Created attachment 263073 [details]
Cleaning patch
Created attachment 263074 [details]
Patch for landing
Comment on attachment 263074 [details] Patch for landing Clearing flags on attachment: 263074 Committed r191043: <http://trac.webkit.org/changeset/191043> All reviewed patches have been landed. Closing bug. |