Bug 187039 – Add support for testdriver WPT tests

• Status: RESOLVED
• Resolution: DUPLICATE
• Component: Tools / Tests

[Frédéric Wang (:fredw)]

Some new WPT tests upstream rely on testdriver.js to emulate user interaction (for now click and key events). Currently, it does not seem we have imported any one into WebKit repository.

For documentation, see
https://web-platform-tests.org/writing-tests/testdriver.html
https://github.com/web-platform-tests/wpt/blob/master/docs/_writing-tests/testdriver-tutorial.md
https://github.com/web-platform-tests/wpt/blob/master/resources/testdriver.js
https://github.com/web-platform-tests/wpt/blob/master/resources/testdriver-vendor.js
https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/testdriver-extra.js

I guess when running WebKit tests using WebDriver, we have to change the executor to support the new commands (see https://github.com/web-platform-tests/wpt/blob/master/docs/_writing-tests/testdriver-tutorial.md#other-browsers )

When running run-webkit-tests, I think we can just modify testdriver-vendor.js to implement click and key events. For example, on some WebKit ports this is possible via testRunner.runUIScript (see e.g. tapAt or keyDown in LayoutTests/resources/ui-helper.js).

[youenn fablet]

I also believe writing our own testdriver-vendor.js using internals/testRunner APIs is the way to go for WPT tests run by WTR.

[darshan]

Created attachment 351375 [details]
Patch

[Frédéric Wang (:fredw)]

Comment on attachment 351375 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=351375&action=review

> LayoutTests/imported/w3c/ChangeLog:6
> +

Extra white spaces.

> LayoutTests/imported/w3c/ChangeLog:7
> +        Add support for testdriver WPT tess

tests

> LayoutTests/imported/w3c/ChangeLog:15
> +

I think you should unskip testdriver tests from LayoutTests/infrastructure in order to have something to test.

> LayoutTests/imported/w3c/web-platform-tests/resources/testdriver.js:3
> +

So talking with Youenn, the proper way to implement this is actually to import the existing testdriver.js from upstream WPT (so that it implements window.test_driver) and implement testdriver-vendor.js so that it overrides window.test_driver_internal (or whatever else must be modified).

For example infrastructure/testdriver/click.html loads both files:
<script src="/resources/testdriver.js"></script>
<script src="/resources/testdriver-vendor.js"></script>

BTW, there should be some doc on upstream WPT explaining that.

[darshan]

Created attachment 351410 [details]
Patch

[youenn fablet]

Comment on attachment 351410 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=351410&action=review

> LayoutTests/TestExpectations:-858
> -webkit.org/b/187039 imported/w3c/web-platform-tests/infrastructure/testdriver [ Skip ] # testdriver not supported yet.

If we are unskipping these tests, we should be running these tests, but they do not have -expected.txt so I am unsure what happens there.

[Frédéric Wang (:fredw)]

Comment on attachment 351410 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=351410&action=review

> LayoutTests/ChangeLog:8
> +        * TestExpectations:

It would be nice to give details about the change in the changelog. e.g. IIUC you plan not to include any macOS/iOS implementation yet in this bug?

>> LayoutTests/TestExpectations:-858
>> -webkit.org/b/187039 imported/w3c/web-platform-tests/infrastructure/testdriver [ Skip ] # testdriver not supported yet.
> 
> If we are unskipping these tests, we should be running these tests, but they do not have -expected.txt so I am unsure what happens there.

Yes, please run the tests and add the expectation files.

> LayoutTests/imported/w3c/web-platform-tests/resources/testdriver-vendor.js:2
> +    window.test_driver_internal.click =  function(element, coords){

the spacing around { and = operators does not look correct in this file. Please check how it is done for testdriver.js

[darshan]

Created attachment 362391 [details]
Patch

[EWS Watchlist]

Comment on attachment 362391 [details]
Patch

Attachment 362391 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11205779

New failing tests:
imported/w3c/web-platform-tests/payment-request/payment-request-canmakepayment-method.https.html
imported/w3c/web-platform-tests/payment-request/payment-request-canmakepayment-method-protection.https.html

[EWS Watchlist]

Created attachment 362401 [details]
Archive of layout-test-results from ews105 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews105  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6

[EWS Watchlist]

Comment on attachment 362391 [details]
Patch

Attachment 362391 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/11205980

New failing tests:
imported/w3c/web-platform-tests/payment-request/payment-request-canmakepayment-method.https.html
imported/w3c/web-platform-tests/payment-request/payment-request-canmakepayment-method-protection.https.html

[EWS Watchlist]

Created attachment 362412 [details]
Archive of layout-test-results from ews124 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews124  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6

[Radar WebKit Bug Importer]

<rdar://problem/121074803>

[Sam Sneddon [:gsnedders]]

We started adding support back in bug 197011, so let's close this as a duplicate.

*** This bug has been marked as a duplicate of bug 197011 ***