Bug 180915 – Allow run-api-tests to start/stop layout tests servers

• Status: NEW
• Component: Tools / Tests

[youenn fablet]

This is handy to write API tests that require HTTP/HTTPS/WebSocket behavior.

[youenn fablet]

Created attachment 329603 [details]
Patch

[youenn fablet]

Created attachment 329605 [details]
Patch

[Alexey Proskuryakov]

Comment on attachment 329605 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=329605&action=review

> Tools/ChangeLog:3
> +        Allow run-api-tests to start/stop layout tests servers

This is something that we explicitly refused to do in the past, as it made more sense to add functionality to WKTR anyway. Also, API tests are super slow, as they are not running in parallel.

> Tools/ChangeLog:9
> +        Added a python wrapper around it to control start/stop of HTTP/WS servers.

Jonathan is working on converting run-api-tests to Python now.

[Jonathan Bedard]

A few comments on this:

1) I've been working on a Python replacement for run-api-tests so that we can manager Simulators better and run tests as multiprocess (might address Alexey's concern about .  This change may be worth postponing until we are running in Python, rather than requiring this wrapper.

2) It seems wrong to have API tests assets as a subdirectory in LayoutTests.

[youenn fablet]

(In reply to Jonathan Bedard from comment #4)
> A few comments on this:
> 
> 1) I've been working on a Python replacement for run-api-tests so that we
> can manager Simulators better and run tests as multiprocess (might address
> Alexey's concern about .  This change may be worth postponing until we are
> running in Python, rather than requiring this wrapper.

Great to see it happening!
I am fine waiting for this, that will remove the FIXME inside this patch

> 2) It seems wrong to have API tests assets as a subdirectory in LayoutTests.

It should be fairly easy to add additional root paths like we do now for http/wpt, so that is fine.
Main motivation is to be able to do api tests on features that require http/https.
Allowing to use existing infrastructure resources such as existing LayoutTests JS utilities seems an additional bonus. This might blur the line between API tests specific resources and layout tests resources.