Bug 171578
| Summary: | run-javascriptcore-tests misses tests JSTests/asyncFunctionTests.yaml JSTests/heapProfiler.yaml and JSTests/test262.yaml | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Carlos Alberto Lopez Perez <clopez> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | NEW | ||
| Severity: | Normal | CC: | aakash_jain, ap, fpizlo, guijemont, joepeck, keith_miller, lforschler, ossy, ysuzuki |
| Priority: | P2 | ||
| Version: | WebKit Nightly Build | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| See Also: | https://bugs.webkit.org/show_bug.cgi?id=170523 | ||
Carlos Alberto Lopez Perez
The script run-javascript-tests has a list of tests that it should run:
https://trac.webkit.org/browser/trunk/Tools/Scripts/run-javascriptcore-tests?rev=215108#L268
On this list, currently there are 3 tests missing:
JSTests/asyncFunctionTests.yaml
JSTests/heapProfiler.yaml
JSTests/test262.yaml
I have checked this tests, and all 3 work fine. So this seems an overlook.
In bug 170523, new bots were added to just run the JSTests/test262.yaml alone for the Mac port instead of adding JSTests/test262.yaml to the list of tests that the JSC bots run with the script run-javascript-tests.
The other two tests (JSTests/asyncFunctionTests.yaml and JSTests/heapProfiler.yaml) are currently not tested by any bot.
In order to fix this, my proposal is the following:
1) Remove the list of tests to run from run-javascriptcore-tests
2) Add instead a list of tests to be skipped
3) Make the script auto-discover current tests by looking at *.yaml files on the test directories
(shell example: find JSTests/ LayoutTests/ PerformanceTests/ -type f -name \*.yaml )
So the tool will be automatically running all the discovered tests except the ones skipped.
This will ensure we don't hit this situation in the future of JavaScriptCore tests that are not ran by any bot.
This is similar to how we run the layout tests (tests to run are auto-discovered from the filesystem).
The skipped list should contain JSTests/test262.yaml (this test is currently run in another step on the Mac bots).
- The skipped list could be empty if the Mac port maintainers think that it would be a good idea to run JSTests/test262.yaml as part of run-javascript-tests, and migrate the curent test262 bots to general JSC testers (A build slave can have several workers). In any case this should be done in another bug.
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Alexey Proskuryakov
We do want tests to run fast, so adding more tests suites to the same bots should not be the norm.
Joseph Pecoraro
(In reply to Alexey Proskuryakov from comment #1)
> We do want tests to run fast, so adding more tests suites to the same bots
> should not be the norm.
I don't think that applies here, these are very small suites.
> JSTests/asyncFunctionTests.yaml
On my machine this takes 4 seconds to run.
> JSTests/heapProfiler.yaml
On my machine this takes 2 seconds to run.