Right now we have
which are all pretty fast (with the possible exception of run-javasciptcore-tests, which can take a few minutes).
There should be a way to run all of these with one command so that people do not have to mentally keep track of all the tests to run.
What I would like is that the name run-webkit-tests would be used for the single script that runs as many of the tests as is practical for a typical engineer to do on a routine basis. Tests that run fast and should all be passing. Here's how you would run it:
The output would try to be as terse as possible when tests are all succeeding. Easy to read without a mix of different formats chosen for each of the different test-running scripts.
The script to run just our core regression tests from the LayoutTests directory could be given a less “canonical” name.
I would like to move the current run-webkit-tests to run-webkit-layout-tests. My patch will introduce a new utility called run-webkit-tests that, when run without arguments, will execute the run-webkit-layout-test. It will also introduce two new flags, the semantic meaning of which is detailed below:
% run-webkit-tests //calls run-webkit-layout-tests
% run-webkit-tests --include-fast-tests //calls run-webkit-layout-tests and the five tests above
% run-webkit-tests --skip-fast-tests // calls run-webkit-layout-tests
This will be the first of three implementation stages, which are as follows:
1. Move the current run-webkit-tests to run-webkit-layout-tests and add the --include fast-tests and --skip-fast-tests flags to run-webkit-tests
2. Update the bots to use the --skip-fast-tests flag
3. Change the default behavior of run-webkit-tests so that the fast tests are included by default
Any feedback on this approach is welcome.
It would be beneficial to see all places in Opensource where "run-webkit-tests" is currently used. These will be places we must not break, and which will likely need tracking bugs to move to the new functionality.
It sounds as though this change will be transparent, but we should double check.