Bug 122017 - Would like a single way to run all the fast test suites
Summary: Would like a single way to run all the fast test suites
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2013-09-27 09:39 PDT by Jessie Berlin
Modified: 2013-10-16 05:56 PDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jessie Berlin 2013-09-27 09:39:59 PDT
Right now we have

run-bindings-tests
test-webkitpy
test-webkitperl
run-api-tests
run-javascriptcore-tests 

which are all pretty fast (with the possible exception of run-javasciptcore-tests, which can take a few minutes).

There should be a way to run all of these with one command so that people do not have to mentally keep track of all the tests to run.
Comment 1 Radar WebKit Bug Importer 2013-09-27 09:40:36 PDT
<rdar://problem/15098698>
Comment 2 Darin Adler 2013-09-28 08:50:56 PDT
What I would like is that the name run-webkit-tests would be used for the single script that runs as many of the tests as is practical for a typical engineer to do on a routine basis. Tests that run fast and should all be passing. Here's how you would run it:

    % run-webkit-tests

The output would try to be as terse as possible when tests are all succeeding. Easy to read without a mix of different formats chosen for each of the different test-running scripts.

The script to run just our core regression tests from the LayoutTests directory could be given a less “canonical” name.
Comment 3 Matthew Hanson 2013-10-04 10:45:20 PDT
I would like to move the current run-webkit-tests to run-webkit-layout-tests.  My patch will introduce a new utility called run-webkit-tests that, when run without arguments, will execute the run-webkit-layout-test.  It will also introduce two new flags, the semantic meaning of which is detailed below:

    % run-webkit-tests //calls run-webkit-layout-tests

    % run-webkit-tests --include-fast-tests //calls run-webkit-layout-tests and the five tests above

    % run-webkit-tests --skip-fast-tests // calls run-webkit-layout-tests

This will be the first of three implementation stages, which are as follows:
    1. Move the current run-webkit-tests to run-webkit-layout-tests and add the --include fast-tests and --skip-fast-tests flags to run-webkit-tests
    2. Update the bots to use the --skip-fast-tests flag
    3. Change the default behavior of run-webkit-tests so that the fast tests are included by default

Any feedback on this approach is welcome.
Comment 4 Lucas Forschler 2013-10-04 11:02:45 PDT
It would be beneficial to see all places in Opensource where "run-webkit-tests" is currently used.  These will be places we must not break, and which will likely need tracking bugs to move to the new functionality.  
It sounds as though this change will be transparent, but we should double check.