Bug 9794 – Teach run-webkit-tests how to ignore tests with performance improvements

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[David Kilzer (:ddkilzer)]

There are times when specific tests are broken when you want to run the whole test suite, but don't want to have to specify every single subdirectory of LayoutTests (or worse, LayoutTests/fast) to run all but one test.  It would be nice to have an "--ignore-tests" command-line argument that allowed you ignore just one test (or one directory or one subdirectory) while running all of the rest of the tests.

As I was creating this patch, I noticed that the code that generated the list of tests to run (in @tests) was not ignoring .svn directories.  Ignoring these directories cut the time to generate the list in 1/2.  Switching from using the find(1) command to the File::Find module cut the time to generate the list by 1/2 or 1/3 again, and allowed me to implement the --ignore-tests feature much easier.

Overall, the time required to generate the full list of tests (running run-webkit-tests with no arguments) was cut from 45s-50s to about 10s.

[David Kilzer (:ddkilzer)]

Created attachment 9280 [details]
Patch v1

Changes in Patch v1:

- Added --ignore-tests command-line switch.  Takes a comma-separated list of tests and test directories to ignore (exclude) when running run-webkit-tests.  Added processIgnoreTests() subroutine to process the arguments.
- Added --help command-line switch with usage statement.
- Moved call to setConfiguration() before GetOptions() so that --debug/--release arguments are removed from @ARGV before calling GetOptions().  Keep result from call for use with build-dumprendertree later.
- Check result of GetOptions() and exit run-webkit-tests if invalid arguments are used.
- Made --singly imply --verbose so all of the "$verbose || $singly" checks could be reduced to "$verbose".
- Switched from using the find(1) command via subshell to using the File::Find Perl module.  Directory filtering and file filtering are now expressed as anonymous subroutines rather than obscure find(1) command-line arguments.  It's faster, too!
- Miscellaneous minor clean up.

[Timothy Hatcher]

Comment on attachment 9280 [details]
Patch v1

Nice cleanup.

[David Kilzer (:ddkilzer)]

Committed revision 15248.