Bug 60350 - run_webkit_tests_unittest.MainTest is very slow
Summary: run_webkit_tests_unittest.MainTest is very slow
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-06 00:38 PDT by Adam Barth
Modified: 2012-03-12 18:24 PDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Adam Barth 2011-05-06 00:38:16 PDT
run_webkit_tests_unittest.MainTest accounts for 57% of the total running time of test-webkitpy even though it only accounts for 5.4% of the tests:

$ ./Tools/Scripts/test-webkitpy  webkitpy.layout_tests.run_webkit_tests_unittest.MainTest
Ran 56 tests in 14.183s

$ ./Tools/Scripts/test-webkitpy  webkitpy.layout_tests.run_webkit_tests_unittest.MainTest
Ran 1036 tests in 24.807s

I don't know why these tests are so slow, but there's something wrong with them.
Comment 1 Adam Barth 2011-05-06 09:42:52 PDT
Sorry:

$ ./Tools/Scripts/test-webkitpy
Ran 1036 tests in 24.807s
Comment 2 Dirk Pranke 2011-05-06 12:19:57 PDT
This bug does not depend on 60354 or vice versa. I am clearing the field.
Comment 3 Dirk Pranke 2011-05-06 12:59:10 PDT
There is a time.sleep() in layout_tests/manager.py that is there to avoid what appears to be some sort of race condition when we spawn off a bunch of DumpRenderTrees when running the real tests. This accounts for ~7 seconds, and we could push this down to a port-specific hook to be mocked out when running the unittests.

That's the only really obvious thing in the profiling stats.

I think the combination of moving to the new manager/worker model and the fact that we've been gradually accumulating more tests that get executed for each combination of flags to NRWT accounts for the rest of the slowness. There's simply a lot of different test paths to execute, and these tests do a lot of work to cover them all.
Comment 4 Adam Barth 2011-05-06 13:06:49 PDT
Another perspective is that this test suite is taking a poor approach to testing this code and that this code is not well-designed for testability.  Unfortunately, I don't think this is the right moment to refactor NRWT for testability because actually turning it on for the rest of the project is a higher priority.
Comment 5 Dirk Pranke 2012-03-12 18:24:19 PDT
it's much faster now :). Closing.