Bug 60350 – run_webkit_tests_unittest.MainTest is very slow

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Adam Barth]

run_webkit_tests_unittest.MainTest accounts for 57% of the total running time of test-webkitpy even though it only accounts for 5.4% of the tests:

$ ./Tools/Scripts/test-webkitpy  webkitpy.layout_tests.run_webkit_tests_unittest.MainTest
Ran 56 tests in 14.183s

$ ./Tools/Scripts/test-webkitpy  webkitpy.layout_tests.run_webkit_tests_unittest.MainTest
Ran 1036 tests in 24.807s

I don't know why these tests are so slow, but there's something wrong with them.

[Adam Barth]

Sorry:

$ ./Tools/Scripts/test-webkitpy
Ran 1036 tests in 24.807s

[Dirk Pranke]

This bug does not depend on 60354 or vice versa. I am clearing the field.

[Dirk Pranke]

There is a time.sleep() in layout_tests/manager.py that is there to avoid what appears to be some sort of race condition when we spawn off a bunch of DumpRenderTrees when running the real tests. This accounts for ~7 seconds, and we could push this down to a port-specific hook to be mocked out when running the unittests.

That's the only really obvious thing in the profiling stats.

I think the combination of moving to the new manager/worker model and the fact that we've been gradually accumulating more tests that get executed for each combination of flags to NRWT accounts for the rest of the slowness. There's simply a lot of different test paths to execute, and these tests do a lot of work to cover them all.

[Adam Barth]

Another perspective is that this test suite is taking a poor approach to testing this code and that this code is not well-designed for testability.  Unfortunately, I don't think this is the right moment to refactor NRWT for testability because actually turning it on for the rest of the project is a higher priority.

[Dirk Pranke]

it's much faster now :). Closing.