Bug 78185 – test-webkitpy should support parallel execution of tests

• Status: RESOLVED
• Resolution: WONTFIX
• Component: Tools / Tests

[Dirk Pranke]

assuming it's relatively easy. Currently the tests run in ~53 seconds or thereabouts on a mac pro (and we aren't running some of the integration tests even then); running them in parallel should give us a near-linear speedup.

[Dirk Pranke]

Created attachment 126231 [details]
work in progress

[Dirk Pranke]

Okay, I have a preliminary version of the patch that adds roughly ~100 lines of code and shards tests by the individual test_method. I suspect once I clean things up a bit I can probably get this down to 50-75 LOC.

Running with 8 or 16 tests in parallel, total test time is about ~10 seconds. It looks like there's ~3 seconds of purely serial work to actually get all of the tests (this is expensive since we have to load all of the test files and parse them in order to shard by method). There are also several tests that take multiple seconds to complete, so we're not getting it down much further w/o speeding up individual tests. That said, we can probably get this down to ~5-6 seconds without very much work. That's a pretty good speedup from ~53 seconds.

Running tests individually, we have at least two tests that are flaky and depend on previous tests being run; that should be easy to fix.

Also, perhaps unsurprisingly, it looks like tests that actually use multiprocessing get confused and fail. I would need to look into that further.

If we shard tests by file (instead of method), the speedups are only about 2x, since run_webkit_tests_integrationtest.py is by far the long pole. Clearly we could speed up the individual tests in this file regardless of whether we want to run things in parallel or not, and that would be good.

[Dirk Pranke]

(I would say that the results are encouraging but not conclusive as to whether this is worth it).

[Dirk Pranke]

Created attachment 126445 [details]
dramatically cleaned up patch

[Dirk Pranke]

Comment on attachment 126445 [details]
dramatically cleaned up patch

here's a cleaned up patch adding parallel support, as a delta from attachment 126446 [details] on bug 77687

[Dirk Pranke]

Created attachment 128514 [details]
work in progress

[Dirk Pranke]

Created attachment 128842 [details]
Work in progress (includes rollup of work in 77687)

[Dirk Pranke]

closing ... I will file new bugs as I have patches for review. The rollup in bug 76022 contains the real code, so we're not losing anything here.