| Summary: | test-webkitpy should support parallel execution of tests | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Dirk Pranke <dpranke> | ||||||||||
| Component: | Tools / Tests | Assignee: | Dirk Pranke <dpranke> | ||||||||||
| Status: | RESOLVED WONTFIX | ||||||||||||
| Severity: | Normal | CC: | abarth, eric, levin, ojan, rniwa, tony | ||||||||||
| Priority: | P2 | ||||||||||||
| Version: | 528+ (Nightly build) | ||||||||||||
| Hardware: | Unspecified | ||||||||||||
| OS: | Unspecified | ||||||||||||
| Bug Depends on: | 76765, 77807, 78171, 78181, 78187 | ||||||||||||
| Bug Blocks: | 76022 | ||||||||||||
| Attachments: |
|
||||||||||||
|
Description
Dirk Pranke
2012-02-08 17:06:14 PST
Created attachment 126231 [details]
work in progress
Okay, I have a preliminary version of the patch that adds roughly ~100 lines of code and shards tests by the individual test_method. I suspect once I clean things up a bit I can probably get this down to 50-75 LOC. Running with 8 or 16 tests in parallel, total test time is about ~10 seconds. It looks like there's ~3 seconds of purely serial work to actually get all of the tests (this is expensive since we have to load all of the test files and parse them in order to shard by method). There are also several tests that take multiple seconds to complete, so we're not getting it down much further w/o speeding up individual tests. That said, we can probably get this down to ~5-6 seconds without very much work. That's a pretty good speedup from ~53 seconds. Running tests individually, we have at least two tests that are flaky and depend on previous tests being run; that should be easy to fix. Also, perhaps unsurprisingly, it looks like tests that actually use multiprocessing get confused and fail. I would need to look into that further. If we shard tests by file (instead of method), the speedups are only about 2x, since run_webkit_tests_integrationtest.py is by far the long pole. Clearly we could speed up the individual tests in this file regardless of whether we want to run things in parallel or not, and that would be good. (I would say that the results are encouraging but not conclusive as to whether this is worth it). Created attachment 126445 [details]
dramatically cleaned up patch
Comment on attachment 126445 [details] dramatically cleaned up patch here's a cleaned up patch adding parallel support, as a delta from attachment 126446 [details] on bug 77687 Created attachment 128514 [details]
work in progress
Created attachment 128842 [details]
Work in progress (includes rollup of work in 77687)
|