Bug 138958 – [meta] Run http tests parallel

• Component: New Bugs
• Status: RESOLVED FIXED

[Csaba Osztrogonác]

It is a meta bug to collect issues related to running http tests parallel.
The final goal is to be able run them parallel as the other tests everywhere
and be able remove the complex logic needed to serialize only http tests.

[Alexey Proskuryakov]

I think that we are ready to flip the switch (at least on Mac), but I can't figure out which switch to flip.

There are still some failures when running with --fully-parallel, but they should not affect normal mode, as far as I can tell.

[Csaba Osztrogonác]

(In reply to comment #1)
> I think that we are ready to flip the switch (at least on Mac), but I can't
> figure out which switch to flip.
> 
> There are still some failures when running with --fully-parallel, but they
> should not affect normal mode, as far as I can tell.

"-f, --fully-parallel - run all tests in parallel"

Ah, this help message is a little bit confusing.

Tests are sharded by subdirectories, leaf subdirectories are atomic 
and can't be distributed to different threads by default. Threads
pick up shards from the queue.

This fully parallel option disables sharding and each 
threads can pick up tests from the queue instead of shards.
I think we aren't ready for it yet, but it can reveal many
flakiness. It would be a good expetiment.

Back to parallel http test running. Now http and perf tests are considered as locked tests and locked tests run on 1 thread by default. You can easily override it with: --max-locked-shards command line option or WEBKIT_TEST_MAX_
LOCKED_SHARDS. (cmdline is the stronger one) The default option is defined to 1 in base.py in default_max_locked_shards. First we can override it for ports one by one. And then in base.py once it is stable everywhere. The ideal number would be the same as default_child_processes, but it depends on how much concurrent request can be served by the http server.

[Alexey Proskuryakov]

The question is how do I make HTTP tests parallel while keeping perf tests non-parallel.

[Csaba Osztrogonác]

(In reply to comment #3)
> The question is how do I make HTTP tests parallel while keeping perf tests
> non-parallel.

I think the easiest way is what I suggested in bug136722 - run them in
a different buildstep on the bots.

[Csaba Osztrogonác]

One more thing. perf tests can run parallel now with non perf tests. If
we increasw max_locked_shards, they will run parallel with more http test
thread instead of random non-http test. But 2 perf test won't run parallel,
because they are in the same shard.

[Alexey Proskuryakov]

Created attachment 242595 [details]
proposed patch

This makes debug tests run in 8 minutes and 12 seconds on my Mac Pro. Without the patch, they used to take over 15 minutes.

[Daniel Bates]

Comment on attachment 242595 [details]
proposed patch

View in context: https://bugs.webkit.org/attachment.cgi?id=242595&action=review

Looks sane to me.

> Tools/ChangeLog:17
> +        think that these parameters do what we wanted them to do; Apache handles the load

Nit: do => did

Or

wanted => want

[Alexey Proskuryakov]

Created attachment 242599 [details]
patch for landing

[WebKit Commit Bot]

Comment on attachment 242599 [details]
patch for landing

Clearing flags on attachment: 242599

Committed r176830: <http://trac.webkit.org/changeset/176830>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.