new-run-webkit-tests: order shards by name, not number of tests
Created attachment 97356 [details] Patch
I don't know if we even care about this, but I'll file it just to track things.
What's the rationale behind this? Is the theory that tests will be less flaky because we're more likely to run them in the same order as running them serially? I guess since the http tests are by far the gating factor on cycle time, this would only slow down test runs that aren't running the http tests.
See the blocker bug ... I was attempting to ensure that NRWT ran the same tests in the same order as ORWT. In theory, running tests in a different order could reveal inter-test dependencies and cause failures with NRWT that don't occur with ORWT. That said, I don't actually care personally if we do make NRWT match ORWT test-for-test, I just wanted to satisfy to myself that it could and there weren't other bugs (which I did find, and I am now satisfied). I'm happy to close this as WONTFIX, or land it; it doesn't much matter to me. Given that we now shard tests by directory, rather than only top-level directory, there are usually far more shards than workers, and I don't think any ordering matters much for determining the total test run time.
Comment on attachment 97356 [details] Patch Seems fine to me. If it ends up reducing flakiness, it's certainly worth it. If it doesn't it's not a big deal. We can try it and look at the before/after cycle time on the chromium bots to make sure it doesn't have a significant impact.
Comment on attachment 97356 [details] Patch Clearing flags on attachment: 97356 Committed r89302: <http://trac.webkit.org/changeset/89302>
All reviewed patches have been landed. Closing bug.
Looking at http://build.chromium.org/p/chromium.webkit/stats, the doesn't seem to be any cycle time regression on any of the bots.