Once the fix for Bug 16869 lands, we need a way to record and reply a randomized list of tests easily. The simplest way would be to write out a file with the list of tests, then add a mechanism to run-webkit-tests to use that file for its test order (unless it already does that).
The replay-part should be mostly handled by
Although it might re-segment the tests in some arbitrary way unless you specify --run-singly. I think for purposes of automatically bisecting, you would want run-webkit-tests to dump a per-worker list of executed tests.