| Summary: | fast/encoding/parser-tests often timeout on debug bots | ||||||
|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Dirk Pranke <dpranke> | ||||
| Component: | Tools / Tests | Assignee: | Jenn Braithwaite <jennb> | ||||
| Status: | RESOLVED FIXED | ||||||
| Severity: | Normal | CC: | abarth, dglazkov, eric, jennb | ||||
| Priority: | P2 | ||||||
| Version: | 528+ (Nightly build) | ||||||
| Hardware: | PC | ||||||
| OS: | OS X 10.5 | ||||||
| Attachments: |
|
||||||
|
Description
Dirk Pranke
2011-05-03 17:37:28 PDT
I will halve the size of each batch of tests again. Created attachment 92351 [details]
patch
Adam, could you review this as you reviewed the similar change in bug 51721 - this time I further halved each batch of tests. Comment on attachment 92351 [details]
patch
I don't really understand the details of what this patch is doing, but it looks fine to me.
(In reply to comment #4) > (From update of attachment 92351 [details]) > I don't really understand the details of what this patch is doing, but it looks fine to me. Each batch used to run 10 tests, it now runs 5. The script takes the first and last test, e.g. runtests(10, 19) is changed to runtests(10, 14). A new test is added to invoke runtests(15, 19). The expected results files were also updated so that the expected failures are in the expected results file for the correct batch. I'm sorry, the bug report wasn't clear (editing to fix). It's not the batch size that's the problem. If you look at the numbers on the dashboard, you'll see that most of the time the batches run in a couple of seconds, and once in a while they'll take a really long time and time out. We need to figure out why they're timing out, not speed up the average case. (In reply to comment #6) > I'm sorry, the bug report wasn't clear (editing to fix). > > It's not the batch size that's the problem. If you look at the numbers on the dashboard, you'll see that most of the time the batches run in a couple of seconds, and once in a while they'll take a really long time and time out. > > We need to figure out why they're timing out, not speed up the average case. Is there a way to tell from the logs whether the machine was under load when the timeouts occurred? Reducing the batch sizes will reduce timeouts that occur due to the machine being under load, so it is still worth doing. (In reply to comment #7) > (In reply to comment #6) > > I'm sorry, the bug report wasn't clear (editing to fix). > > > > It's not the batch size that's the problem. If you look at the numbers on the dashboard, you'll see that most of the time the batches run in a couple of seconds, and once in a while they'll take a really long time and time out. > > > > We need to figure out why they're timing out, not speed up the average case. > > Is there a way to tell from the logs whether the machine was under load when the timeouts occurred? Reducing the batch sizes will reduce timeouts that occur due to the machine being under load, so it is still worth doing. Not really, but because of the way new-run-webkit-tests works, you should assume the machine is under load all the time (because that's the design goal, to get through the tests as quickly as possible). It would be interesting to see if the tests never timed out if the machine wasn't under load; there is a patch floating around to try that, but it hasn't landed yet. Do we know why these tests are slow? (In reply to comment #10) > Do we know why these tests are slow? I do not know why theses tests are occasionally slow. It's interesting, but these tests never failed on upstream Chromium bots: http://test-results.appspot.com/dashboards/flakiness_dashboard.html#group=%40ToT%20-%20webkit.org&tests=fast%2Fencoding%2Fparser-tests *** Bug 64498 has been marked as a duplicate of this bug. *** Comment on attachment 92351 [details]
patch
Please re-nominate for review if this is still a problem.
This has been resolved in Blink; it's not clear if this was ever an issue for non-Chromium ports given that they had a much longer timeout. |