Bug 95096 - Random test-webkitpy failures on the buildbot
Summary: Random test-webkitpy failures on the buildbot
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Zan Dobersek
Depends on:
Reported: 2012-08-27 09:39 PDT by Zan Dobersek
Modified: 2012-08-29 23:18 PDT (History)
3 users (show)

See Also:

Run test-webkitpy in 1 child process on GTK builders (2.13 KB, patch)
2012-08-29 06:20 PDT, Zan Dobersek
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Zan Dobersek 2012-08-27 09:39:20 PDT
When running webkitpy tests on the GTK 64-bit Release there are failures occurring every few runs:

This is really annoying as it's causing unnecessary redness. Looking through other builders, I only found one similar failure occurrence on the Chromium Linux Tests builder:

This probably assures that the failures are not port- or builder-specific. Rather than that I'd blame the python tests being run in parallel. On the GTK 64-bit Release builder this means up to 24 tests run at the same time.

I can't reproduce the failures locally in any way, the system being capable of running only 4 tests in parallel. OTOH, with all the tests being run serially (child_processes=1) the tests pass consistently and still take roughly 10 seconds to run (as compared to the 3-3.5 seconds when run in parallel (locally)).

Because of that I'd propose to either run the python tests serially on all the builders or at least the GTK-specific ones.

Dirk, your comments are most welcome.
Comment 1 Dirk Pranke 2012-08-28 14:51:37 PDT
ideally we should figure out why the tests are failing when run in parallel.

that said, redness is bad so I'm fine if we disable parallel testing on some of the builders, either as a temporary fix while we fix these tests, or indefinitely until you start worrying about cycle time. 

That said, I don't want to disable it on all of them, I'm happy to leave it on for Chromium, for example.

On a side note, we should poke into why test-webkitpy thinks it's logging to a terminal rather than a logfile. Any chance usePty = True in your buildbot slave config?
Comment 2 Zan Dobersek 2012-08-29 06:20:43 PDT
Created attachment 161209 [details]
Run test-webkitpy in 1 child process on GTK builders
Comment 3 Zan Dobersek 2012-08-29 23:18:52 PDT
Comment on attachment 161209 [details]
Run test-webkitpy in 1 child process on GTK builders

Clearing flags on attachment: 161209

Committed r127104: <http://trac.webkit.org/changeset/127104>
Comment 4 Zan Dobersek 2012-08-29 23:18:57 PDT
All reviewed patches have been landed.  Closing bug.