Bug 30809 - buildbots should use --exit-after-N-failures
Summary: buildbots should use --exit-after-N-failures
Status: RESOLVED WONTFIX
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P2 Normal
Assignee: Csaba Osztrogonác
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-10-27 01:43 PDT by Eric Seidel (no email)
Modified: 2009-12-16 16:16 PST (History)
5 users (show)

See Also:


Attachments
Patch v1 (1.49 KB, patch)
2009-10-27 01:51 PDT, Eric Seidel (no email)
no flags Details | Formatted Diff | Diff
Upped limit to 20 per Mark's request. Also added failure line to phase output. (1.94 KB, patch)
2009-10-27 02:05 PDT, Eric Seidel (no email)
no flags Details | Formatted Diff | Diff
upped limit to 40 (1.46 KB, patch)
2009-11-24 08:16 PST, Csaba Osztrogonác
eric: review-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Seidel (no email) 2009-10-27 01:43:44 PDT
buildbots should use --exit-after-N-failures

http://build.webkit.org/builders/Leopard%20Intel%20Release%20%28Tests%29/builds/6544
has taken over an hour to run because every layout test is crashing!

http://build.webkit.org/builders/Leopard%20Intel%20Release%20%28Tests%29/builds/6544/steps/layout-test/logs/stdio

It should have exited after the first 10 failures or similar.
Comment 1 Eric Seidel (no email) 2009-10-27 01:51:40 PDT
Created attachment 41936 [details]
Patch v1
Comment 2 Eric Seidel (no email) 2009-10-27 02:05:38 PDT
Created attachment 41937 [details]
Upped limit to 20 per Mark's request.  Also added failure line to phase output.
Comment 3 David Levin 2009-10-28 21:35:36 PDT
Comment on attachment 41937 [details]
Upped limit to 20 per Mark's request.  Also added failure line to phase output.

Looks good to me and it sounds like you got feedback from Mark as well.

I'll not cq+ this at the moment in case anyone else (Mark) wants a last chance to comment.
Comment 4 Eric Seidel (no email) 2009-11-01 18:20:12 PST
Comment on attachment 41937 [details]
Upped limit to 20 per Mark's request.  Also added failure line to phase output.

No comments.  Mark has to manually update the buildbots anyway, so he will see it again then.
Comment 5 WebKit Commit Bot 2009-11-01 19:31:02 PST
Comment on attachment 41937 [details]
Upped limit to 20 per Mark's request.  Also added failure line to phase output.

Clearing flags on attachment: 41937

Committed r50401: <http://trac.webkit.org/changeset/50401>
Comment 6 WebKit Commit Bot 2009-11-01 19:31:07 PST
All reviewed patches have been landed.  Closing bug.
Comment 7 Csaba Osztrogonác 2009-11-24 08:16:33 PST
Created attachment 43773 [details]
upped limit to 40

Yesterday master config was updated and it made QtBuildBot unhappy,
because we had 16 failing and 31 new tests. ( 31+16 > 20 ) On the
one hand failing tests counter shouldn't include new tests, that's why
I propose to modify it: https://bugs.webkit.org/show_bug.cgi?id=31829.
On the other hand if a patch committed make some layout tests fail,
and later an other patch make more failing tests, etc, we can easily 
lost which commit caused which fails. If we don't have this information,
debugging can be more difficult.

We make an effort to fix Qt related bugs ASAP, but quite a lot Qt-port 
maintainer sleep when american guys work. (because of timezone difference)

To make QtBuildbot and Qt guys happier, I suggest increasing 
the failure limit to 40.
Comment 8 Adam Barth 2009-11-30 12:37:20 PST
style-queue successfully ran check-webkit-style on attachment 43773 [details] without any errors
Comment 9 Eric Seidel (no email) 2009-12-09 00:37:24 PST
Comment on attachment 43773 [details]
upped limit to 40

I don't think we want this change.  I think this is fixing a symptom, not the root problem.  If I correctly overheard in #webkit, Mark Rowe feels similar.