Bug 52364 - Make old-run-webkit-tests more robust on buildbots
Summary: Make old-run-webkit-tests more robust on buildbots
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: All All
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-01-13 05:28 PST by Csaba Osztrogonác
Modified: 2011-01-13 13:11 PST (History)
8 users (show)

See Also:


Attachments
old-run-webkit-tests fix (1.16 KB, patch)
2011-01-13 05:31 PST, Csaba Osztrogonác
no flags Details | Formatted Diff | Diff
exit-after-n-failures 500 patch (1.49 KB, patch)
2011-01-13 05:36 PST, Csaba Osztrogonác
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Csaba Osztrogonác 2011-01-13 05:28:17 PST
Maciej committed a debug print accidentally (http://trac.webkit.org/changeset/75682)
and it made all layout tests fail and buildbots sick. The size of results for 
~22000 failing layout tests is more than 100Mb.

I propose some changes to avoid overloading buildbot slaves and master:

1.) add --exit-after-n-failures 500 option to the old-run-webkit-tests
It is enough for online rebaselineing and avoiding 100Mb sizes results

2.) old-run-webkit-tests should remove results directory before testing
(Now test-result-archive does it after zipping. But if it is terminated
or killed, the results directory will contain some mess.)

Patches are coming soon.
Comment 1 Csaba Osztrogonác 2011-01-13 05:31:05 PST
Created attachment 78798 [details]
old-run-webkit-tests fix
Comment 2 Csaba Osztrogonác 2011-01-13 05:36:15 PST
Created attachment 78799 [details]
exit-after-n-failures 500 patch
Comment 3 Darin Adler 2011-01-13 09:14:23 PST
Comment on attachment 78798 [details]
old-run-webkit-tests fix

Seems OK. As long as rmtree is available in all the versions of perl we support, and error handling works. I’ll be optimistic.
Comment 4 Darin Adler 2011-01-13 09:15:12 PST
Comment on attachment 78799 [details]
exit-after-n-failures 500 patch

View in context: https://bugs.webkit.org/attachment.cgi?id=78799&action=review

Not great to attach two patches to the same bug.

> Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:196
> +               WithProperties("--%(configuration)s"), "--exit-after-n-crashes-or-timeouts", "20",  "--exit-after-n-failures", "500"]

Would be slightly better to have a comment explaining the reason for this. There’s also an extra space after the comma.
Comment 5 WebKit Commit Bot 2011-01-13 10:23:15 PST
Comment on attachment 78798 [details]
old-run-webkit-tests fix

Clearing flags on attachment: 78798

Committed r75721: <http://trac.webkit.org/changeset/75721>
Comment 6 WebKit Commit Bot 2011-01-13 10:37:49 PST
Comment on attachment 78799 [details]
exit-after-n-failures 500 patch

Clearing flags on attachment: 78799

Committed r75726: <http://trac.webkit.org/changeset/75726>
Comment 7 WebKit Commit Bot 2011-01-13 10:37:55 PST
All reviewed patches have been landed.  Closing bug.
Comment 8 WebKit Review Bot 2011-01-13 13:11:42 PST
http://trac.webkit.org/changeset/75726 might have broken Leopard Intel Release (Tests)