Bug 52364

Summary:

Make old-run-webkit-tests more robust on buildbots

Product:

WebKit

Reporter:

Csaba Osztrogonác <ossy>

Component:

Tools / Tests

Assignee:

Nobody <webkit-unassigned>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

abarth, aroben, commit-queue, eric, levin, mjs, webkit.review.bot, wsiegrist

Priority:

Version:

528+ (Nightly build)

Hardware:

All

OS:

All

Attachments:

Description	Flags
old-run-webkit-tests fix	none
exit-after-n-failures 500 patch	none

Description Csaba Osztrogonác 2011-01-13 05:28:17 PST

Maciej committed a debug print accidentally (http://trac.webkit.org/changeset/75682)
and it made all layout tests fail and buildbots sick. The size of results for 
~22000 failing layout tests is more than 100Mb.

I propose some changes to avoid overloading buildbot slaves and master:

1.) add --exit-after-n-failures 500 option to the old-run-webkit-tests
It is enough for online rebaselineing and avoiding 100Mb sizes results

2.) old-run-webkit-tests should remove results directory before testing
(Now test-result-archive does it after zipping. But if it is terminated
or killed, the results directory will contain some mess.)

Patches are coming soon.

Comment 1 Csaba Osztrogonác 2011-01-13 05:31:05 PST

Created attachment 78798 [details]
old-run-webkit-tests fix

Comment 2 Csaba Osztrogonác 2011-01-13 05:36:15 PST

Created attachment 78799 [details]
exit-after-n-failures 500 patch

Comment 3 Darin Adler 2011-01-13 09:14:23 PST

Comment on attachment 78798 [details]
old-run-webkit-tests fix

Seems OK. As long as rmtree is available in all the versions of perl we support, and error handling works. I’ll be optimistic.

Comment 4 Darin Adler 2011-01-13 09:15:12 PST

Comment on attachment 78799 [details]
exit-after-n-failures 500 patch

View in context: https://bugs.webkit.org/attachment.cgi?id=78799&action=review

Not great to attach two patches to the same bug.

> Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:196
> +               WithProperties("--%(configuration)s"), "--exit-after-n-crashes-or-timeouts", "20",  "--exit-after-n-failures", "500"]

Would be slightly better to have a comment explaining the reason for this. There’s also an extra space after the comma.

Comment 5 WebKit Commit Bot 2011-01-13 10:23:15 PST

Comment on attachment 78798 [details]
old-run-webkit-tests fix

Clearing flags on attachment: 78798

Committed r75721: <http://trac.webkit.org/changeset/75721>

Comment 6 WebKit Commit Bot 2011-01-13 10:37:49 PST

Comment on attachment 78799 [details]
exit-after-n-failures 500 patch

Clearing flags on attachment: 78799

Committed r75726: <http://trac.webkit.org/changeset/75726>

Comment 7 WebKit Commit Bot 2011-01-13 10:37:55 PST

All reviewed patches have been landed.  Closing bug.

Comment 8 WebKit Review Bot 2011-01-13 13:11:42 PST

http://trac.webkit.org/changeset/75726 might have broken Leopard Intel Release (Tests)