Bug 128388 - Make EWS retry one more time after cleaning the build directory
Summary: Make EWS retry one more time after cleaning the build directory
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-02-07 12:07 PST by Adrian Perez de Castro (EWS)
Modified: 2014-10-10 14:46 PDT (History)
9 users (show)

See Also:


Attachments
Patch (3.50 KB, patch)
2014-02-07 12:09 PST, Adrian Perez de Castro (EWS)
no flags Details | Formatted Diff | Diff
Patch (3.55 KB, patch)
2014-02-07 12:23 PST, Adrian Perez
no flags Details | Formatted Diff | Diff
Patch (3.45 KB, patch)
2014-02-10 07:13 PST, Adrian Perez
ap: review-
ap: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Adrian Perez de Castro (EWS) 2014-02-07 12:07:55 PST
Make EWS retry one more time after cleaning the build directory
Comment 1 Adrian Perez de Castro (EWS) 2014-02-07 12:09:44 PST
Created attachment 223482 [details]
Patch
Comment 2 Adrian Perez 2014-02-07 12:16:10 PST
This patch will help in cases where derived sources (or other files
created in the build directory) are causing builds to fail and cleaning
the build directory is needed. Usually in those cases we have been
manually handling the situation, and making the EWS bots do that
automatically would make them more resilient.

This would help situations like the one mentioned in bug #128370.
Comment 3 Adrian Perez 2014-02-07 12:23:08 PST
Created attachment 223483 [details]
Patch
Comment 4 Csaba Osztrogonác 2014-02-10 05:42:28 PST
I understand your intention to fix this annoying problem,  
but I'm not sure if it is the best fix for it. 

In my opinion it is too intrusive to remove the whole WebKitBuild (
including WebKitBuild/Dependencies too) always when the tree is red
or an incremental build issue occur. I think a simple "tree redness"
occur more often than the mentioned gtkdoc issue. And it seems it is
a GTK only problem, why should we remove the WebKitBuild on every
"tree redness" on every EWS?

I would prefer fixing gtkdoc somehow. Until the proper fix I can accept
a GTK only workaround like you proposed or a less intrusive workaround,
for example removing only the WebKitBuild/.../DerivedSources directory,
Comment 5 Csaba Osztrogonác 2014-02-10 05:43:09 PST
Comment on attachment 223483 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=223483&action=review

> Tools/Scripts/webkitpy/tool/bot/earlywarningsystemtask.py:64
> +        subprocess.call(["rm", "-rf", self._build_directory],
> +                stdout=None, stderr=None)

Just out of curiosity: Does it work on Windows too?
Comment 6 Adrian Perez 2014-02-10 07:13:53 PST
Created attachment 223706 [details]
Patch
Comment 7 Adrian Perez 2014-02-10 07:15:35 PST
(In reply to comment #5)
> (From update of attachment 223483 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=223483&action=review
> 
> > Tools/Scripts/webkitpy/tool/bot/earlywarningsystemtask.py:64
> > +        subprocess.call(["rm", "-rf", self._build_directory],
> > +                stdout=None, stderr=None)
> 
> Just out of curiosity: Does it work on Windows too?

I have uploaded a new version of the patch which uses shutil.rmtree()
from the Python standard library — that certainly works in Windows, too.
Comment 8 Adrian Perez 2014-02-10 08:11:54 PST
(In reply to comment #4)
> I understand your intention to fix this annoying problem,  
> but I'm not sure if it is the best fix for it. 
> 
> In my opinion it is too intrusive to remove the whole WebKitBuild (
> including WebKitBuild/Dependencies too) always when the tree is red
> or an incremental build issue occur. I think a simple "tree redness"
> occur more often than the mentioned gtkdoc issue. And it seems it is
> a GTK only problem, why should we remove the WebKitBuild on every
> "tree redness" on every EWS?

Note that the “webkit-build-directory” script is used to obtain the
build directory, so “WebKitBuild/Debug” or “WebKitBuild/Release” can
be removed, but “WebKitBuild/Dependencies” is never removed. Also,
provided that I would expect EWS bots to have “ccache” installed,
so in reality applying the patch is not that intrusive.
 
> I would prefer fixing gtkdoc somehow. Until the proper fix I can accept
> a GTK only workaround like you proposed or a less intrusive workaround,
> for example removing only the WebKitBuild/.../DerivedSources directory,

Fixing the invocation of gtk-doc is something that should also be
done. Nevertheless, there can be other cases in which the EWS bots
could choke and need a clean build—wiping just the derived sources
may not be enough.
Comment 9 Alexey Proskuryakov 2014-02-26 14:49:03 PST
I agree with Ossy, this looks like it will make EWS substantially slower for little benefit.