Bug 210446 – delete-stale-build-files: Clean up sandbox and app caches

• Component: Tools / Tests
• Status: NEW

[Jonathan Bedard]

There are a number of sandboxes and app caches in /var/folders which can get quite large when repeatedly running tests. We should remove those along with stale build files.

[Jonathan Bedard]

<rdar://problem/54613619>

[Jonathan Bedard]

Created attachment 396315 [details]
Patch

[Geoffrey Garen]

Comment on attachment 396315 [details]
Patch

r=me

[Alexey Proskuryakov]

Comment on attachment 396315 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=396315&action=review

> Tools/BuildSlaveSupport/delete-stale-build-files:80
> +        '/usr/bin/find', '/var/folders', '-type', 'd', '-name', '*WebKit*',

This regex is very permissive, chances are that it will get too much. 

Also, aren’t error messages printed to inaccessible directories?

[Alexey Proskuryakov]

Comment on attachment 396315 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=396315&action=review

>> Tools/BuildSlaveSupport/delete-stale-build-files:80
>> +        '/usr/bin/find', '/var/folders', '-type', 'd', '-name', '*WebKit*',
> 
> This regex is very permissive, chances are that it will get too much. 
> 
> Also, aren’t error messages printed to inaccessible directories?

about inaccessible directories

[Alexey Proskuryakov]

Wrong radar was associated with this. Correct one:

rdar://problem/61269473

[Jonathan Bedard]

(In reply to Alexey Proskuryakov from comment #5)
> Comment on attachment 396315 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=396315&action=review
> 
> >> Tools/BuildSlaveSupport/delete-stale-build-files:80
> >> +        '/usr/bin/find', '/var/folders', '-type', 'd', '-name', '*WebKit*',
> > 
> > This regex is very permissive, chances are that it will get too much. 

I could narrow the regex to only include WebKitTestRunner, DumpRenderTree and sandboxes.

On the other hand, everything that matched that regex on the bot seemed like something we wanted to delete. between test runs.

> > 
> > Also, aren’t error messages printed to inaccessible directories?
> 
> about inaccessible directories

Yes, that's why I'm not passing any arguments to subprocess, by default, stdout and stderr are printed to the parent process's stdout and stderr. Although interestingly, there are not inaccessible directories on the bots.

[Alexey Proskuryakov]

Both findings are surprising to me. When I run this command on my development machine, there are a LOT of inaccessible directories, and many directories are matched that should not be. E.g. T/com.apple.Safari/WebKit, T/com.apple.Safari.CacheDeleteExtension/WebKit, T/WebKitPlugin-71Fvu9, T/com.apple.WebKit.WebContent+com.apple.Safari, C/com.apple.WebKit.WebContent+com.apple.finder/com.apple.WebKit.WebContent.

In fact, why even "sudo rm"? Directories that we want to delete are owned by the buildbot user.

[Jonathan Bedard]

(In reply to Alexey Proskuryakov from comment #8)
> ...
> 
> In fact, why even "sudo rm"? Directories that we want to delete are owned by
> the buildbot user.

Actually, that's a good point. And to take it even further, we shouldn't run find with sudo either. The only catch (and the reason I used sudo in the first place) is that find will return with a non-zero exit code when it can't traverse a directory.

[Jonathan Bedard]

Created attachment 396336 [details]
Patch

[Alexey Proskuryakov]

Comment on attachment 396336 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=396336&action=review

> Tools/BuildSlaveSupport/delete-stale-build-files:79
> +        '/usr/bin/find', '/var/folders', '-type', 'd', '-name', '*WebKit*',

I still think that this need to be more specific.

[Jonathan Bedard]

Discussed with Alexey, this makes more sense as a Buildbot step.

[Alexey Proskuryakov]

Comment on attachment 396336 [details]
Patch

Marking r- per the above.