Summary: | Large logs can bring down the webkit master | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Product: | WebKit | Reporter: | Lucas Forschler <lforschler> | ||||||||||
Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> | ||||||||||
Status: | RESOLVED CONFIGURATION CHANGED | ||||||||||||
Severity: | Normal | CC: | ap, bfulgham, commit-queue, dbates, dburkart, lforschler, ossy, webkit-bug-importer | ||||||||||
Priority: | P2 | Keywords: | InRadar | ||||||||||
Version: | 528+ (Nightly build) | ||||||||||||
Hardware: | Unspecified | ||||||||||||
OS: | Unspecified | ||||||||||||
Attachments: |
|
Description
Lucas Forschler
2013-09-30 10:28:29 PDT
I believe this is the stdio of the leaks process that is consuming so much memory. buildbot streams the stdio from the slave back to the master. Evidently (maybe?) buildbot does not write this to disk until it is finished. We don't need this data, so we can hopefully redirect it to /dev/null. We should verify we still get the results in the leaks viewer. Created attachment 271073 [details]
Patch
This patch instructs buildbot to throw away stderr and stdout, instead of transferring that output to the master.
Comment on attachment 271073 [details]
Patch
LGTM!
build master restarted. Reopening because this didn't fix the problem. Created attachment 271108 [details]
Patch take 2
I think the problem with the earlier patch is that these values need to be passed in via constructor so that they percolate to the ShellMixin class.
Attachment 271108 [details] did not pass style-queue:
ERROR: Websites/test-results/ChangeLog:1: ChangeLog entry has no bug number [changelog/bugnumber] [5]
Total errors found: 1 in 4 files
If any of these errors are false positives, please file a bug against check-webkit-style.
(In reply to comment #9) > Attachment 271108 [details] did not pass style-queue: > > > ERROR: Websites/test-results/ChangeLog:1: ChangeLog entry has no bug number > [changelog/bugnumber] [5] > Total errors found: 1 in 4 files > > > If any of these errors are false positives, please file a bug against > check-webkit-style. I think this is because the diff came out looking weird, which must confuse check-webkit-style. Comment on attachment 271108 [details]
Patch take 2
Dana, your changelog entries are messed up, please make sure that your patch modifies only the top of the Tools/ChangeLog file, and also fill in the Websites/test-results/ChangeLog file.Dana, your changelog entries are messed up, please make sure that your patch modifies only the top of the Tools/ChangeLog file, and also fill in the Websites/test-results/ChangeLog file.
(In reply to comment #11) > Comment on attachment 271108 [details] > Patch take 2 > > Dana, your changelog entries are messed up, please make sure that your patch > modifies only the top of the Tools/ChangeLog file, and also fill in the > Websites/test-results/ChangeLog file.Dana, your changelog entries are messed > up, please make sure that your patch modifies only the top of the > Tools/ChangeLog file, and also fill in the Websites/test-results/ChangeLog > file. The first ChangeLog file is not messed up; it happened because the next entry down simply has the same text as the newest entry (as it was the first attempt at this patch, which didn't work). I will remove the changed files in Websites/test-results, as that was a mistaken commit, and post a new patch. Created attachment 271472 [details]
Patch take 3
Fixes up the patch to remove changes to Websites/test-results directory.
Comment on attachment 271472 [details] Patch take 3 Attachment 271472 [details] did not pass mac-debug-ews (mac): Output: http://webkit-queues.webkit.org/results/841801 New failing tests: js/regress/Float64Array-to-Int16Array-set.html js/regress/Float32Array-to-Float64Array-set.html Created attachment 271483 [details]
Archive of layout-test-results from ews116 for mac-yosemite
The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116 Port: mac-yosemite Platform: Mac OS X 10.10.5
Comment on attachment 271472 [details]
Patch take 3
OK.
(In reply to comment #16) > Comment on attachment 271472 [details] > Patch take 3 > > OK. This is OK as a temporary fix assuming it still happens and Lucas Forschler's remark that the stdout/stderr output is not useful is accurate. Does this issue still happen? I mean this issue was first reported in 2013 and this patch (attachment #271472 [details]) was posted over two months ago (02/16). In the long term we should look to fix the leaks bot so that it does not write 7TB of output. Why is it writing so much data? If this data is not important then can we teach the bot to discard it on its side instead of sending it to the Buildbot master to discard. If you chose to land this patch (attachment #271472 [details]) then please file a follow up bug to investigate and address the issue about the leaks bot writing large amounts of output to stdout and stderr. |