Bug 112978 – [chromium] Linux ASAN WebKit canary is running out of memory

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Dirk Pranke]

turns out that running the ASAN processing on the stderr takes an enormous amount of memory, and if you run multiple of these at a time, you can run out of memory on the bot (looks like we can manage two on the bot now, but three probably causes us to run out of memory).

See, e.g.: http://build.chromium.org/p/chromium.webkit/builders/WebKit%20Linux%20ASAN/builds/5269/steps/webkit_tests/logs/stdio

It'd not obvious that there's a great answer here, but for the moment I'm going to try serializing the asan_filter script so we only run one at a time ...

[Dirk Pranke]

Created attachment 194383 [details]
Patch

[Tony Chang]

Are you trying to lock across python processes?  Why don't we just prepend flock to the commands you want to serialize?

[Dirk Pranke]

(In reply to comment #2)
> Are you trying to lock across python processes?  Why don't we just prepend flock to the commands you want to serialize?

Well, this is portable to windows (not that we run asan on windows) and happened to reuse the well-tested stuff we have lying around :). I can change it to just use flock if you'd prefer.

[Dirk Pranke]

but, yes, we're trying to lock across processes.

[Dirk Pranke]

oh, one minor additional difference is that http_lock code ensures access is granted in order (FIFO). AFAIK, regular flock doesn't do that?

[Dirk Pranke]

that may also not really matter in this case.

[Tony Chang]

Comment on attachment 194383 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=194383&action=review

I think it's OK to use HttpLock, but it's a bit confusing (due to naming) and I have more confidence in flock.

> Tools/Scripts/webkitpy/layout_tests/port/chromium.py:383
> +                    stderr = self._executive.run_command(['c++filt'], input=output, decode_output=False)

Can we run c++filt outside the lock?  That's not expensive, is it?

Should there be a unit test for this?

[Tony Chang]

Comment on attachment 194383 [details]
Patch

Feel free to try this, I won't be around later tonight for further reviews.

[Dirk Pranke]

Comment on attachment 194383 [details]
Patch

thanks!

[WebKit Review Bot]

Comment on attachment 194383 [details]
Patch

Clearing flags on attachment: 194383

Committed r146546: <http://trac.webkit.org/changeset/146546>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.