Bug 93372 – Web Inspector: add memory used by memory allocator itself to NMI

• Component: Web Inspector (Deprecated)
• Status: RESOLVED FIXED

[Alexei Filippov]

EOM

[Alexei Filippov]

Created attachment 156942 [details]
Patch

[WebKit Review Bot]

Comment on attachment 156942 [details]
Patch

Attachment 156942 [details] did not pass chromium-ews (chromium-xvfb):
Output: http://queues.webkit.org/results/13449487

[Alexei Filippov]

Created attachment 156948 [details]
Patch

[Alexei Filippov]

Created attachment 156958 [details]
Patch

[WebKit Review Bot]

Please wait for approval from [e-mail], [e-mail], [e-mail], [e-mail] or [e-mail] before submitting, as this patch contains changes to the Chromium public API. See also https://trac.webkit.org/wiki/ChromiumWebKitAPI.

[Alexei Filippov]

Chromium counterpart is at https://chromiumcodereview.appspot.com/10823205/

[Pavel Feldman]

Comment on attachment 156958 [details]
Patch

It has been pending for too long. Clearing r?

[Alexei Filippov]

Created attachment 164740 [details]
Patch

[Pavel Feldman]

Comment on attachment 164740 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=164740&action=review

> Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:78
> +    if (base::allocator::GetProperty("generic.heap_size", &heapSize)

I don't see how this resolves concerns raised my jamesr@.

[Alexei Filippov]

(In reply to comment #9)
> (From update of attachment 164740 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=164740&action=review
> 
> > Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:78
> > +    if (base::allocator::GetProperty("generic.heap_size", &heapSize)
> 
> I don't see how this resolves concerns raised my jamesr@.

Sorry, what exact concerns are you talking about? AFAIR main James' concern was that the previous patch introduced a dedicated API function for malloc waste.

With the current patch I don't change the API but rather use the memoryUsageByComponents function that purpose is exactly to report memory from components external to WebKit.

[James Robinson]

Comment on attachment 164740 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=164740&action=review

>>> Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:78
>>> +    if (base::allocator::GetProperty("generic.heap_size", &heapSize)
>> 
>> I don't see how this resolves concerns raised my jamesr@.
> 
> Sorry, what exact concerns are you talking about? AFAIR main James' concern was that the previous patch introduced a dedicated API function for malloc waste.
> 
> With the current patch I don't change the API but rather use the memoryUsageByComponents function that purpose is exactly to report memory from components external to WebKit.

Don't call base:: directly from WebKit code - you have to go through the API.

[Alexei Filippov]

ok. Got it! Then let's go back to my previous patch.

[James Robinson]

I'd recommend a less charged term than "waste". To me that term implies it should be zero but in reality an allocator has to use at least some memory to work at all.

Other than that I'll defer to Pavel.

[Alexei Filippov]

James, I'm fine with using another term. But I'm not sure which one.
I can think of several variants:
  a) Memory allocator reserve
  b) Memory allocator free lists
  c) Memory allocator internals
others?

But I don't like a) because it sounds like it can be controlled how much memory to reserve, while it is not, because it mostly comes from fragmentation.
Also c) sounds like that the memory is needed purely for the allocator purposes.
b) sounds better because most of the retained memory indeed belongs to free lists but not all.

May be you can suggest something else?

[James Robinson]

This memory is needed purely for allocator internals, so (c) sounds correct.

[Alexei Filippov]

Created attachment 165163 [details]
Patch

[Yury Semikhatsky]

Comment on attachment 165163 [details]
Patch

This part of the change looks good to me, but we need to come to agreement with James on the Chromium-side support for this: https://chromiumcodereview.appspot.com/10823205/

[James Robinson]

I don't think any additional input from me is going to be helpful here - I suggest seeing what the other WebKit API owners think and go with that.

[Alexei Filippov]

Created attachment 177580 [details]
Patch

[Eric Seidel (no email)]

Comment on attachment 177580 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=177580&action=review

> Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:77
> +    // TCMalloc memory reporting.
> +    if (WebKit::Platform::current()->memoryAllocatorWasteInBytes(&size))
> +        components.append(ComponentInfo("MallocWaste", size));

Chrome on Mac also uses FastMalloc (aka an older TCMalloc) down in WTF.  Also I think libpng has its own allocator which hands out 8kb chunks.  These may need similar support.  Mac (and android) don't use Chrome's TCMalloc I believe.

[Alexei Filippov]

Comment on attachment 177580 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=177580&action=review

>> Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:77
>> +        components.append(ComponentInfo("MallocWaste", size));
> 
> Chrome on Mac also uses FastMalloc (aka an older TCMalloc) down in WTF.  Also I think libpng has its own allocator which hands out 8kb chunks.  These may need similar support.  Mac (and android) don't use Chrome's TCMalloc I believe.

Eric, thanks for the info. The webkit part is intended to be platform independent, so these details should go under memoryAllocatorWasteInBytes implementation. At some point I may break down this malloc waste into subcomponents like libpng stuff, free lists, etc. But I'm not sure if it adds a value at the moment.

I agree the TCMalloc comment is a bit misleading. I'll change it.

[Alexei Filippov]

Created attachment 177620 [details]
Patch

[Eric Seidel (no email)]

Comment on attachment 177620 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=177620&action=review

> Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:75
> +    // Memory allocator waste reporting.

I don't think these comments really add anything. :)  The code is pretty self-documenting.

Could you link to the Chrome-side patch?  I worry you may not be able to implement the wtf-fastMalloc part (for Chrome Mac) in the chrome-side.

[Alexei Filippov]

The link is in the comment #6, but I can repeat it ;-)
https://chromiumcodereview.appspot.com/10823205/

[Eric Seidel (no email)]

(In reply to comment #23)
> (From update of attachment 177620 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=177620&action=review
>
> Could you link to the Chrome-side patch?  I worry you may not be able to implement the wtf-fastMalloc part (for Chrome Mac) in the chrome-side.

Thanks. :)  But I still worry.  I don't believe it's possible to gather all the information from the Chrome side.  That assumes that the embedder always controls the allocator, which is not the case. :)

[Alexei Filippov]

Yes, the work on allocators waste is not over yet.
This patch gives the embedder opportunity to report its allocator stats. I believe wtf-fastMalloc is a bit different story, so I'd like to have it as a separate change.

[Alexei Filippov]

Created attachment 179019 [details]
Patch

[Alexei Filippov]

Removed the comments.

[WebKit Review Bot]

Comment on attachment 179019 [details]
Patch

Clearing flags on attachment: 179019

Committed r137457: <http://trac.webkit.org/changeset/137457>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.