RESOLVED FIXED 93372
Web Inspector: add memory used by memory allocator itself to NMI
https://bugs.webkit.org/show_bug.cgi?id=93372
Summary Web Inspector: add memory used by memory allocator itself to NMI
Alexei Filippov
Reported 2012-08-07 08:06:41 PDT
EOM
Attachments
Patch (3.72 KB, patch)
2012-08-07 08:15 PDT, Alexei Filippov
no flags
Patch (3.76 KB, patch)
2012-08-07 08:43 PDT, Alexei Filippov
no flags
Patch (4.42 KB, patch)
2012-08-07 11:11 PDT, Alexei Filippov
no flags
Patch (3.25 KB, patch)
2012-09-19 07:56 PDT, Alexei Filippov
pfeldman: review-
Patch (4.32 KB, patch)
2012-09-21 11:43 PDT, Alexei Filippov
no flags
Patch (4.25 KB, patch)
2012-12-04 15:05 PST, Alexei Filippov
no flags
Patch (4.26 KB, patch)
2012-12-04 17:32 PST, Alexei Filippov
no flags
Patch (4.10 KB, patch)
2012-12-12 04:01 PST, Alexei Filippov
no flags
Alexei Filippov
Comment 1 2012-08-07 08:15:05 PDT
WebKit Review Bot
Comment 2 2012-08-07 08:19:22 PDT
Comment on attachment 156942 [details] Patch Attachment 156942 [details] did not pass chromium-ews (chromium-xvfb): Output: http://queues.webkit.org/results/13449487
Alexei Filippov
Comment 3 2012-08-07 08:43:41 PDT
Alexei Filippov
Comment 4 2012-08-07 11:11:05 PDT
WebKit Review Bot
Comment 5 2012-08-07 11:20:09 PDT
Please wait for approval from abarth@webkit.org, dglazkov@chromium.org, fishd@chromium.org, jamesr@chromium.org or tkent@chromium.org before submitting, as this patch contains changes to the Chromium public API. See also https://trac.webkit.org/wiki/ChromiumWebKitAPI.
Alexei Filippov
Comment 6 2012-08-07 11:40:55 PDT
Pavel Feldman
Comment 7 2012-08-31 02:09:14 PDT
Comment on attachment 156958 [details] Patch It has been pending for too long. Clearing r?
Alexei Filippov
Comment 8 2012-09-19 07:56:15 PDT
Pavel Feldman
Comment 9 2012-09-21 03:11:09 PDT
Comment on attachment 164740 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=164740&action=review > Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:78 > + if (base::allocator::GetProperty("generic.heap_size", &heapSize) I don't see how this resolves concerns raised my jamesr@.
Alexei Filippov
Comment 10 2012-09-21 05:23:43 PDT
(In reply to comment #9) > (From update of attachment 164740 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=164740&action=review > > > Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:78 > > + if (base::allocator::GetProperty("generic.heap_size", &heapSize) > > I don't see how this resolves concerns raised my jamesr@. Sorry, what exact concerns are you talking about? AFAIR main James' concern was that the previous patch introduced a dedicated API function for malloc waste. With the current patch I don't change the API but rather use the memoryUsageByComponents function that purpose is exactly to report memory from components external to WebKit.
James Robinson
Comment 11 2012-09-21 09:54:02 PDT
Comment on attachment 164740 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=164740&action=review >>> Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:78 >>> + if (base::allocator::GetProperty("generic.heap_size", &heapSize) >> >> I don't see how this resolves concerns raised my jamesr@. > > Sorry, what exact concerns are you talking about? AFAIR main James' concern was that the previous patch introduced a dedicated API function for malloc waste. > > With the current patch I don't change the API but rather use the memoryUsageByComponents function that purpose is exactly to report memory from components external to WebKit. Don't call base:: directly from WebKit code - you have to go through the API.
Alexei Filippov
Comment 12 2012-09-21 09:57:52 PDT
ok. Got it! Then let's go back to my previous patch.
James Robinson
Comment 13 2012-09-21 10:04:32 PDT
I'd recommend a less charged term than "waste". To me that term implies it should be zero but in reality an allocator has to use at least some memory to work at all. Other than that I'll defer to Pavel.
Alexei Filippov
Comment 14 2012-09-21 11:08:04 PDT
James, I'm fine with using another term. But I'm not sure which one. I can think of several variants: a) Memory allocator reserve b) Memory allocator free lists c) Memory allocator internals others? But I don't like a) because it sounds like it can be controlled how much memory to reserve, while it is not, because it mostly comes from fragmentation. Also c) sounds like that the memory is needed purely for the allocator purposes. b) sounds better because most of the retained memory indeed belongs to free lists but not all. May be you can suggest something else?
James Robinson
Comment 15 2012-09-21 11:18:20 PDT
This memory is needed purely for allocator internals, so (c) sounds correct.
Alexei Filippov
Comment 16 2012-09-21 11:43:00 PDT
Yury Semikhatsky
Comment 17 2012-09-24 06:55:56 PDT
Comment on attachment 165163 [details] Patch This part of the change looks good to me, but we need to come to agreement with James on the Chromium-side support for this: https://chromiumcodereview.appspot.com/10823205/
James Robinson
Comment 18 2012-09-24 17:11:15 PDT
I don't think any additional input from me is going to be helpful here - I suggest seeing what the other WebKit API owners think and go with that.
Alexei Filippov
Comment 19 2012-12-04 15:05:15 PST
Eric Seidel (no email)
Comment 20 2012-12-04 15:10:35 PST
Comment on attachment 177580 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=177580&action=review > Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:77 > + // TCMalloc memory reporting. > + if (WebKit::Platform::current()->memoryAllocatorWasteInBytes(&size)) > + components.append(ComponentInfo("MallocWaste", size)); Chrome on Mac also uses FastMalloc (aka an older TCMalloc) down in WTF. Also I think libpng has its own allocator which hands out 8kb chunks. These may need similar support. Mac (and android) don't use Chrome's TCMalloc I believe.
Alexei Filippov
Comment 21 2012-12-04 17:22:46 PST
Comment on attachment 177580 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=177580&action=review >> Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:77 >> + components.append(ComponentInfo("MallocWaste", size)); > > Chrome on Mac also uses FastMalloc (aka an older TCMalloc) down in WTF. Also I think libpng has its own allocator which hands out 8kb chunks. These may need similar support. Mac (and android) don't use Chrome's TCMalloc I believe. Eric, thanks for the info. The webkit part is intended to be platform independent, so these details should go under memoryAllocatorWasteInBytes implementation. At some point I may break down this malloc waste into subcomponents like libpng stuff, free lists, etc. But I'm not sure if it adds a value at the moment. I agree the TCMalloc comment is a bit misleading. I'll change it.
Alexei Filippov
Comment 22 2012-12-04 17:32:52 PST
Eric Seidel (no email)
Comment 23 2012-12-04 17:48:31 PST
Comment on attachment 177620 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=177620&action=review > Source/WebCore/platform/chromium/MemoryUsageSupportChromium.cpp:75 > + // Memory allocator waste reporting. I don't think these comments really add anything. :) The code is pretty self-documenting. Could you link to the Chrome-side patch? I worry you may not be able to implement the wtf-fastMalloc part (for Chrome Mac) in the chrome-side.
Alexei Filippov
Comment 24 2012-12-10 01:42:28 PST
The link is in the comment #6, but I can repeat it ;-) https://chromiumcodereview.appspot.com/10823205/
Eric Seidel (no email)
Comment 25 2012-12-10 01:46:35 PST
(In reply to comment #23) > (From update of attachment 177620 [details]) > View in context: https://bugs.webkit.org/attachment.cgi?id=177620&action=review > > Could you link to the Chrome-side patch? I worry you may not be able to implement the wtf-fastMalloc part (for Chrome Mac) in the chrome-side. Thanks. :) But I still worry. I don't believe it's possible to gather all the information from the Chrome side. That assumes that the embedder always controls the allocator, which is not the case. :)
Alexei Filippov
Comment 26 2012-12-10 03:48:09 PST
Yes, the work on allocators waste is not over yet. This patch gives the embedder opportunity to report its allocator stats. I believe wtf-fastMalloc is a bit different story, so I'd like to have it as a separate change.
Alexei Filippov
Comment 27 2012-12-12 04:01:45 PST
Alexei Filippov
Comment 28 2012-12-12 04:03:12 PST
Removed the comments.
WebKit Review Bot
Comment 29 2012-12-12 05:33:48 PST
Comment on attachment 179019 [details] Patch Clearing flags on attachment: 179019 Committed r137457: <http://trac.webkit.org/changeset/137457>
WebKit Review Bot
Comment 30 2012-12-12 05:33:54 PST
All reviewed patches have been landed. Closing bug.
Note You need to log in before you can comment on or make changes to this bug.