Bug 171944 - [iOS] Use memory footprint to dynamically adjust behavior of allocators
Summary: [iOS] Use memory footprint to dynamically adjust behavior of allocators
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Michael Saboff
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2017-05-10 13:54 PDT by Michael Saboff
Modified: 2017-05-15 09:41 PDT (History)
11 users (show)

See Also:


Attachments
Almost done patch for review (16.82 KB, patch)
2017-05-10 14:45 PDT, Michael Saboff
fpizlo: review+
Details | Formatted Diff | Diff
Updated patch - should fix build issues. (27.36 KB, patch)
2017-05-11 11:15 PDT, Michael Saboff
no flags Details | Formatted Diff | Diff
Patch with another build fix (27.36 KB, patch)
2017-05-11 11:28 PDT, Michael Saboff
no flags Details | Formatted Diff | Diff
Another build fix for EWS to try (28.73 KB, patch)
2017-05-11 12:10 PDT, Michael Saboff
no flags Details | Formatted Diff | Diff
Another build fix. Moved AvailableMemory files to stdlib and added the .cpp to CMakeLists.txt (29.32 KB, patch)
2017-05-11 12:35 PDT, Michael Saboff
no flags Details | Formatted Diff | Diff
Hopefully last EWS patch - Moved AvailableMemory.{cpp, h} into main bmalloc directory. (29.25 KB, patch)
2017-05-11 16:00 PDT, Michael Saboff
no flags Details | Formatted Diff | Diff
Another speculative GTK build fix. Eliminated Darwin conditional inclusion of AvailableMemory.h in bmalloc.h (29.23 KB, patch)
2017-05-11 16:07 PDT, Michael Saboff
no flags Details | Formatted Diff | Diff
Speculative GTK build fix (29.41 KB, patch)
2017-05-11 16:16 PDT, Michael Saboff
no flags Details | Formatted Diff | Diff
Another speculative GTK build fix. Eliminated Darwin conditional inclusion of AvailableMemory.h in bmalloc.h (29.40 KB, patch)
2017-05-11 16:52 PDT, Michael Saboff
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews113 for mac-elcapitan (1.77 MB, application/zip)
2017-05-11 18:48 PDT, Build Bot
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Saboff 2017-05-10 13:54:52 PDT
On iOS, there is a soft memory limit of 840MB for WebContent processes.  When we get close to or above that limit and don’t reduce memory usage it is likely the OS will jetsam the process.  The OS provides memory pressure notifications, but that mechanism is course and may not provide a timely way to modify behavior.

It makes more sense to modify our allocation policies and return free memory back to the OS more agressively as we approach our memory limit.

<rdar://problem/32116376>
Comment 1 Michael Saboff 2017-05-10 14:45:51 PDT
Created attachment 309640 [details]
Almost done patch for review

My only question is how to layer bmalloc and WTF.   Currently bmalloc uses ramSize() from WTF, but I could be convinced to move ramSize to bmalloc.  I'm open to other ideas as well.
Comment 2 Filip Pizlo 2017-05-10 16:08:50 PDT
Comment on attachment 309640 [details]
Almost done patch for review

View in context: https://bugs.webkit.org/attachment.cgi?id=309640&action=review

This looks good to me!

> Source/JavaScriptCore/heap/Heap.cpp:117
> +    size_t memoryFootprint = bmalloc::api::memoryFootprint();

Would be cool if bmalloc::api::memoryFootprint was abstracted in WTF behind fastMallocMemoryFootprint, or something.

Not super important.

> Source/bmalloc/bmalloc/Heap.cpp:45
> +#if BPLATFORM(IOS)
> +namespace WTF {
> +
> +size_t ramSize();
> +
> +}
> +#endif

Would be a lot better if you moved WTF::ramSize into bmalloc, and then WTF::ramSize would be a forwarding function that calls bmalloc::api::ramSize or whatever.
Comment 3 Michael Saboff 2017-05-11 11:15:59 PDT
Created attachment 309728 [details]
Updated patch - should fix build issues.

(In reply to Filip Pizlo from comment #2)
> Comment on attachment 309640 [details]
> Almost done patch for review
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=309640&action=review
> 
> This looks good to me!
> 
> > Source/JavaScriptCore/heap/Heap.cpp:117
> > +    size_t memoryFootprint = bmalloc::api::memoryFootprint();
> 
> Would be cool if bmalloc::api::memoryFootprint was abstracted in WTF behind
> fastMallocMemoryFootprint, or something.
> 
> Not super important.

I'll do this is in a followup patch.

> > Source/bmalloc/bmalloc/Heap.cpp:45
> > +#if BPLATFORM(IOS)
> > +namespace WTF {
> > +
> > +size_t ramSize();
> > +
> > +}
> > +#endif
> 
> Would be a lot better if you moved WTF::ramSize into bmalloc, and then
> WTF::ramSize would be a forwarding function that calls bmalloc::api::ramSize
> or whatever.

Done.
Comment 4 Build Bot 2017-05-11 11:17:41 PDT
Attachment 309728 [details] did not pass style-queue:


ERROR: Source/bmalloc/bmalloc/Heap.cpp:38:  Alphabetical sorting problem.  [build/include_order] [4]
ERROR: Source/bmalloc/bmalloc/darwin/AvailableMemory.cpp:26:  Found header this file implements before WebCore config.h. Should be: config.h, primary header, blank line, and then alphabetically sorted.  [build/include_order] [4]
Total errors found: 2 in 15 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 5 Michael Saboff 2017-05-11 11:28:19 PDT
Created attachment 309731 [details]
Patch with another build fix
Comment 6 Build Bot 2017-05-11 11:29:39 PDT
Attachment 309731 [details] did not pass style-queue:


ERROR: Source/bmalloc/bmalloc/Heap.cpp:38:  Alphabetical sorting problem.  [build/include_order] [4]
ERROR: Source/bmalloc/bmalloc/darwin/AvailableMemory.cpp:26:  Found header this file implements before WebCore config.h. Should be: config.h, primary header, blank line, and then alphabetically sorted.  [build/include_order] [4]
Total errors found: 2 in 15 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 7 Michael Saboff 2017-05-11 12:10:44 PDT
Created attachment 309747 [details]
Another build fix for EWS to try
Comment 8 Build Bot 2017-05-11 12:11:38 PDT
Attachment 309747 [details] did not pass style-queue:


ERROR: Source/bmalloc/bmalloc/Heap.cpp:38:  Alphabetical sorting problem.  [build/include_order] [4]
ERROR: Source/bmalloc/bmalloc/darwin/AvailableMemory.cpp:26:  Found header this file implements before WebCore config.h. Should be: config.h, primary header, blank line, and then alphabetically sorted.  [build/include_order] [4]
Total errors found: 2 in 16 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 9 Michael Saboff 2017-05-11 12:35:33 PDT
Created attachment 309752 [details]
Another build fix.  Moved AvailableMemory files to stdlib and added the .cpp to CMakeLists.txt
Comment 10 Build Bot 2017-05-11 12:36:39 PDT
Attachment 309752 [details] did not pass style-queue:


ERROR: Source/bmalloc/bmalloc/Heap.cpp:38:  Alphabetical sorting problem.  [build/include_order] [4]
ERROR: Source/bmalloc/bmalloc/darwin/AvailableMemory.cpp:26:  Found header this file implements before WebCore config.h. Should be: config.h, primary header, blank line, and then alphabetically sorted.  [build/include_order] [4]
Total errors found: 2 in 17 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 11 Michael Saboff 2017-05-11 16:00:23 PDT
Created attachment 309804 [details]
Hopefully last EWS patch - Moved AvailableMemory.{cpp, h} into main bmalloc directory.
Comment 12 Build Bot 2017-05-11 16:03:13 PDT
Attachment 309804 [details] did not pass style-queue:


ERROR: Source/bmalloc/bmalloc/Heap.cpp:38:  Alphabetical sorting problem.  [build/include_order] [4]
ERROR: Source/bmalloc/bmalloc/AvailableMemory.cpp:26:  Found header this file implements before WebCore config.h. Should be: config.h, primary header, blank line, and then alphabetically sorted.  [build/include_order] [4]
Total errors found: 2 in 17 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 13 Michael Saboff 2017-05-11 16:07:30 PDT
Created attachment 309807 [details]
Another speculative GTK build fix.  Eliminated Darwin conditional inclusion of AvailableMemory.h in bmalloc.h
Comment 14 Build Bot 2017-05-11 16:08:17 PDT
Attachment 309807 [details] did not pass style-queue:


ERROR: Source/bmalloc/bmalloc/Heap.cpp:38:  Alphabetical sorting problem.  [build/include_order] [4]
ERROR: Source/bmalloc/bmalloc/bmalloc.h:30:  Alphabetical sorting problem.  [build/include_order] [4]
ERROR: Source/bmalloc/bmalloc/AvailableMemory.cpp:26:  Found header this file implements before WebCore config.h. Should be: config.h, primary header, blank line, and then alphabetically sorted.  [build/include_order] [4]
Total errors found: 3 in 17 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 15 Michael Saboff 2017-05-11 16:16:26 PDT
Created attachment 309811 [details]
Speculative GTK build fix
Comment 16 Build Bot 2017-05-11 16:18:07 PDT
Attachment 309811 [details] did not pass style-queue:


ERROR: Source/bmalloc/bmalloc/AvailableMemory.cpp:26:  Found header this file implements before WebCore config.h. Should be: config.h, primary header, blank line, and then alphabetically sorted.  [build/include_order] [4]
Total errors found: 1 in 17 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 17 Michael Saboff 2017-05-11 16:52:14 PDT
Created attachment 309823 [details]
Another speculative GTK build fix.  Eliminated Darwin conditional inclusion of AvailableMemory.h in bmalloc.h
Comment 18 Build Bot 2017-05-11 16:56:59 PDT
Attachment 309823 [details] did not pass style-queue:


ERROR: Source/bmalloc/bmalloc/AvailableMemory.cpp:26:  Found header this file implements before WebCore config.h. Should be: config.h, primary header, blank line, and then alphabetically sorted.  [build/include_order] [4]
Total errors found: 1 in 17 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 19 Build Bot 2017-05-11 18:48:13 PDT
Comment on attachment 309823 [details]
Another speculative GTK build fix.  Eliminated Darwin conditional inclusion of AvailableMemory.h in bmalloc.h

Attachment 309823 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/3722299

New failing tests:
fast/workers/dedicated-worker-lifecycle.html
Comment 20 Build Bot 2017-05-11 18:48:15 PDT
Created attachment 309844 [details]
Archive of layout-test-results from ews113 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 21 Mark Lam 2017-05-11 19:19:47 PDT
(In reply to Build Bot from comment #19)
> New failing tests:
> fast/workers/dedicated-worker-lifecycle.html

FYI, this is not your failure.
Comment 22 Michael Saboff 2017-05-12 07:15:10 PDT
Committed r216763: <http://trac.webkit.org/changeset/216763>
Comment 23 Geoffrey Garen 2017-05-13 09:16:36 PDT
This looks like a 1% regression on the MBP and MBA JetStream bots.

I suspect the change to speed up the scavenger is the cause, since the rest of this code shouldn't run on Mac.
Comment 24 Michael Saboff 2017-05-15 09:41:38 PDT
(In reply to Geoffrey Garen from comment #23)
> This looks like a 1% regression on the MBP and MBA JetStream bots.
> 
> I suspect the change to speed up the scavenger is the cause, since the rest
> of this code shouldn't run on Mac.

In addition to the scavenger delay being reduced to 250ms, the scavenger thread QOS was changed from QOS_CLASS_UNSPECIFIED to QOS_CLASS_USER_INITIATED.  I agree though that the likely reason is cutting the delay in half.  I'll check locally and post a followup patch.  Tracked in <https://bugs.webkit.org/show_bug.cgi?id=172124>