Bug 227128 - Build consumes too much memory on 8gb M1
Summary: Build consumes too much memory on 8gb M1
Status: RESOLVED WONTFIX
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Local Build
Hardware: Mac (Apple Silicon) macOS 11
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-06-17 08:53 PDT by Kimmo Kinnunen
Modified: 2022-05-02 22:56 PDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kimmo Kinnunen 2021-06-17 08:53:26 PDT
Build should take into account the number of performance cores, number of efficiency cores and memory.
My M1 has 8gb and seems to build with 8 clang processes that take 1gb each. This causes swapping and cpu is being used in kernel_task instead of clang..
Comment 1 Radar WebKit Bug Importer 2021-06-24 08:54:23 PDT
<rdar://problem/79726938>
Comment 2 Elliott Williams 2022-05-02 16:51:55 PDT
I'm not sure if there is much we can do about the underlying issue of how heavy our compile tasks are.

In my most recent testing, on a 20-core Mac Studio, the system peaked at 32GB memory usage during a build. For comparison, on a 24-core / 16 GB linux system I recently built WebKitGTK on, I had to limit the number of concurrent jobs to 14 to avoid running out of memory.

On Apple platforms, we can control the amount of parallelism, but only (easily) from build scripts. I'm not convinced we should do this project-wide, because:

- On systems like yours, building from Make would magically be faster than building the same scheme/targets in Xcode.
- Less parallelism would be available for non-memory-intensive parts of the build, like copying files and generating code.


That said, you can customize it yourself via defaults:

    defaults write com.apple.dt.Xcode IDEBuildOperationMaxNumberOfConcurrentCompileTasks 6

or

    make ... ARGS="-IDEBuildOperationMaxNumberOfConcurrentCompileTasks 6"
Comment 3 Kimmo Kinnunen 2022-05-02 22:56:50 PDT
Yeah, it's mostly not a real problem..
I was thinking the realistic solution would be to set IDEBuildOperationMaxNumberOfConcurrentCompileTasks as something like min(#cores, #GBs - 2) but if Xcode doesn't support it it's unrealistic to get it done.