Followup to https://bugs.webkit.org/show_bug.cgi?id=53352 and https://bugs.webkit.org/show_bug.cgi?id=53401. In WebkitGtk 1.4.0 (which was recently commited to OpenBSD's portstree), on OpenBSD/amd64 at startup webkit browsers preallocate 1Gb of ram, which is more than the default value for ulimit (512mb), so they quickly explode. #if CPU(ARM) typedef PageTables16MB FixedVMPoolPageTables; #elif CPU(X86_64) typedef PageTables1GB FixedVMPoolPageTables; #else typedef PageTables32MB FixedVMPoolPageTables; #endif From my understanding of the comments in bug 53352 (i might be wrong), the 1Gb allocation on X86_64 is done to allow huge regressions tests to pass fine on linux/x86_64. While i won't debate on that particular point, could it be made to be linux-only, so that we don't have to patch that code to remove that insane greedy allocation ? Ie as of now, we apply http://www.openbsd.org/cgi-bin/cvsweb/ports/www/webkit/patches/patch-Source_JavaScriptCore_jit_ExecutableAllocatorFixedVMPool_cpp?rev=1.1 to workaround that. Maybe it could be commited as #elif CPU(X86_64) && OS(LINUX), so that defaults are more sane ?
> From my understanding of the comments in bug 53352 (i might be wrong), the 1Gb allocation on X86_64 is done to allow huge regressions tests to pass fine on linux/x86_64. No, this does not relate specifically to huge regression tests, or to Linux. We allocate all VM space for JIT code up front, such that we can ensure locality in the VM space. > While i won't debate on that particular point, could it be made to be linux-only, so that we don't have to patch that code to remove that insane greedy allocation ? 1GB is less than 1% of 1% of the process's available VM, which can hardly be described as insanely greedy. :-) The key here is that the allocation should be of VM space only. OS X & Windows support lazy commit of memory, so this isn't a problem. On Linux, the solution to get the right behaviour was landed in this bug: https://bugs.webkit.org/show_bug.cgi?id=42756 , passing the MAP_NORESERVE flag to mmap. If OpenBSD supports this flag, this is likely the right fix for you, too. If not, you need to find a mechanism to allocate VM without committing the physical pages up front. If you wish to submit changes back to trunk, please attach a patch, instructions here: http://www.webkit.org/coding/contributing.html
Retitling to make the problem more explicit; do we have anyone maintaining the BSD port?
(In reply to comment #2) > Retitling to make the problem more explicit; do we have anyone maintaining the BSD port? I doubt you have a webkit dev working specifically on BSD, but you have people who use it there (and probably more than you think), be it on NetBSD, OpenBSD, FreeBSD or DragonflyBSD, and maintainers for the corresponding ports. So please consider it when making changes that might impact us :)
https://bugs.webkit.org/show_bug.cgi?id=65766 may be related
Looks like Linux has a fix for this, might be worth considering whether this would work for BSD too. See https://bugs.webkit.org/show_bug.cgi?id=65766
(In reply to comment #5) > Looks like Linux has a fix for this, might be worth considering whether this would work for BSD too. See https://bugs.webkit.org/show_bug.cgi?id=65766 As i said on that bug report, there's no way for us to allocate memory without actually having it reserved/written by the OS. So that 'fix' won't apply (at least on OpenBSD, dunno for the others). We have a low ulimit -m to 512Mb by default on purpose, so maybe only OpenBSD is affected by that problem because of that 1Gb allocation.
And i still don't understand why X86_64 allocates 1Gb, while other archs (i386, powerpc....) will allocate 32Mb. Why is that change X86_64 only, if not for the failing regress tests in bug 53401 ?
(In reply to comment #7) > And i still don't understand why X86_64 allocates 1Gb, while other archs (i386, powerpc....) will allocate 32Mb. Why is that change X86_64 only, if not for the failing regress tests in bug 53401 ? The fixed allocator is only used on X86-64, and when running on iOS. On all other CPUs and platforms we use the on-demand allocator. The on-demand allocator allocates memory at the point it needs it, and frees it when it is done. The on-demand allocator may allocate memory anywhere in the VM space, and this is a problem for x86-64, since relative branches have a range of +/- 2Gb. If the on-demand allocator were to allocate two blocks of memory for use by the JIT more than 2Gb apart, linking could fail. This is why instead we perform a single VM allocation, and commit/decommit subregions, so we can ensure a degree of locality within VM space. The 1Gb size is intended to be more than one might possibly ever need for JIT code, whilst still a minuscule fraction of the x86-64 address space. You are certainly welcome to drop this cap on OpenBSD, but I wouldn't recommend this as a solution, since the JIT isn't the only place we use this model of eagerly allocating VM space & commiting subregions on demand (e.g. we do something similar for the JS program's stack). I'd recommend that finding a way to decouple memory commit from VM allocation (e.g. Matthew Dempsky's proposal in bug #61137) is going to be a much more satisfactory solution for you in the long run.
> The on-demand allocator may allocate memory anywhere in the VM space, and this is a problem for x86-64, since relative branches have a range of +/- 2Gb. If the on-demand allocator were to allocate two blocks of memory for use by the JIT more than 2Gb apart, linking could fail. So on OpenBSD, for this reason we actually guarantee that if you use a hinted mmap(), the returned address will be within +/-2GB of the address. E.g., if p != NULL, then mmap(p, ...) will return an address between p - 2GB and p + 2GB. Would it be possible to modify the on-demand allocator to take advantage of this property on OpenBSD, and to use it on OpenBSD/x86-64 instead of the fixed allocator?