Bug 21272 – Switching between custom allocators (TCmalloc, JEmalloc) in linux-qt

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Zoltan Horvath]

We modified build system of WebKit under linux-qt to use TCmalloc if “--tcmalloc” parameter is given to build-webkit script. Even more, we integrated JEmalloc into WebKit as well which is used if "--jemalloc" parameter is given to build-webkit script. (In Mozilla the default allocator is JEmalloc.) 
If we do not give any of these parameters to the build these libraries are not used.

There are three patches:
 (enable_tcmalloc.patch) enables TCmalloc in WebKit (without JEmalloc)
 (enable_jemalloc.patch JEmalloc.h.patch JEmalloc.c.patch rb.h.patch) adds JEmalloc to WebKit (without enabling TCmalloc)
 (enable_jetcmalloc.patch JEmalloc.h.patch JEmalloc.c.patch rb.h.patch) enables TCmalloc and adds JEmalloc at the same time (in this case the user can switch between the allocators in compile time)

Since under linux-qt global operator new and delete do not invoke allocators of TC/JEmalloc, therefore only fastMalloc invokes them. On the other hand, the experimental patches of bug #20422 (https://bugs.webkit.org/show_bug.cgi?id=20422) gives a solution where new and delete also invoke custom allocators (TC/JEmalloc).

[Zoltan Horvath]

Created attachment 23974 [details]
Enable TCmalloc under linux-qt (enable_tcmalloc.patch)

[Zoltan Horvath]

Created attachment 23975 [details]
Enable JEmalloc under linux-qt (enable_jemalloc.patch)

[Zoltan Horvath]

Created attachment 23976 [details]
JEmalloc.h (JEmalloc.h.patch)

[Zoltan Horvath]

Created attachment 23977 [details]
JEmalloc.c (JEmalloc.c.patch)

[Zoltan Horvath]

Created attachment 23978 [details]
rb.h (red-black tree for JEmalloc)

[Zoltan Horvath]

Created attachment 23979 [details]
Enable TC/JEmalloc under linux-qt (enable_jetcmalloc.patch)

[Zoltan Horvath]

*** Bug 21273 has been marked as a duplicate of this bug. ***

[Mark Rowe (bdash)]

If you're going to submit a patch, please do it as a single patch rather than attaching each file independently.

[Mark Rowe (bdash)]

What is the purpose of adding jemalloc to WebKit?  What benefits does it bring?

We should not be adding an additional memory allocator to the WebKit source tree.  We can evaluate whether jemalloc would perform better than our custom TCMalloc, and if it does we can consider replacing TCMalloc with it.  If it doesn't, I see no reason to add it to our tree, and many reasons not to.

[Mark Rowe (bdash)]

Three of the patches look like they do conflicting things. Clearing review flags, since it's not clear that they're ready for review.

[Zoltan Horvath]

The patches do not conflict but the comment was not clear enough.

If you want to use TCmalloc without JEmalloc then use the enable_tcmalloc.patch patch file only and the others are unnecessary in this case (do not use them).

If you want to use JEmalloc without TCmalloc then use enable_jemalloc.patch, JEmalloc.h.patch, JEmalloc.c.patch, rb.h.patch patch files and the others are unnecessary in this case (do not use them).

If you want to use JEmalloc and TCmalloc together use the enable_jetcmalloc.patch, JEmalloc.h.patch, JEmalloc.c.patch, rb.h.patch patch files only and the are unnecessary in this case (do not use them).

Since the TCmalloc and JEmalloc patches conflict, therefore we created them separately (the first two cases) and we created a third one which contains both of them.

I think the possibility of changing between different memory allocators would be a useful feature (even in embedded systems).

[Mark Rowe (bdash)]

You flagged all of the patches for review but, as you say, three of them are mutually exclusive. Which are you proposing that we review?

[Mark Rowe (bdash)]

Further, what is it that you're proposing with these patches? That we drop TCMalloc in favour of  jemalloc? That we add code for a second memory allocator, along side our existing TCMalloc? It would be great if you could clarify this. 

[Zoltan Horvath]

You don’t need to review these patches because we have been improving them and we will send exactly one as soon as we achieve better results with the custom allocators.

[Kwang Yul Seo]

I can think of one reason why jemalloc is a good alternative to tcmalloc. 

jemalloc is more portable. While tcmalloc can't be used on Windows CE, jemalloc is ported to Windows CE by Mozilla. Fennec uses jemalloc and it defeats the 32MB virtual memory space limitation of Windows CE.

[Ismail Donmez]

Any news about this patch? This is truly needed for WinCE ports.

[Zoltan Horvath]

Is the switching needed or the JEmalloc implementation is needed?

I'm working on the custom allocation framework now what is needed to the switching allocators for the whole WebKit. If you want to test JavaScriptCore with other allocators it is possible already.

If you have further questions, please ask. :)

[Kwang Yul Seo]

(In reply to comment #17)
> Is the switching needed or the JEmalloc implementation is needed?
> 
> I'm working on the custom allocation framework now what is needed to the
> switching allocators for the whole WebKit. If you want to test JavaScriptCore
> with other allocators it is possible already.
> 
> If you have further questions, please ask. :)

What's the current position of WebKit on switching the memory allocator? As described in your blog article "War of allocators in JavaScriptCore", adding jemalloc makes no sense for Linux-qt build. However, in Windows CE, jemalloc gives almost 2x speedup in page loading time over the system malloc because the system malloc is simply too slow.  

Is it okay to use jemalloc at least for Windows CE? If the answer is yes, I will file a bug for this.

[Zoltan Horvath]

(In reply to comment #18)
> What's the current position of WebKit on switching the memory allocator? As
> described in your blog article "War of allocators in JavaScriptCore", adding
> jemalloc makes no sense for Linux-qt build. However, in Windows CE, jemalloc
> gives almost 2x speedup in page loading time over the system malloc because the
> system malloc is simply too slow.  
> 
> Is it okay to use jemalloc at least for Windows CE? If the answer is yes, I
> will file a bug for this.

Custom allocation framework is ready (only few (<5) classes aren't inherited from FastAllocBase), so you can safely use custom allocators - trough FastMalloc - without overriding global operator new/delete.
Now, I'm benchmarking JEmalloc on multi-threaded benchmarks,  tomorrow I'll write a post about the results. 

First, I think we should make TCmalloc's implementation WinCE compatible, and test that. Adding and maintaining an extra allocator - because of one platform - is not a good idea.

What are the depedencies of enabling TCmalloc on WinCe? TCmalloc uses pthreads... This might be a problem, isn't it?

[Kwang Yul Seo]

(In reply to comment #19)
> Custom allocation framework is ready (only few (<5) classes aren't inherited
> from FastAllocBase), so you can safely use custom allocators - trough
> FastMalloc - without overriding global operator new/delete.
> Now, I'm benchmarking JEmalloc on multi-threaded benchmarks,  tomorrow I'll
> write a post about the results. 
> 
> First, I think we should make TCmalloc's implementation WinCE compatible, and
> test that. Adding and maintaining an extra allocator - because of one platform
> - is not a good idea.
> 
> What are the depedencies of enabling TCmalloc on WinCe? TCmalloc uses
> pthreads... This might be a problem, isn't it?

I agree with you. TCmalloc needs to be ported to WinCE before we consider adding an extra allocator.

Implementing threading and spin lock for WinCE ARM is one thing, but the big obstacle is the WinCE's virtual memory space limitation. WinCE is unique as it allows only 32MB virtual memory space to each process. Windows CE 6.0 Embedded removed the limitation though.

Check out the memory map of Windows Mobile 6.1 (based on Windows CE 5.0)

http://bolingconsulting.com/blog/?p=4

Fortunately, VirtualAlloc(size > 2MB) allocates the memory outside the 32MB process virtual memory space. It allocates memory in Large Memory Area(LBA). TCmalloc must be modified to address this issue.

Mozilla's Fennec uses jemalloc to overcome this virtual memory space issue as jemalloc is ported to WinCE to circumvent the limitation.

My colleague is currently working on this issue. He will be able to submit a patch soon or later.

[Ismail Donmez]

(In reply to comment #20)
> Mozilla's Fennec uses jemalloc to overcome this virtual memory space issue as
> jemalloc is ported to WinCE to circumvent the limitation.
> 
> My colleague is currently working on this issue. He will be able to submit a
> patch soon or later.

Using jemalloc is fine but once you have mismatched allocators i.e stuff allocated with jemalloc and freed by CRT's free/delete you are gonna end up crashing.

It will be tough road imho.

[Zoltan Horvath]

In the past few weeks, I wrote some blog posts about results of custom allocators:
http://webkit.sed.hu/taxonomy/term/3

[Zoltan Horvath]

Created attachment 50881 [details]
Switching between system allocator and TCmalloc in build-webkit

[Zoltan Horvath]

Created attachment 50882 [details]
Switching between system allocator and TCmalloc in build-webkit

[Oliver Hunt]

Comment on attachment 50882 [details]
Switching between system allocator and TCmalloc in build-webkit

r=me

[Zoltan Horvath]

Comment on attachment 50882 [details]
Switching between system allocator and TCmalloc in build-webkit

Clearing flags on attachment: 50882

Committed r56227: <http://trac.webkit.org/changeset/56227>

[Zoltan Horvath]

All reviewed patches have been landed.  Closing bug.