Bug 106304 – [Qt] High memory consumption and memory growth in Qt5's WebKit compared to Qt4.8

• Status: RESOLVED
• Resolution: INVALID
• Component: WebKit Qt

Created attachment 181655 [details]
ZIP file with test program and test page

Visual Studio 2010 and Windows 7 64bit (as well as Windows XP 32bit)

The initial memory consumption is 3-4 times higher in Qt5's QtWebKit compared to Qt4.8
The memory growth when using the JS bridge in Qt5 is 300% while in Qt4.8 it's 20%
--> that's an overwhelming difference!

Showing initially a simple html page with only a few buttons takes
7400 KB in Qt4.8
28300 KB in Qt5
(Initial memory consumption, without any interaction, just starting a simple browser and showing the page)

Sending some data (sending a few times 100000 strings in a loop) about the JS bridge to a C++ Slot lets the memory grow up to
8900 KB in Qt4.8
83800 KB in Qt5 (wow!)

How to retest:
I attached a simple test program with a html page. You can translate it with Qt5 as well as Qt4.8 (see the readme.txt). I also included the exe files.

Some details:
The memory numbers above are average numbers from five tests for each Qt version. The memory consumption variation was very stable from test to test (maybe 10% up and down), so the average numbers above are very realistic.

To see the memory growth just hit the first button in the html page. With each click you will send 100000 strings (in a loop) to a C++ slot that expects a QString. 

In Windows 7 you can display different memory sets with the Task Manager and with with the Resource Monitor. I measured the private working set ('privater Arbeitssatz' in my German version), because that's the real deal.

Of course the html page is simple and the inital _absolute_ difference between the Qt versions is only 20900 KB. However we have the same _relative_ difference also with our real application and some more sophisticated pages (our _real_ numbers show a initial memory consumption of 45 MB in Qt4.8 and 190 MB in Qt 5. This is a stopper for us to go on with Qt5.

[Sergio Villar Senin]

I guess you're testing the exactly same version of webkit built against two different qt versions.

I have a couple of questions:
   * have you tested it just in the windows platform?
   * how do you measure memory consumption?

[Sergio Villar Senin]

(In reply to comment #1)
> I guess you're testing the exactly same version of webkit built against two different qt versions.
> 
> I have a couple of questions:
>    * have you tested it just in the windows platform?
>    * how do you measure memory consumption?

Also, you're just testing WK1 right?

> I guess you're testing the exactly same version of webkit built against two different qt versions.

1) I do not know which underlying WebKit version is used by QtWebKit in both cases. But I tested QtWebKit 5.0.0 (shipped with Qt5) and I tested QtWebKit 2.2.3 (shipped with Qt 4.8.4)


> have you tested it just in the windows platform?

2) Yes, just as written in my post (Win7 and WinXP)


> how do you measure memory consumption?

3) How I measure it, is written in my post.


> Also, you're just testing WK1 right?

4) Yes. We have a running prototype in Qt 4.8.4 and we can not afford to switch to WK2.

[Michael Brüning]

I found the test application to also incur a higher memory use on Linux with Qt5 than on Qt 4.8.4 (tested with valgrind/massif). This is most likely due to the JS-QObject bridge now being based on the JSC C API and some additional copies in that implementation.

I found the most siginificant spikes in memory usage when using the QString test case and I suspect the changed implementation when converting strings in convertValueToQVariant to be the culprit.

I will see what we can do to get this fixed or at least lower memory usage for this.

(In reply to comment #4)
> I found the test application to also incur a higher memory use on Linux with Qt5 than on Qt 4.8.4 (tested with valgrind/massif). This is most likely due to the JS-QObject bridge now being based on the JSC C API and some additional copies in that implementation.
> 
> I found the most siginificant spikes in memory usage when using the QString test case and I suspect the changed implementation when converting strings in convertValueToQVariant to be the culprit.
> 
> I will see what we can do to get this fixed or at least lower memory usage for this.

Thank you for taking a look. However, it's also very strange that the initial memory usage is 3 times higher, even is you do not use the JS-QObject bridge. You can even out-comment the whole JS-QObject bridge stuff and run the program: the initial memory usage is still 3 times higher.
(In my test program the absolut numbers are not so high, but in a real application we have a view which initially takes 60 MB with Qt4 and initially approx. 200 MB in Qt5!)

[Michael Brüning]

I have looked into it a bit more and while I am not able to explain the higher base memory consumption yet, the memory growth (caused by GC not being run) should be mitigated by the patches for the following two bugs:

https://bugs.webkit.org/show_bug.cgi?id=103996

https://bugs.webkit.org/show_bug.cgi?id=103998

Those are included in the current stable branch of the qtwebkit module from the Qt Project, but I will submit them for the release branch as well.

Michael, thank you! I can confirm that the memory growth is no longer an issue. If I now use the JS-QObject bridge the memory goes up high, but it immediately drops down to its (more or less) original value - so the garbage collector seems to work.
As you stated, the initial memory consumption is still high compared to Qt4.8. Shall I write a new issue for that with a simplified test program?