Bug 40111

Summary: [Qt] Startup memory consumption is almost doubled when comparing qtwebkit-2.0 against QtWebKit in Qt 4.6
Product: WebKit Reporter: nokiabugz
Component: New BugsAssignee: QtWebKit Unassigned <webkit-qt-unassigned>
Status: RESOLVED WORKSFORME    
Severity: Normal CC: alexandra.1.santos, benjamin, ext-ananth.jasty, hao.1.lu, hausmann, jesus, jukka.jokiniva, kenneth, kling, koshuin, laszlo.gombos, zoltan
Priority: P3 Keywords: Qt, QtTriaged
Version: 528+ (Nightly build)   
Hardware: S60 Hardware   
OS: S60 3rd edition   
Bug Depends on: 41278    
Bug Blocks: 35784    

Description nokiabugz 2010-06-03 04:14:24 PDT
Our latest daily BAT r34916 shows that the startup memory is 12.28MB. But r34916+webkit 2.0 consumes 21.46MB!!! More than 9MB consumed. This will impact the stability of the browser.
Comment 1 Simon Hausmann 2010-06-04 14:27:34 PDT
So this is the difference between WebKit that comes with Qt 4.6 and QtWebKit 2.0?
Comment 2 Laszlo Gombos 2010-06-08 10:53:57 PDT
I think we need answer to Simon's question here to proceed. How are these r numbers related to QtWebKit releases ?
Comment 3 Kenneth Rohde Christiansen 2010-06-08 11:10:35 PDT
Apparently they are related to the non-Qt based Nokia WebKit browser.
Comment 4 nokiabugz 2010-06-09 04:32:09 PDT
Yes, Simon's assumption is correct: the doubling of memory is the difference we see when we replace Qt 4.6's WebKit with QtWebKit 2.0.  This type of memory consumption has been observed on two different Browser versions.
Comment 5 Simon Hausmann 2010-06-16 23:12:17 PDT
*** Bug 40070 has been marked as a duplicate of this bug. ***
Comment 6 Simon Hausmann 2010-06-16 23:14:25 PDT
Andreas and Benjamin have done some changes in Qt to reduce the memory usage and there's more in the pipeline. I hope to have a Qt 4.6 sis file next week that we can use for re-testing this situation.
Comment 7 Kenneth Rohde Christiansen 2010-06-17 19:03:23 PDT
What are you comparing with? QGraphicsWebView? qtwebkit2.0 with tiling? It would be nice to know the setup or see the test application.
Comment 8 Jukka Jokiniva 2010-06-18 02:29:03 PDT
I think this can tested with following steps.
Precondition: Symbian Phone with RD software and Qt4.6.3 installed
1. start "MemMon" tool from "RD Tools" menu, check the available free memory and leave it running on the background
2. Start the QtTestBrowser and load a test page. Switch back to "MemMon" and check the free memory. Do the math how much more memory is being used.
3. Quit the Browser
4. Update to QtWebKit 2.0. Switch to "MemMon" & check the free memory.
5. Start the QtTestBrowser and load the same test page. Switch back to "MemMon" and check the free memory. Do the math.
This is quite rough method but should reveal if the memory consumption is double.
Comment 9 Janne Koskinen 2010-06-24 00:26:55 PDT
Checklist to go through in this order:

-bytepair vs. deflate. are the new binaries you are testing bytepair compressed?
This would double the memory consumption easily if whole DLL has to be loaded instead of just needed pages. This would instantly add that 10MB.

-The new block allocator
We create new heap with large reserve. This is intentional behaviour and not to be considered a bug. Should not show as system wide memory consumption though but seen funnier things happening :)

-did the compilation end up as debug instead of release binary?
I know sounds weird but I've noticed a bug that we end up adding debugging info even when doing release builds.
Doesn't add up to ~10MB.

- and finally.. something actually caused regression.
Comment 10 ananth jasty 2010-06-24 15:19:30 PDT
The extraordinary amount of memory allocated makes me wonder if some of this is pre-reserved pages as per the new javascript and webcore heap allocators.

Assuming this is before any pages are opened, do we know if this memory is reserved or allocated?
Comment 11 Simon Hausmann 2010-06-29 02:09:53 PDT
Ok, here are some measurements:

Device: N8

Application: QtTestBrowser using a plain QWebView

Qt 4.6 - 0b618c7739c904ff35572886dbded9113a517dc0

Website: news.google.com

Free ram: ~134 MB

Free ram with QtWebKit from Qt 4.6: ~120MB

Free ram with QtWebkit 2.0 (38d650efc92427cd6452f6685d3c40d22428cdb7) : ~116 MB

-> 4 MB more memory is used.
Comment 12 Janne Koskinen 2010-06-29 03:46:38 PDT
I received 3 QtWebkit.dll dumps for comparison and all 3 were different in .code size and supposedly were built from the same source.

First, all builds are paged and thus bytepair compressed. You can cross that out from checklist.

Difference in largest and smallest build of QtWebkit 2.0 that I received had 3.5MB difference smallest being 8.5MB and largest 12MB

I bet largest build is statically linking against sqlite3.lib which has been an issue before. Search .mmp file for staticlibrary statement.
Difference between the middle sized 10.5MB and small 8.5MB is most likely -O3 and -OTime.

These differences could explain 4MB vs. 9MB increase in consumption. Or even it could explain it completely as 4.6.2 release was built using thumb. That is if 4.6 webkit in the report is 4.6.2 QtWebkit and not 4.6.3.
Comment 13 Simon Hausmann 2010-06-29 05:50:44 PDT
(In reply to comment #12)
> These differences could explain 4MB vs. 9MB increase in consumption. Or even it could explain it completely as 4.6.2 release was built using thumb. That is if 4.6 webkit in the report is 4.6.2 QtWebkit and not 4.6.3.

This sounds like a likely culprit!

I'm preparing a patch to build JavaScriptCore as arm and the rest of WebKit in thumb mode.
Comment 14 Simon Hausmann 2010-06-29 05:56:18 PDT
Kent had another suggestion: It might be that the one or other magic constant in JSC changed with regards to the default allocation of various buffers, between when Qt 4.6's WebKit was branched and now.
Comment 15 Simon Hausmann 2010-06-29 06:19:40 PDT
Great, compiling WebCore with thumb I get:

 FAILED compile for armv5_urel:  WebCore\generated\HTMLNames.cpp
  mmp: WebCore\WebCore_0x200267C2.mmp
   Internal fault: [0x040b:220686] in _ZN7WebCore9HTMLNames4initEv
   Please contact your supplier.
Comment 16 Simon Hausmann 2010-06-29 06:40:19 PDT
(In reply to comment #15)
> Great, compiling WebCore with thumb I get:
> 
>  FAILED compile for armv5_urel:  WebCore\generated\HTMLNames.cpp
>   mmp: WebCore\WebCore_0x200267C2.mmp
>    Internal fault: [0x040b:220686] in _ZN7WebCore9HTMLNames4initEv
>    Please contact your supplier.

Tracking this issue now in Bug #41349
Comment 17 Simon Hausmann 2010-06-29 07:04:49 PDT
(In reply to comment #11)
> Ok, here are some measurements:
> 
> Device: N8
> 
> Application: QtTestBrowser using a plain QWebView
> 
> Qt 4.6 - 0b618c7739c904ff35572886dbded9113a517dc0
> 
> Website: news.google.com
> 
> Free ram: ~134 MB
> 
> Free ram with QtWebKit from Qt 4.6: ~120MB
> 
> Free ram with QtWebkit 2.0 (38d650efc92427cd6452f6685d3c40d22428cdb7) : ~116 MB
> 
> -> 4 MB more memory is used.

Update on these numbers!

When QtWebKit as part of Qt 4.6 is built with arm instead of thumb - which is how it's shipped on the N8, then the memory consumption goes up by 2 MB, reducing the gap from 4 MB to 2 MB between Qt 4.6's WebKit and QtWebKit 2.0.

That means taking away build differences, it appears that QtWebKit 2.0 allocates two more megabytes of RAM in this setup with this site.
Comment 18 Kenneth Rohde Christiansen 2010-06-30 00:14:36 PDT
(In reply to comment #17)
> (In reply to comment #11)
> > Ok, here are some measurements:
> > 
> > Device: N8
> > 
> > Application: QtTestBrowser using a plain QWebView
> > 
> > Qt 4.6 - 0b618c7739c904ff35572886dbded9113a517dc0
> > 
> > Website: news.google.com
> > 
> > Free ram: ~134 MB
> > 
> > Free ram with QtWebKit from Qt 4.6: ~120MB
> > 
> > Free ram with QtWebkit 2.0 (38d650efc92427cd6452f6685d3c40d22428cdb7) : ~116 MB
> > 
> > -> 4 MB more memory is used.
> 
> Update on these numbers!

Have you tried with trunk to see how that compares?
Comment 19 Simon Hausmann 2010-06-30 01:58:17 PDT
(In reply to comment #18)
> (In reply to comment #17)
> > (In reply to comment #11)
> > > Ok, here are some measurements:
> > > 
> > > Device: N8
> > > 
> > > Application: QtTestBrowser using a plain QWebView
> > > 
> > > Qt 4.6 - 0b618c7739c904ff35572886dbded9113a517dc0
> > > 
> > > Website: news.google.com
> > > 
> > > Free ram: ~134 MB
> > > 
> > > Free ram with QtWebKit from Qt 4.6: ~120MB
> > > 
> > > Free ram with QtWebkit 2.0 (38d650efc92427cd6452f6685d3c40d22428cdb7) : ~116 MB
> > > 
> > > -> 4 MB more memory is used.
> > 
> > Update on these numbers!
> 
> Have you tried with trunk to see how that compares?

Yep, with the trunk there's about 112MB RAM left after loading, which is even worse.
Comment 20 Hao Lu 2010-06-30 07:47:51 PDT
The original reported bug https://bsetpl02.americas.nokia.com/browse/BR-3449 in Jira is no longer reproducible.
Comment 21 Jukka Jokiniva 2010-07-01 00:29:26 PDT
According to the reporter not repro any more => closed.