Bug 31719 – [Qt] Loading of large page can take up to 10 seconds

• Status: CLOSED
• Resolution: FIXED
• Component: Page Loading

[Jocelyn Turcotte]

A problem have been reported that loading a large page can take up to 10 seconds in the demo browser, keeping the application busy and unresponsive.
The page complains about scripts problems.

This has been observed on Windows x64

[Antonio Gomes]

jocelyn any URL (testcase) has been reported on this ?

[Eike Ziller]

Created attachment 49893 [details]
Open this in your browser

Open the attached file in Safari, Chrome, Firefox: basically no time.
Open the attached file in the Qt Demo browser (Qt Webkit): in the range of 20 seconds.

This is not a Windows 7 64bit special thing. It's the same on other Windows versions and on the Mac, at the very least, and I've heard complains from Linux users too.

[Eike Ziller]

Created attachment 49894 [details]
Open this in your browser, real life url

This is a real life example (online QString documentation) where it also takes several seconds to show the page in QtWebkit, where the other browsers take "basically no time".

[Jocelyn Turcotte]

Comment on attachment 49893 [details]
Open this in your browser

This attachment reproduce a different bug related to large plain text files. See bug #20054

Obsoleting the attachment to prevent confusion.

[Jocelyn Turcotte]

(In reply to comment #3)
> Created an attachment (id=49894) [details]
> Open this in your browser, real life url
> 
> This is a real life example (online QString documentation) where it also takes
> several seconds to show the page in QtWebkit, where the other browsers take
> "basically no time".

This bug probably require specific conditions to be reproduced, the QString
documentation loads instantly on the setup I have seen.
Since you have the golden conditions, could you provide this information?

- Do you use a release build? If no could you try downloading the latest Qt
version and try in the demo browser to see if you have a major improvement?

- There is a problem with the proxy detection mechanism on Windows where it
will try to detect the proxy on each HTTP request. Could you disable
"Automatically detect settings" in ControlPanel/InternetOptions/Connections/LANSettings and try again?

- Do you have any specific settings that could allow us to reproduce the bug?
You might be able to see what happen a bit more by looking at the web inspector's resource tracking tab or by looking at Wireshark while loading the page.

[Eike Ziller]

(In reply to comment #4)
> (From update of attachment 49893 [details])
> This attachment reproduce a different bug related to large plain text files.
> See bug #20054
> 
> Obsoleting the attachment to prevent confusion.

Now, that becomes confusing to me.

Obviously "Page Loading" doesn't include rendering the page for you... which is most noticable slow for me.

Still, regarding "page loading", i.e. fetching the stuff. "Resources" Inspector tells me that loading the QString documentation takes 4.4 seconds, the whole 4.4 seconds being spent on qstring.html, a negligible amount of time on classic.css, and some 2.2 seconds interval on qt-logo.png (in parallel to qstring.html).
That is the browser demo from the 4.6.2 release. Qt/Carbon.
I can't check current 4.7, because whenever I click on a tab in the webinspector I get a crash (using Qt/Cocoa).

[Eike Ziller]

(In reply to comment #5)
> (In reply to comment #3)
> > Created an attachment (id=49894) [details] [details]
> > Open this in your browser, real life url
> > 
> > This is a real life example (online QString documentation) where it also takes
> > several seconds to show the page in QtWebkit, where the other browsers take
> > "basically no time".
> 
> This bug probably require specific conditions to be reproduced, the QString
> documentation loads instantly on the setup I have seen.
> Since you have the golden conditions, could you provide this information?
> 
> - Do you use a release build? If no could you try downloading the latest Qt
> version and try in the demo browser to see if you have a major improvement?
> 
> - There is a problem with the proxy detection mechanism on Windows where it
> will try to detect the proxy on each HTTP request. Could you disable
> "Automatically detect settings" in
> ControlPanel/InternetOptions/Connections/LANSettings and try again?
> 

> - Do you have any specific settings that could allow us to reproduce the bug?
> You might be able to see what happen a bit more by looking at the web
> inspector's resource tracking tab or by looking at Wireshark while loading the
> page.

Not that I know of. Standard Snow Leopard installation (also elsewhere with Leopard), no proxies etc.
Fetching from disk is fast enough (90ms), though the rendering is still slow.

Resource inspector tells me that fetching from network is between 4 and 20 seconds, doing repeated reloads.

[Jocelyn Turcotte]

(In reply to comment #6)
> Now, that becomes confusing to me.
> 
> Obviously "Page Loading" doesn't include rendering the page for you... which is
> most noticable slow for me.
> 
We try to separate bug reports by potential causes instead of symptoms. Unfortunately I had to use a wide title for this one when I created it since I didn't see the problem by myself.

(In reply to comment #7)
> Resource inspector tells me that fetching from network is between 4 and 20
> seconds, doing repeated reloads.

I tried to compare with Chrome and the inspector reports that it takes 1.4 seconds in arora compared to 0.4 in Chrome.
By looking at Wireshark I can see many times where the TCP window become full and we just stop picking up data from the network.

Could you try running Wireshark with a capture filter looking like "host doc.qt.nokia.com", reload the qstring doc a couple of times and attach the saved .pcap file to this bug report?

[Strahinja Markovic]

In my experience, the loading issue is not with the network, but the rendering speed. Here's a comment I made on the resize speed bug (http://bugreports.qt.nokia.com/browse/QTBUG-5929):

I'd also like to note that performance is worst on Windows machines. For a very large HTML file with source and images all *on disk*, rendering time on Win7 x64 is 45 seconds, but the same file renders in 2 seconds in the same application on Kubuntu Karmic x64. Granted, the second machine is slightly more powerful, but not 20+ times.

This is not file specific: for any really large HTML file, the rendering speed is terrible on Windows machines and more than an order of magnitude faster on Linux.

[Jocelyn Turcotte]

(In reply to comment #9)
> I'd also like to note that performance is worst on Windows machines. For a very
> large HTML file with source and images all *on disk*, rendering time on Win7
> x64 is 45 seconds, but the same file renders in 2 seconds in the same
> application on Kubuntu Karmic x64. Granted, the second machine is slightly more
> powerful, but not 20+ times.

The rendering on my Windows system is usually not that noticeably slow.

If you can attach a zip file with an HTML file that takes up to 45 seconds to render that would help us reproduce the problem.

[Strahinja Markovic]

I can't do it now, but I'll have a test case for you tomorrow.

[Strahinja Markovic]

Created attachment 50026 [details]
Testcase for slow windows/fast linux

Here is a test case for what I was talking about. It's an HTML version of the novel David Copperfield, with images (both text and images are in the publiv domain). The main.cpp creates a QWebView, loads the HTML and then displays the widget.

On my Windows machine, it takes 55 seconds from starting the application to the widget being displayed. Machine is Win7 x64, Core 2 Duo 6400 @ 2.13GHz, 4GB RAM, compiled in release mode on MSVC 9 (2008).

On my Linux machine, it takes 6 seconds for the same. Machine is Kubuntu Karmic x64, Core 2 Duo P8700 @ 2.53GHz, 4GB RAM, compiled in release mode on GCC 4.4.1.

This is not the same file I was talking about earlier (can't find it), but that file exhibited 45sec/2sec loading behavior on the same machines, compiled the same way.

[Strahinja Markovic]

Oh and the file renders *instantaneously* on both Windows and Linux machines in FF 3.6./3.5.8, respectively.

[Jocelyn Turcotte]

(In reply to comment #12)
> Created an attachment (id=50026) [details]
> Testcase for slow windows/fast linux
> 
> Here is a test case for what I was talking about. It's an HTML version of the
> novel David Copperfield, with images (both text and images are in the publiv
> domain). The main.cpp creates a QWebView, loads the HTML and then displays the
> widget.
> 
> On my Windows machine, it takes 55 seconds from starting the application to the
> widget being displayed. Machine is Win7 x64, Core 2 Duo 6400 @ 2.13GHz, 4GB
> RAM, compiled in release mode on MSVC 9 (2008).
> 
> On my Linux machine, it takes 6 seconds for the same. Machine is Kubuntu Karmic
> x64, Core 2 Duo P8700 @ 2.53GHz, 4GB RAM, compiled in release mode on GCC
> 4.4.1.
> 
> This is not the same file I was talking about earlier (can't find it), but that
> file exhibited 45sec/2sec loading behavior on the same machines, compiled the
> same way.

Thanks, I could reproduce it in trunk, the problem is obvious with that test case.

[Strahinja Markovic]

(In reply to comment #14)
> Thanks, I could reproduce it in trunk, the problem is obvious with that test
> case.

Does that mean we'll see FF-like rendering speed in the next version of Qt (or QtWebkit 2.0)? Or does it just mean that Windows rendering speed would become the same as that on Linux?

I'd *love* #1, but #2 would be pretty awesome too.

I understand this may all be just wishful thinking on my part, I'm sure it's not something you can easily fix.

[Jocelyn Turcotte]

(In reply to comment #15)
> Does that mean we'll see FF-like rendering speed in the next version of Qt (or
> QtWebkit 2.0)? Or does it just mean that Windows rendering speed would become
> the same as that on Linux?

Fixing this bug will only bring QtWebKit performance on Windows closer to the ones on Linux for large files.
If you have other test cases that helps isolate bottlenecks we'll be happy to have a look at them.

[Andreas Kling]

*** Bug 20054 has been marked as a duplicate of this bug. ***

[Simon Hausmann]

Created attachment 50792 [details]
Patch

[Simon Hausmann]

Could someone try the attached patch?

Holger, what do you think? I have vague memories of you trying the same thing not only for spaces.

[Benjamin Poulain]

It seems that patch help performance.
Some result before-after from the cycler:
http://en.wikipedia.org/wiki/Nokia: 1555 - 1374
http://www.amazon.com/Kindle-Wireless-Reading-Display-Generation/dp/B0015TG12Q/: 2294 - 2116
http://www.aol.com/: 2634 - 2588

Here are the raw results for a layouting stress case:
Test, data_source, median, deviation, average
"cycler::load","http://en.wikipedia.org/wiki/Line_of_succession_to_the_British_throne",4369,18,4359
"cycler::load","http://en.wikipedia.org/wiki/Line_of_succession_to_the_British_throne",3530,41,3539

[Holger Freyther]

(In reply to comment #19)
> Could someone try the attached patch?
> 
> Holger, what do you think? I have vague memories of you trying the same thing
> not only for spaces.

Interesting that the EWS is still green. I created a test case in reductions/float_width and I experimented using QFontMetrics not only for space but also other characters. The sad news was that my verification method tst_FloatWidth::floatWidth_western_compare failed.

[Holger Freyther]

Created attachment 50859 [details]
data file for the float_width test

You can copy that file in the float_width directory and then do your experiments in the floatWidth_fasta method. Simon's patch is getting rid of a WebCore::String -> QString conversion which is not going to show up in this reduction though.

[Eric Seidel (no email)]

Comment on attachment 50792 [details]
Patch

OK.  I trust you.

[Eric Seidel (no email)]

I see there is some discussion on this bug.  If you need more of a qt-specific reviewer to look at this, you should obviously wait to land. :)

[Andreas Kling]

(In reply to comment #18)
> Created an attachment (id=50792) [details]
> Patch

While definitely faster, QFontMetrics::width() yields slightly different (off by one) results than the QTextLayout approach.

I haven't found any dramatic regressions, so this looks like a good idea to me.

[Eric Seidel (no email)]

Attachment 50792 [details] was posted by a committer and has review+, assigning to Simon Hausmann for commit.

[Benjamin Poulain]

*** Bug 35652 has been marked as a duplicate of this bug. ***

[Simon Hausmann]

Ossy, could you try this patch on the bot?

I'm curious about the differences in the results there.

[Csaba Osztrogonác]

(In reply to comment #28)
> Ossy, could you try this patch on the bot?
> 
> I'm curious about the differences in the results there. 

I downloaded patch and binary text_data_western,
but I have no idea what kind of test I should run. 
I can't find any "tst_FloatWidth" filename/string
and float_width directory in WebKit trunk.

[Simon Hausmann]

(In reply to comment #29)
> (In reply to comment #28)
> > Ossy, could you try this patch on the bot?
> > 
> > I'm curious about the differences in the results there. 
> 
> I downloaded patch and binary text_data_western,
> but I have no idea what kind of test I should run. 
> I can't find any "tst_FloatWidth" filename/string
> and float_width directory in WebKit trunk.

Sorry, I mean layout tests :)

[Csaba Osztrogonác]

Created attachment 53170 [details]
results for patch (tar.gz)

(In reply to comment #28)
> Ossy, could you try this patch on the bot?
> I'm curious about the differences in the results there.

Results (DRT and pixel results) for the patch attached. 
There are 44 different tests, but all of them have minor
pixel differences and some different place of linebrakes
(caused by minor pixel differences).

As far as I see, size of whitespaces differs. 
You can check it with clicking on "image diff" link.

[Simon Hausmann]

(In reply to comment #31)
> Created an attachment (id=53170) [details]
> results for patch (tar.gz)
> 
> (In reply to comment #28)
> > Ossy, could you try this patch on the bot?
> > I'm curious about the differences in the results there.
> 
> Results (DRT and pixel results) for the patch attached. 
> There are 44 different tests, but all of them have minor
> pixel differences and some different place of linebrakes
> (caused by minor pixel differences).
> 
> As far as I see, size of whitespaces differs. 
> You can check it with clicking on "image diff" link.

Yeah, the results look okay to me, too.

Ossy, could you land my patch together with the updated results?

[Csaba Osztrogonác]

> Yeah, the results look okay to me, too.
> 
> Ossy, could you land my patch together with the updated results?
Sure, but I have a question before it.

How should we merge this patch with http://trac.webkit.org/changeset/57516 ?
Should we change only the non-Qt-4.7 case?

[Simon Hausmann]

(In reply to comment #33)
> > Yeah, the results look okay to me, too.
> > 
> > Ossy, could you land my patch together with the updated results?
> Sure, but I have a question before it.
> 
> How should we merge this patch with http://trac.webkit.org/changeset/57516 ?
> Should we change only the non-Qt-4.7 case?

Good point, I realize my patch needs a slight adaption. I'll do that first and get a new review first.

[Simon Hausmann]

Created attachment 54302 [details]
Patch

[Csaba Osztrogonác]

Created attachment 54691 [details]
followup patch for "54302: Patch"

[Simon Hausmann]

Comment on attachment 54691 [details]
followup patch for "54302: Patch"

rs=me

[Csaba Osztrogonác]

Patches landed in http://trac.webkit.org/changeset/58507 and http://trac.webkit.org/changeset/58508

[Simon Hausmann]

<cherry-pick-for-backport: r58507>

[Simon Hausmann]

<cherry-pick-for-backport: r58508>

[Simon Hausmann]

Revision r58507 cherry-picked into qtwebkit-2.0 with commit 3fee69ef1fb148f9d959a2d7eaf93ef97989caa4
Revision r58508 cherry-picked into qtwebkit-2.0 with commit 2c346f58ae70470d88dcd856bfe59b04a144b65a

[Csaba Osztrogonác]

(In reply to comment #41)
r58508 made svg/text/selection-background-color.xhtml test "fail" in the QtWebKit2.0 release branch: http://webkit.sed.hu/buildbot/results/QtWebKit2.0-branch%20x86-32%20Linux%20Release/r2c346f58ae70470d88dcd856bfe59b04a144b65a%20%28925%29/svg/text/selection-background-color-pretty-diff.html

It isn't a real bug. The root of the problem is that r58508 was a patch for trunk, but not for relase branch. r58212 modified something near SVG, for example rendertree dump. Qt expected file for svg/text/selection-background-color.xhtml was modified by r58215.

What should we do? Is it possible to rebaseline platform/qt/svg/text/selection-background-color-actual.txt only in release branch? Or should we cherry pick r58508 and r58212 too?

I reopened the bug, because we should decide how to make QtWebKit-2.0 release bot green.

[Csaba Osztrogonác]

(In reply to comment #42)
>Or should we cherry pick r58508 and r58212 too?
I mean r58212 and r58215.

[Simon Hausmann]

(In reply to comment #42)
> (In reply to comment #41)
> r58508 made svg/text/selection-background-color.xhtml test "fail" in the
> QtWebKit2.0 release branch:
> http://webkit.sed.hu/buildbot/results/QtWebKit2.0-branch%20x86-32%20Linux%20Release/r2c346f58ae70470d88dcd856bfe59b04a144b65a%20%28925%29/svg/text/selection-background-color-pretty-diff.html
> 
> It isn't a real bug. The root of the problem is that r58508 was a patch for
> trunk, but not for relase branch. r58212 modified something near SVG, for
> example rendertree dump. Qt expected file for
> svg/text/selection-background-color.xhtml was modified by r58215.
> 
> What should we do? Is it possible to rebaseline
> platform/qt/svg/text/selection-background-color-actual.txt only in release
> branch? Or should we cherry pick r58508 and r58212 too?
> 
> I reopened the bug, because we should decide how to make QtWebKit-2.0 release
> bot green.

I think rebaselining is the way to go, instead of cherry-picking a change where we don't know what the impact is. I'll adjust the result in the branch.

[Eric Seidel (no email)]

Comment on attachment 50792 [details]
Patch

Cleared Eric Seidel's review+ from obsolete attachment 50792 [details] so that this bug does not appear in http://webkit.org/pending-commit.

[Csaba Osztrogonác]

Rebase of platform/qt/svg/text/selection-background-color-actual.txt
pushed into qtwebkit-2.0 with commit 2babe0e90a662b2da29673ab751cf6f13e1cb52b

[Guillaume LACHAUD]

*** Bug 38386 has been marked as a duplicate of this bug. ***

[Strahinja Markovic]

Created attachment 55488 [details]
Profile for David Copperfield testcase with Qt 4.7 beta1 on Windows 7 x64 (32bit Qt and app)

Enticed by bug 38386, I've downloaded the precompiled binaries of Qt 4.7 beta for Windows, VS 2008 and timed the testcase I uploaded previously (the David Copperfield testcase, "slow windows/fast linux").

With Qt 4.6.2 (Nokia binaries), the window is shown after 47 seconds.
With Qt 4.7.0beta1 (Nokia binaries), the window is shown after 21 seconds.

Machine is Core 2 Duo P8700 @ 2.53GHz, 4GB RAM.

So it's now twice as fast, but still slow. Back in March on this same machine in Linux the testcase runs in 6 seconds with Qt 4.6.2. I haven't had time to test on Linux with the new beta, but I'm sure it's not slower than it used to be. So on Windows, QtWebKit loads pages at least 4 times slower.

I'm attaching some very thorough profile data generated with the AQtime profiler. There's a README file inside that explains what each file is. The profile covers the entire loading procedure.

If you need more data, just ask.

[Strahinja Markovic]

Just wanted to point out a few things for those unfamiliar with the terminology used by the AQtime profiler:

"Time with children" means time spent in the function and all the functions that that function calls.

"Time" means just the time spent in the function, NOT counting the time spent in child calls.

These are the two most important data items for each function.

[Strahinja Markovic]

One last thing... the README file states that times are in milliseconds. This is wrong. Times are in seconds.