Bug 31719 - [Qt] Loading of large page can take up to 10 seconds
Summary: [Qt] Loading of large page can take up to 10 seconds
Status: CLOSED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Page Loading (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC Windows 7
: P2 Major
Assignee: Simon Hausmann
URL:
Keywords: Performance, Qt
: 20054 35652 (view as bug list)
Depends on:
Blocks: 34501 35784
  Show dependency treegraph
 
Reported: 2009-11-20 04:34 PST by Jocelyn Turcotte
Modified: 2010-05-08 17:20 PDT (History)
14 users (show)

See Also:


Attachments
Open this in your browser (1.00 MB, text/plain)
2010-03-03 03:13 PST, Eike Ziller
no flags Details
Open this in your browser, real life url (40 bytes, text/plain)
2010-03-03 03:16 PST, Eike Ziller
no flags Details
Testcase for slow windows/fast linux (2.63 MB, application/zip)
2010-03-04 08:19 PST, Strahinja Markovic
no flags Details
Patch (1.48 KB, patch)
2010-03-16 08:16 PDT, Simon Hausmann
no flags Details | Formatted Diff | Diff
data file for the float_width test (287.83 KB, application/octet-stream)
2010-03-16 17:51 PDT, Holger Freyther
no flags Details
results for patch (tar.gz) (1.69 MB, application/octet-stream)
2010-04-12 08:46 PDT, Csaba Osztrogonác_OOO_until_21st_Aug
no flags Details
Patch (1.64 KB, patch)
2010-04-26 08:36 PDT, Simon Hausmann
kenneth: review+
Details | Formatted Diff | Diff
followup patch for "54302: Patch" (141.00 KB, patch)
2010-04-29 04:52 PDT, Csaba Osztrogonác_OOO_until_21st_Aug
hausmann: review+
Details | Formatted Diff | Diff
Profile for David Copperfield testcase with Qt 4.7 beta1 on Windows 7 x64 (32bit Qt and app) (5.99 MB, application/zip)
2010-05-08 17:08 PDT, Strahinja Markovic
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jocelyn Turcotte 2009-11-20 04:34:31 PST
A problem have been reported that loading a large page can take up to 10 seconds in the demo browser, keeping the application busy and unresponsive.
The page complains about scripts problems.

This has been observed on Windows x64
Comment 1 Antonio Gomes 2009-11-20 18:56:34 PST
jocelyn any URL (testcase) has been reported on this ?
Comment 2 Eike Ziller 2010-03-03 03:13:17 PST
Created attachment 49893 [details]
Open this in your browser

Open the attached file in Safari, Chrome, Firefox: basically no time.
Open the attached file in the Qt Demo browser (Qt Webkit): in the range of 20 seconds.

This is not a Windows 7 64bit special thing. It's the same on other Windows versions and on the Mac, at the very least, and I've heard complains from Linux users too.
Comment 3 Eike Ziller 2010-03-03 03:16:35 PST
Created attachment 49894 [details]
Open this in your browser, real life url

This is a real life example (online QString documentation) where it also takes several seconds to show the page in QtWebkit, where the other browsers take "basically no time".
Comment 4 Jocelyn Turcotte 2010-03-03 04:26:21 PST
Comment on attachment 49893 [details]
Open this in your browser

This attachment reproduce a different bug related to large plain text files. See bug #20054

Obsoleting the attachment to prevent confusion.
Comment 5 Jocelyn Turcotte 2010-03-03 04:35:08 PST
(In reply to comment #3)
> Created an attachment (id=49894) [details]
> Open this in your browser, real life url
> 
> This is a real life example (online QString documentation) where it also takes
> several seconds to show the page in QtWebkit, where the other browsers take
> "basically no time".

This bug probably require specific conditions to be reproduced, the QString
documentation loads instantly on the setup I have seen.
Since you have the golden conditions, could you provide this information?

- Do you use a release build? If no could you try downloading the latest Qt
version and try in the demo browser to see if you have a major improvement?

- There is a problem with the proxy detection mechanism on Windows where it
will try to detect the proxy on each HTTP request. Could you disable
"Automatically detect settings" in ControlPanel/InternetOptions/Connections/LANSettings and try again?

- Do you have any specific settings that could allow us to reproduce the bug?
You might be able to see what happen a bit more by looking at the web inspector's resource tracking tab or by looking at Wireshark while loading the page.
Comment 6 Eike Ziller 2010-03-03 06:14:01 PST
(In reply to comment #4)
> (From update of attachment 49893 [details])
> This attachment reproduce a different bug related to large plain text files.
> See bug #20054
> 
> Obsoleting the attachment to prevent confusion.

Now, that becomes confusing to me.

Obviously "Page Loading" doesn't include rendering the page for you... which is most noticable slow for me.

Still, regarding "page loading", i.e. fetching the stuff. "Resources" Inspector tells me that loading the QString documentation takes 4.4 seconds, the whole 4.4 seconds being spent on qstring.html, a negligible amount of time on classic.css, and some 2.2 seconds interval on qt-logo.png (in parallel to qstring.html).
That is the browser demo from the 4.6.2 release. Qt/Carbon.
I can't check current 4.7, because whenever I click on a tab in the webinspector I get a crash (using Qt/Cocoa).
Comment 7 Eike Ziller 2010-03-03 06:53:16 PST
(In reply to comment #5)
> (In reply to comment #3)
> > Created an attachment (id=49894) [details] [details]
> > Open this in your browser, real life url
> > 
> > This is a real life example (online QString documentation) where it also takes
> > several seconds to show the page in QtWebkit, where the other browsers take
> > "basically no time".
> 
> This bug probably require specific conditions to be reproduced, the QString
> documentation loads instantly on the setup I have seen.
> Since you have the golden conditions, could you provide this information?
> 
> - Do you use a release build? If no could you try downloading the latest Qt
> version and try in the demo browser to see if you have a major improvement?
> 
> - There is a problem with the proxy detection mechanism on Windows where it
> will try to detect the proxy on each HTTP request. Could you disable
> "Automatically detect settings" in
> ControlPanel/InternetOptions/Connections/LANSettings and try again?
> 

> - Do you have any specific settings that could allow us to reproduce the bug?
> You might be able to see what happen a bit more by looking at the web
> inspector's resource tracking tab or by looking at Wireshark while loading the
> page.

Not that I know of. Standard Snow Leopard installation (also elsewhere with Leopard), no proxies etc.
Fetching from disk is fast enough (90ms), though the rendering is still slow.

Resource inspector tells me that fetching from network is between 4 and 20 seconds, doing repeated reloads.
Comment 8 Jocelyn Turcotte 2010-03-03 09:26:58 PST
(In reply to comment #6)
> Now, that becomes confusing to me.
> 
> Obviously "Page Loading" doesn't include rendering the page for you... which is
> most noticable slow for me.
> 
We try to separate bug reports by potential causes instead of symptoms. Unfortunately I had to use a wide title for this one when I created it since I didn't see the problem by myself.

(In reply to comment #7)
> Resource inspector tells me that fetching from network is between 4 and 20
> seconds, doing repeated reloads.

I tried to compare with Chrome and the inspector reports that it takes 1.4 seconds in arora compared to 0.4 in Chrome.
By looking at Wireshark I can see many times where the TCP window become full and we just stop picking up data from the network.

Could you try running Wireshark with a capture filter looking like "host doc.qt.nokia.com", reload the qstring doc a couple of times and attach the saved .pcap file to this bug report?
Comment 9 Strahinja Markovic 2010-03-03 09:52:26 PST
In my experience, the loading issue is not with the network, but the rendering speed. Here's a comment I made on the resize speed bug (http://bugreports.qt.nokia.com/browse/QTBUG-5929):

I'd also like to note that performance is worst on Windows machines. For a very large HTML file with source and images all *on disk*, rendering time on Win7 x64 is 45 seconds, but the same file renders in 2 seconds in the same application on Kubuntu Karmic x64. Granted, the second machine is slightly more powerful, but not 20+ times.

This is not file specific: for any really large HTML file, the rendering speed is terrible on Windows machines and more than an order of magnitude faster on Linux.
Comment 10 Jocelyn Turcotte 2010-03-03 10:32:36 PST
(In reply to comment #9)
> I'd also like to note that performance is worst on Windows machines. For a very
> large HTML file with source and images all *on disk*, rendering time on Win7
> x64 is 45 seconds, but the same file renders in 2 seconds in the same
> application on Kubuntu Karmic x64. Granted, the second machine is slightly more
> powerful, but not 20+ times.

The rendering on my Windows system is usually not that noticeably slow.

If you can attach a zip file with an HTML file that takes up to 45 seconds to render that would help us reproduce the problem.
Comment 11 Strahinja Markovic 2010-03-03 11:17:08 PST
I can't do it now, but I'll have a test case for you tomorrow.
Comment 12 Strahinja Markovic 2010-03-04 08:19:36 PST
Created attachment 50026 [details]
Testcase for slow windows/fast linux

Here is a test case for what I was talking about. It's an HTML version of the novel David Copperfield, with images (both text and images are in the publiv domain). The main.cpp creates a QWebView, loads the HTML and then displays the widget.

On my Windows machine, it takes 55 seconds from starting the application to the widget being displayed. Machine is Win7 x64, Core 2 Duo 6400 @ 2.13GHz, 4GB RAM, compiled in release mode on MSVC 9 (2008).

On my Linux machine, it takes 6 seconds for the same. Machine is Kubuntu Karmic x64, Core 2 Duo P8700 @ 2.53GHz, 4GB RAM, compiled in release mode on GCC 4.4.1.

This is not the same file I was talking about earlier (can't find it), but that file exhibited 45sec/2sec loading behavior on the same machines, compiled the same way.
Comment 13 Strahinja Markovic 2010-03-04 08:41:15 PST
Oh and the file renders *instantaneously* on both Windows and Linux machines in FF 3.6./3.5.8, respectively.
Comment 14 Jocelyn Turcotte 2010-03-05 09:15:55 PST
(In reply to comment #12)
> Created an attachment (id=50026) [details]
> Testcase for slow windows/fast linux
> 
> Here is a test case for what I was talking about. It's an HTML version of the
> novel David Copperfield, with images (both text and images are in the publiv
> domain). The main.cpp creates a QWebView, loads the HTML and then displays the
> widget.
> 
> On my Windows machine, it takes 55 seconds from starting the application to the
> widget being displayed. Machine is Win7 x64, Core 2 Duo 6400 @ 2.13GHz, 4GB
> RAM, compiled in release mode on MSVC 9 (2008).
> 
> On my Linux machine, it takes 6 seconds for the same. Machine is Kubuntu Karmic
> x64, Core 2 Duo P8700 @ 2.53GHz, 4GB RAM, compiled in release mode on GCC
> 4.4.1.
> 
> This is not the same file I was talking about earlier (can't find it), but that
> file exhibited 45sec/2sec loading behavior on the same machines, compiled the
> same way.

Thanks, I could reproduce it in trunk, the problem is obvious with that test case.
Comment 15 Strahinja Markovic 2010-03-05 14:19:56 PST
(In reply to comment #14)
> Thanks, I could reproduce it in trunk, the problem is obvious with that test
> case.

Does that mean we'll see FF-like rendering speed in the next version of Qt (or QtWebkit 2.0)? Or does it just mean that Windows rendering speed would become the same as that on Linux?

I'd *love* #1, but #2 would be pretty awesome too.

I understand this may all be just wishful thinking on my part, I'm sure it's not something you can easily fix.
Comment 16 Jocelyn Turcotte 2010-03-09 01:31:39 PST
(In reply to comment #15)
> Does that mean we'll see FF-like rendering speed in the next version of Qt (or
> QtWebkit 2.0)? Or does it just mean that Windows rendering speed would become
> the same as that on Linux?

Fixing this bug will only bring QtWebKit performance on Windows closer to the ones on Linux for large files.
If you have other test cases that helps isolate bottlenecks we'll be happy to have a look at them.
Comment 17 Andreas Kling 2010-03-10 10:03:52 PST
*** Bug 20054 has been marked as a duplicate of this bug. ***
Comment 18 Simon Hausmann 2010-03-16 08:16:15 PDT
Created attachment 50792 [details]
Patch
Comment 19 Simon Hausmann 2010-03-16 08:17:46 PDT
Could someone try the attached patch?

Holger, what do you think? I have vague memories of you trying the same thing not only for spaces.
Comment 20 Benjamin Poulain 2010-03-16 10:47:51 PDT
It seems that patch help performance.
Some result before-after from the cycler:
http://en.wikipedia.org/wiki/Nokia: 1555 - 1374
http://www.amazon.com/Kindle-Wireless-Reading-Display-Generation/dp/B0015TG12Q/: 2294 - 2116
http://www.aol.com/: 2634 - 2588

Here are the raw results for a layouting stress case:
Test, data_source, median, deviation, average
"cycler::load","http://en.wikipedia.org/wiki/Line_of_succession_to_the_British_throne",4369,18,4359
"cycler::load","http://en.wikipedia.org/wiki/Line_of_succession_to_the_British_throne",3530,41,3539
Comment 21 Holger Freyther 2010-03-16 17:47:39 PDT
(In reply to comment #19)
> Could someone try the attached patch?
> 
> Holger, what do you think? I have vague memories of you trying the same thing
> not only for spaces.

Interesting that the EWS is still green. I created a test case in reductions/float_width and I experimented using QFontMetrics not only for space but also other characters. The sad news was that my verification method tst_FloatWidth::floatWidth_western_compare failed.
Comment 22 Holger Freyther 2010-03-16 17:51:38 PDT
Created attachment 50859 [details]
data file for the float_width test

You can copy that file in the float_width directory and then do your experiments in the floatWidth_fasta method. Simon's patch is getting rid of a WebCore::String -> QString conversion which is not going to show up in this reduction though.
Comment 23 Eric Seidel 2010-03-24 23:57:29 PDT
Comment on attachment 50792 [details]
Patch

OK.  I trust you.
Comment 24 Eric Seidel 2010-03-24 23:58:27 PDT
I see there is some discussion on this bug.  If you need more of a qt-specific reviewer to look at this, you should obviously wait to land. :)
Comment 25 Andreas Kling 2010-03-26 06:16:33 PDT
(In reply to comment #18)
> Created an attachment (id=50792) [details]
> Patch

While definitely faster, QFontMetrics::width() yields slightly different (off by one) results than the QTextLayout approach.

I haven't found any dramatic regressions, so this looks like a good idea to me.
Comment 26 Eric Seidel 2010-03-29 11:39:34 PDT
Attachment 50792 [details] was posted by a committer and has review+, assigning to Simon Hausmann for commit.
Comment 27 Benjamin Poulain 2010-04-06 05:27:00 PDT
*** Bug 35652 has been marked as a duplicate of this bug. ***
Comment 28 Simon Hausmann 2010-04-08 06:44:39 PDT
Ossy, could you try this patch on the bot?

I'm curious about the differences in the results there.
Comment 29 Csaba Osztrogonác_OOO_until_21st_Aug 2010-04-08 07:45:47 PDT
(In reply to comment #28)
> Ossy, could you try this patch on the bot?
> 
> I'm curious about the differences in the results there. 

I downloaded patch and binary text_data_western,
but I have no idea what kind of test I should run. 
I can't find any "tst_FloatWidth" filename/string
and float_width directory in WebKit trunk.
Comment 30 Simon Hausmann 2010-04-08 08:23:35 PDT
(In reply to comment #29)
> (In reply to comment #28)
> > Ossy, could you try this patch on the bot?
> > 
> > I'm curious about the differences in the results there. 
> 
> I downloaded patch and binary text_data_western,
> but I have no idea what kind of test I should run. 
> I can't find any "tst_FloatWidth" filename/string
> and float_width directory in WebKit trunk.

Sorry, I mean layout tests :)
Comment 31 Csaba Osztrogonác_OOO_until_21st_Aug 2010-04-12 08:46:22 PDT
Created attachment 53170 [details]
results for patch (tar.gz)

(In reply to comment #28)
> Ossy, could you try this patch on the bot?
> I'm curious about the differences in the results there.

Results (DRT and pixel results) for the patch attached. 
There are 44 different tests, but all of them have minor
pixel differences and some different place of linebrakes
(caused by minor pixel differences).

As far as I see, size of whitespaces differs. 
You can check it with clicking on "image diff" link.
Comment 32 Simon Hausmann 2010-04-19 18:53:00 PDT
(In reply to comment #31)
> Created an attachment (id=53170) [details]
> results for patch (tar.gz)
> 
> (In reply to comment #28)
> > Ossy, could you try this patch on the bot?
> > I'm curious about the differences in the results there.
> 
> Results (DRT and pixel results) for the patch attached. 
> There are 44 different tests, but all of them have minor
> pixel differences and some different place of linebrakes
> (caused by minor pixel differences).
> 
> As far as I see, size of whitespaces differs. 
> You can check it with clicking on "image diff" link.

Yeah, the results look okay to me, too.

Ossy, could you land my patch together with the updated results?
Comment 33 Csaba Osztrogonác_OOO_until_21st_Aug 2010-04-20 06:41:05 PDT
> Yeah, the results look okay to me, too.
> 
> Ossy, could you land my patch together with the updated results?
Sure, but I have a question before it.

How should we merge this patch with http://trac.webkit.org/changeset/57516 ?
Should we change only the non-Qt-4.7 case?
Comment 34 Simon Hausmann 2010-04-20 07:37:33 PDT
(In reply to comment #33)
> > Yeah, the results look okay to me, too.
> > 
> > Ossy, could you land my patch together with the updated results?
> Sure, but I have a question before it.
> 
> How should we merge this patch with http://trac.webkit.org/changeset/57516 ?
> Should we change only the non-Qt-4.7 case?

Good point, I realize my patch needs a slight adaption. I'll do that first and get a new review first.
Comment 35 Simon Hausmann 2010-04-26 08:36:14 PDT
Created attachment 54302 [details]
Patch
Comment 36 Csaba Osztrogonác_OOO_until_21st_Aug 2010-04-29 04:52:08 PDT
Created attachment 54691 [details]
followup patch for "54302: Patch"
Comment 37 Simon Hausmann 2010-04-29 05:02:43 PDT
Comment on attachment 54691 [details]
followup patch for "54302: Patch"

rs=me
Comment 38 Csaba Osztrogonác_OOO_until_21st_Aug 2010-04-29 07:22:14 PDT
Patches landed in http://trac.webkit.org/changeset/58507 and http://trac.webkit.org/changeset/58508
Comment 39 Simon Hausmann 2010-04-29 07:28:00 PDT
<cherry-pick-for-backport: r58507>
Comment 40 Simon Hausmann 2010-04-29 07:28:20 PDT
<cherry-pick-for-backport: r58508>
Comment 41 Simon Hausmann 2010-04-29 07:40:20 PDT
Revision r58507 cherry-picked into qtwebkit-2.0 with commit 3fee69ef1fb148f9d959a2d7eaf93ef97989caa4
Revision r58508 cherry-picked into qtwebkit-2.0 with commit 2c346f58ae70470d88dcd856bfe59b04a144b65a
Comment 42 Csaba Osztrogonác_OOO_until_21st_Aug 2010-04-29 10:35:51 PDT
(In reply to comment #41)
r58508 made svg/text/selection-background-color.xhtml test "fail" in the QtWebKit2.0 release branch: http://webkit.sed.hu/buildbot/results/QtWebKit2.0-branch%20x86-32%20Linux%20Release/r2c346f58ae70470d88dcd856bfe59b04a144b65a%20%28925%29/svg/text/selection-background-color-pretty-diff.html

It isn't a real bug. The root of the problem is that r58508 was a patch for trunk, but not for relase branch. r58212 modified something near SVG, for example rendertree dump. Qt expected file for svg/text/selection-background-color.xhtml was modified by r58215.

What should we do? Is it possible to rebaseline platform/qt/svg/text/selection-background-color-actual.txt only in release branch? Or should we cherry pick r58508 and r58212 too?

I reopened the bug, because we should decide how to make QtWebKit-2.0 release bot green.
Comment 43 Csaba Osztrogonác_OOO_until_21st_Aug 2010-04-29 10:37:14 PDT
(In reply to comment #42)
>Or should we cherry pick r58508 and r58212 too?
I mean r58212 and r58215.
Comment 44 Simon Hausmann 2010-04-29 23:58:05 PDT
(In reply to comment #42)
> (In reply to comment #41)
> r58508 made svg/text/selection-background-color.xhtml test "fail" in the
> QtWebKit2.0 release branch:
> http://webkit.sed.hu/buildbot/results/QtWebKit2.0-branch%20x86-32%20Linux%20Release/r2c346f58ae70470d88dcd856bfe59b04a144b65a%20%28925%29/svg/text/selection-background-color-pretty-diff.html
> 
> It isn't a real bug. The root of the problem is that r58508 was a patch for
> trunk, but not for relase branch. r58212 modified something near SVG, for
> example rendertree dump. Qt expected file for
> svg/text/selection-background-color.xhtml was modified by r58215.
> 
> What should we do? Is it possible to rebaseline
> platform/qt/svg/text/selection-background-color-actual.txt only in release
> branch? Or should we cherry pick r58508 and r58212 too?
> 
> I reopened the bug, because we should decide how to make QtWebKit-2.0 release
> bot green.

I think rebaselining is the way to go, instead of cherry-picking a change where we don't know what the impact is. I'll adjust the result in the branch.
Comment 45 Eric Seidel 2010-05-02 19:25:44 PDT
Comment on attachment 50792 [details]
Patch

Cleared Eric Seidel's review+ from obsolete attachment 50792 [details] so that this bug does not appear in http://webkit.org/pending-commit.
Comment 46 Csaba Osztrogonác_OOO_until_21st_Aug 2010-05-03 01:02:22 PDT
Rebase of platform/qt/svg/text/selection-background-color-actual.txt
pushed into qtwebkit-2.0 with commit 2babe0e90a662b2da29673ab751cf6f13e1cb52b
Comment 47 Guillaume LACHAUD 2010-05-07 04:51:39 PDT
*** Bug 38386 has been marked as a duplicate of this bug. ***
Comment 48 Strahinja Markovic 2010-05-08 17:08:47 PDT
Created attachment 55488 [details]
Profile for David Copperfield testcase with Qt 4.7 beta1 on Windows 7 x64 (32bit Qt and app)

Enticed by bug 38386, I've downloaded the precompiled binaries of Qt 4.7 beta for Windows, VS 2008 and timed the testcase I uploaded previously (the David Copperfield testcase, "slow windows/fast linux").

With Qt 4.6.2 (Nokia binaries), the window is shown after 47 seconds.
With Qt 4.7.0beta1 (Nokia binaries), the window is shown after 21 seconds.

Machine is Core 2 Duo P8700 @ 2.53GHz, 4GB RAM.

So it's now twice as fast, but still slow. Back in March on this same machine in Linux the testcase runs in 6 seconds with Qt 4.6.2. I haven't had time to test on Linux with the new beta, but I'm sure it's not slower than it used to be. So on Windows, QtWebKit loads pages at least 4 times slower.

I'm attaching some very thorough profile data generated with the AQtime profiler. There's a README file inside that explains what each file is. The profile covers the entire loading procedure.

If you need more data, just ask.
Comment 49 Strahinja Markovic 2010-05-08 17:15:47 PDT
Just wanted to point out a few things for those unfamiliar with the terminology used by the AQtime profiler:

"Time with children" means time spent in the function and all the functions that that function calls.

"Time" means just the time spent in the function, NOT counting the time spent in child calls.

These are the two most important data items for each function.
Comment 50 Strahinja Markovic 2010-05-08 17:20:17 PDT
One last thing... the README file states that times are in milliseconds. This is wrong. Times are in seconds.