Bug 92844 – Add European replay performance tests

• Status: RESOLVED
• Resolution: WONTFIX
• Component: Tools / Tests

[Nandor Huszka]

Add European replay performance tests that do not use external resources.

[Nandor Huszka]

Created attachment 159905 [details]
Patch

[Zoltan Horvath]

Comment on attachment 159905 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=159905&action=review

> PerformanceTests/ChangeLog:8
> +        Add some European sites that use the least external resources possible.

Does this mean that the replayed version of these sites won't use external resources due to some tricky js, right?
If yes, you can just write e.g. "Added new replay tests as follows."

[Nandor Huszka]

(In reply to comment #2)
> (From update of attachment 159905 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=159905&action=review
> 
> > PerformanceTests/ChangeLog:8
> > +        Add some European sites that use the least external resources possible.
> 
> Does this mean that the replayed version of these sites won't use external resources due to some tricky js, right?
> If yes, you can just write e.g. "Added new replay tests as follows."

I chose sites that do not use e.g. Google analytics, or Flash contents that may load external resources using the Chromium's inspect element feature. So in my opinion, there is no remaining inappropriate site in the list.

[Zoltan Horvath]

(In reply to comment #3)
> I chose sites that do not use e.g. Google analytics, or Flash contents that may load external resources using the Chromium's inspect element feature. So in my opinion, there is no remaining inappropriate site in the list.

Right! Then it looks okay with the mentioned changelog change, except I'm not sure we want to add a torrent search site as a test case, Rysouke?

[Peter Beverloo]

Comment on attachment 159905 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=159905&action=review

Some nits. Should we have Italian, Dutch and maybe Greek sites as well?

> PerformanceTests/Replay/Czech/www.computer.cz.replay:1
> +http://web.archive.org/web/20110722035404/http://www.computer.cz/

This is just a login form. Do we gain anything by testing it?

> PerformanceTests/Replay/Czech/www.toplist.cz.replay:1
> +http://web.archive.org/web/20110719062953/http://toplist.cz/

Despite this being a replay test, it doesn't seem like a great idea to include this.. Could we skip it, or include a similar site with less questions around their legality?

> PerformanceTests/Replay/English/torrentz.eu.replay:1
> +http://web.archive.org/web/20110720173242/http://torrentz.eu/

dito.

[Nandor Huszka]

(In reply to comment #5)
> Some nits. Should we have Italian, Dutch and maybe Greek sites as well?

We would like to use these sites for pageload tests and to see how WebKit performances on them as time goes by. So it is good idea putting some more webpage in these languages.

> This is just a login form. Do we gain anything by testing it?

> > PerformanceTests/Replay/Czech/www.toplist.cz.replay:1
> > +http://web.archive.org/web/20110719062953/http://toplist.cz/
> 
> Despite this being a replay test, it doesn't seem like a great idea to include this.. Could we skip it, or include a similar site with less questions around their legality?
>
> > PerformanceTests/Replay/English/torrentz.eu.replay:1
> > +http://web.archive.org/web/20110720173242/http://torrentz.eu/

Of course, I will skip these and search for another ones.

[Nandor Huszka]

Created attachment 160692 [details]
Patch v2

I skipped some site, and extended the list with new ones. Changelog text also changed, as Zoltan has suggested.

[Zoltan Horvath]

It's okay now, it has been sitting here for almost a month...

Informal r+ from me!

[Nandor Huszka]

Created attachment 166649 [details]
Patch v2 rechecked

[Zoltan Horvath]

Comment on attachment 166649 [details]
Patch v2 rechecked

rniwa, review ping

[Ryosuke Niwa]

Comment on attachment 166649 [details]
Patch v2 rechecked

Can we see console output when you run these tests (the first "prepare" phase to generate wpr and subsequent runs)?
Also, what does the results look like?

[Nandor Huszka]

(In reply to comment #11)
> (From update of attachment 166649 [details])
> Can we see console output when you run these tests (the first "prepare" phase to generate wpr and subsequent runs)?
> Also, what does the results look like?

Unfortunately I cannot run replay tests, because  they are currently supported on Mac port and Chromium port on Mac and Linux (it stands on the trac wiki), and I work on Qt port.

[Zoltan Horvath]

(In reply to comment #12)
> (In reply to comment #11)
> > (From update of attachment 166649 [details] [details])
> > Can we see console output when you run these tests (the first "prepare" phase to generate wpr and subsequent runs)?
> > Also, what does the results look like?
> 
> Unfortunately I cannot run replay tests, because  they are currently supported on Mac port and Chromium port on Mac and Linux (it stands on the trac wiki), and I work on Qt port.

What about QtWebkit on linux? It should work, since I used to run it. ;)

[Ryosuke Niwa]

Yeah, I think Zoltan added the support for Qt. It should run just fine there as well.

[Nandor Huszka]

(In reply to comment #14)
> Yeah, I think Zoltan added the support for Qt. It should run just fine there as well.

I've tried Zoltan's suggestion (https://gist.github.com/4236113) and run replay tests with 'sudo Tools/Scripts/run-perf-tests --platform=qt --replay PerformanceTests/Replay' but got a DnsProxyException in dnsproxy.py line 40: "Invalid nameserver: 127.0.0.1 (causes an infinte loop)".

Setting up the http_proxy and https_proxy this way
export http_proxy=http://user:pwd@ip:8080/
export https_proxy=https://user:pwd@ip:8443/
caused the same output. (Capital env. variable names and without username and password were tried also.)

What did I wrong, or is there any other setting we need?

[Nandor Huszka]

(In reply to comment #15)
> Setting up the http_proxy and https_proxy this way
> export http_proxy=http://user:pwd@ip:8080/
> export https_proxy=https://user:pwd@ip:8443/
> caused the same output. (Capital env. variable names and without username and password were tried also.)

I almost forgot I've experimented with localhost (or 127.0.0.1) instead of ip in the envs, but it doesn't help.

[Balazs Kelemen]

(In reply to comment #16)
> (In reply to comment #15)
> > Setting up the http_proxy and https_proxy this way
> > export http_proxy=http://user:pwd@ip:8080/
> > export https_proxy=https://user:pwd@ip:8443/
> > caused the same output. (Capital env. variable names and without username and password were tried also.)
> 
> I almost forgot I've experimented with localhost (or 127.0.0.1) instead of ip in the envs, but it doesn't help.

This is because of the dns is handled in recent Ubuntu. In 12.10 the default dns server is a local daemon. Furthermore wpr does not play well with resolvconf. I worked it around by simply replacing the resolv.conf link generated by resolvconf with a plain old resolv.conf file.

[Nandor Huszka]

After setting the ports from 8080/8443 to 80/43 in perftest.py (as Zoltan suggested) and hacking resolvconf I still have an error: https://gist.github.com/hnandor/4983973
The run-perf-test script gets further on Balazs machine, but fails too:
https://gist.github.com/hnandor/4985693
We've also experimented with stopping dnsmasq and installing zlibc, but they didn't help unfortunately.

What settings/libs do you use with the latest WebKit version to run replay tests? Does the tool in the trunk want to use ipfirewall?

[Csaba Osztrogonác]

Is there anybody still interested fixing this bug? (Nándor doesn't 
work on WebKit anymore) Or should we close as WONTFIX?

[Anders Carlsson]

(In reply to comment #19)
> Is there anybody still interested fixing this bug? (Nándor doesn't 
> work on WebKit anymore) Or should we close as WONTFIX?

It's been over a year since this patch was uploaded, let's WONTFIX this.