Bug 178277 - WPT server is 10x slower than Apache
Summary: WPT server is 10x slower than Apache
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-10-13 12:02 PDT by Alexey Proskuryakov
Modified: 2018-06-07 11:40 PDT (History)
6 users (show)

See Also:


Attachments
IDB sub test list (4.85 KB, text/plain)
2017-11-21 14:41 PST, youenn fablet
no flags Details
Patch to apply to WPT server to have Content-Length set more often (20.32 KB, text/plain)
2017-11-21 15:16 PST, youenn fablet
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Proskuryakov 2017-10-13 12:02:18 PDT
Fetching WPT resources is so slow that it makes WPT tests very slow overall. Furthermore, resources aren't cacheable, so the harness keeps being fetched over and over.

On my MacBook Pro:

$ time curl http://localhost:8800/FileAPI/url/url_xmlhttprequest.html -o /dev/null
...
real	0m0.231s

$ time curl http://localhost:8000/loading/simple-subframe.html -o /dev/null
...
real	0m0.025s

$ curl -i  http://localhost:8800/FileAPI/url/url_xmlhttprequest.html
HTTP/1.1 200 OK
Content-Type: text/html
Server: BaseHTTP/0.3 Python/2.7.10
Date: Fri, 13 Oct 2017 19:01:37 GMT
# where is Last-Modified?
...
Comment 1 youenn fablet 2017-10-13 13:08:46 PDT
We should measure how much WPT tests are slower compared to file system for instance through run-webkit-tests.
I did such measurements a year or so ago and it was more like 10/20% slower.

If the bottleneck is related to some specific URLS like testharness.js/testharnessreport.js, is there a way for DRT/WTR to intercept/optimize these fetches?
Comment 2 youenn fablet 2017-10-13 13:10:00 PDT
We should bring that to W3C and ask for optimizing WPT server.
Comment 3 Chris Dumez 2017-10-13 13:32:11 PDT
The results on my machine (using nsurl instead of curl to be more representative) confirm it is slower but not 10x:


==========================================

$ time nscurl http://localhost:8800/FileAPI/url/url_xmlhttprequest.html -o /dev/null

real	0m0.170s
user	0m0.024s
sys	0m0.021s

$ time nscurl http://localhost:8800/FileAPI/url/url_xmlhttprequest.html -o /dev/null

real	0m0.160s
user	0m0.022s
sys	0m0.018s

$ time nscurl http://localhost:8800/FileAPI/url/url_xmlhttprequest.html -o /dev/null

real	0m0.162s
user	0m0.022s
sys	0m0.019s

real AVG: 0m0.164

==========================================

$ time nscurl http://localhost:8000/loading/simple-subframe.html -o /dev/null

real	0m0.068s
user	0m0.021s
sys	0m0.017s

$ time nscurl http://localhost:8000/loading/simple-subframe.html -o /dev/null

real	0m0.058s
user	0m0.021s
sys	0m0.018s

$ time nscurl http://localhost:8000/loading/simple-subframe.html -o /dev/null

real	0m0.055s
user	0m0.020s
sys	0m0.017s

real AVG: 0m0.060s (2.7x faster)
Comment 4 Chris Dumez 2017-10-13 13:46:22 PDT
Headers are pretty different and I agree this is interesting:
HTTP/1.1 200 OK
Content-Type: text/html
Server: BaseHTTP/0.3 Python/2.7.10
Date: Fri, 13 Oct 2017 20:44:48 GMT

vs

HTTP/1.1 200 OK
Date: Fri, 13 Oct 2017 20:45:07 GMT
Server: Apache/2.4.27 (Unix) PHP/7.1.7 LibreSSL/2.2.7
Last-Modified: Fri, 10 Feb 2017 20:11:59 GMT
ETag: "96-54832b5c981c0"
Accept-Ranges: bytes
Content-Length: 150
Content-Type: text/html
Comment 5 youenn fablet 2017-11-21 14:41:45 PST
Created attachment 327431 [details]
IDB sub test list

Tests used to measure Apache/WPT performance differences
Comment 6 youenn fablet 2017-11-21 14:45:14 PST
I rerun the test list (in attachment) through WPT and Apache.
I observe a 20% slowdown with WPT when using run-webkit-tests.
When using run-webkit-tests with -f option, there is no observable difference in time needed to run the test suite.

This might mean the WPT lack of speed might not have a huge impact on running WPT tests. This should be completed with other tests, typically smaller tests where the difference might still be noticeable.
Comment 7 youenn fablet 2017-11-21 15:16:09 PST
Created attachment 327432 [details]
Patch to apply to WPT server to have Content-Length set more often
Comment 8 youenn fablet 2017-11-21 15:17:39 PST
I just tried the patch to compute Content-Length for WPT served files.
I no longer observe any speed difference between Apache and WPT.
Comment 9 Alexey Proskuryakov 2017-11-27 10:12:45 PST
> The results on my machine (using nsurl instead of curl to be more representative) confirm it is slower but not 10x:

The results seem to depend on the state of the system. Right now, there is virtually no difference between Apache and WPT on my machine, both take ~35 ms per resource. I'm not sure offhand what could be causing such dramatic changes.
Comment 10 youenn fablet 2017-11-27 10:48:39 PST
(In reply to Alexey Proskuryakov from comment #9)
> > The results on my machine (using nsurl instead of curl to be more representative) confirm it is slower but not 10x:
> 
> The results seem to depend on the state of the system. Right now, there is
> virtually no difference between Apache and WPT on my machine, both take ~35
> ms per resource. I'm not sure offhand what could be causing such dramatic
> changes.

There was this issue about TIME_WAIT (https://bugs.webkit.org/show_bug.cgi?id=164822).
Could it be that it would slow down things?
Maybe run some wpt tests on a running WPT server and shortly after do the nsurl check, or repetitively do the nsurl check.
Comment 11 youenn fablet 2017-11-27 10:52:27 PST
(In reply to youenn fablet from comment #8)
> I just tried the patch to compute Content-Length for WPT served files.
> I no longer observe any speed difference between Apache and WPT.

Without the patch, the 25% difference is still showing in my MacBook.
I do not see any difference at all on my iMac (different OS versions though).