Bug 286603 – [Results DB] Really slow with large numbers of configurations

• Status: NEW
• Component: Tools / Tests

[Sam Sneddon [:gsnedders]]

https://results.webkit.org/?suite=layout-tests&suite=layout-tests&test=fonts%2Ffont-weight-invalid-crash.html&test=fonts%2Fligature.html&flavor=wk1&recent=false

Both Safari and Chrome are spending 80% of their time within this JS loop: https://github.com/WebKit/WebKit/blob/dccfae42bb29bd4bdef052e469f604a9387241c0/Tools/Scripts/libraries/resultsdbpy/resultsdbpy/view/static/js/timeline.js#L103-L111

I don't quite understand what this is trying to do, but it feels like we need an algorithmic improvement here?

The majority of the rest of the time is the single line https://github.com/WebKit/WebKit/blob/dccfae42bb29bd4bdef052e469f604a9387241c0/Tools/Scripts/libraries/resultsdbpy/resultsdbpy/view/static/js/timeline.js#L139, because Array.prototype.unshift is O(n) and we have _lots_ of commits.

Given scale just ends up with the same length as commits if I'm reading the code correctly, with the same indicies, we can absolutely certainly do better (even just let scale = Array(commits.length).fill({}) would almost certainly be a win, and then interact with scale[i]).

[Karl Dubost]

Created attachment 474112 [details]
firefox profile screenshot

In Firefox too, it spends around 50s to display the first table 
then another 15-20s for the second table.
https://share.firefox.dev/3WI57YV

[Karl Dubost]

Created attachment 474113 [details]
firefox network tab

The two JSON files are quite huge too. 
https://results.webkit.org/api/results/layout-tests/fonts/ligature.html?flavor=wk1&recent=false 7.96MB
https://results.webkit.org/api/results/layout-tests/fonts/font-weight-invalid-crash.html?flavor=wk1&recent=false 5.67 MB

And it downloads in total 23MB.

[Ahmad Saleem]

From Karl screenshot, it seems we are using HTTP/1.1 as well, can't we use HTTP/2 to alleviate network bottleneck (limited number of connection and one by one)?

[Radar WebKit Bug Importer]

<rdar://problem/144089048>

[Alexey Proskuryakov]

I do not see any evidence in the Network panel that the load is impacted by HTTP/1 limitations. It's probably best to have this bug focused on the algorithmic issue as originally reported.

[Sam Sneddon [:gsnedders]]

Indeed; the bottleneck is entirely in the JS we're executing.

Certainly there's low-hanging fruit here with network traffic here, but that's really worthy of separate issues (API responses are both slow and uncompressed; the latter is probably easy enough to solve).