Having to go through through hundreds to verify whether a patch regressed or progressed performance isn't practical. We need a better way of looking at perf test results.
Created attachment 159854 [details] tabular view Here's a sample output from perfalizer: https://bug-93629-attachments.webkit.org/attachment.cgi?id=158183 And the attachment is an experimental tabular view of the same data.
Better & Worse is computed from the percentage difference between the two. We report better/worse if they're better than the standard deviation of either sample.
Cool! Both the perfalizer and the tabular view compare the same revision. I think it would be useful to provide option to compare a range of revisions (avg of them) with a revision. Hmm... It would be also good if we could provide an overall/summarized result: I mean something like: From 100 FastMalloc results 69 were better 31 were worse What is your opinion?
(In reply to comment #3) > Both the perfalizer and the tabular view compare the same revision. > > I think it would be useful to provide option to compare a range of revisions (avg of them) with a revision. There is a way already! run-perf-tests --description "~" > Hmm... It would be also good if we could provide an overall/summarized result: > > I mean something like: > > From 100 FastMalloc results > 69 were better > 31 were worse > > What is your opinion? Yeah, but counting 0.1% improvement and 70% regression equally as "1" is quite misleading.
Created attachment 160364 [details] New tabular view
Coming this late but I'd say that the tabular view is so cool! I hope we can toggle memory / speed view since it's disturbing to investigate memory result freakiness when my main interest is speed. And I guess inverse is also true.
(In reply to comment #4) > Yeah, but counting 0.1% improvement and 70% regression equally as "1" is quite misleading. Absolutely true! (In reply to comment #5) > Created an attachment (id=160364) [details] > New tabular view Looks promising! Shouldn't we use kbytes on the view? Counting with bytes results too big numbers to handle.
Created attachment 160378 [details] Improved some cosmetics and hides memory results Not sure what kind of UI we need for toggling memory results. Any ideas?
Created attachment 160524 [details] Added Time/Memory toggle button
Created attachment 160557 [details] Got rid of scientific notation and added the toggle button for reference run
Created attachment 164240 [details] Fixes the bug
Created attachment 164241 [details] Sample output
Comment on attachment 164240 [details] Fixes the bug View in context: https://bugs.webkit.org/attachment.cgi?id=164240&action=review If you want a JS review, you should talk to arv or ojan or pavel. But I'm happy to rubber stamp this as an improvment over the current UI. > PerformanceTests/resources/jquery.tablesorter.min.js:1 > + I'm surprised we don't have this in the repo already. :) It's such a useful extension...
Realistically speaking, I'm just looking for a rubber stamp here.
Comment on attachment 164240 [details] Fixes the bug That, I can provide!
Comment on attachment 164240 [details] Fixes the bug Clearing flags on attachment: 164240 Committed r128779: <http://trac.webkit.org/changeset/128779>
All reviewed patches have been landed. Closing bug.