Per discussion in bug 76022, we have a number of "slow" tests that are run during test-webkitpy. This bug analyzes where they are and what to do about them. Roughly speaking, we should fix the really slow ones as much as possible, and perhaps annotate the others with 'slow' (for some definition of slow) so that we can do a 'fast' run that skips them. Building on the newly-enhanced test-webkitpy (rollup patch for it in bug 77687; not ready for review) and the attached script to bucket the output of -t (test-webkitpy -t 2>&1 | egrep 'passed|failed' | egrep -v 'all passed' | awk '{ printf("%s %f", $1, $4); }' | buckets) , we get the following ouput: For what it's worth, here's the output of said script: bucket count ccount %ccount time ctime %time %ctime 0.00001 - 0.00003 80 80 0.06 0.00 0.00 0.00 0.00 0.00003 - 0.00010 131 211 0.16 0.01 0.01 0.00 0.00 0.00010 - 0.00032 227 438 0.34 0.04 0.05 0.00 0.00 0.00032 - 0.00100 175 613 0.47 0.10 0.15 0.00 0.00 0.00100 - 0.00316 205 818 0.63 0.36 0.51 0.01 0.01 0.00316 - 0.01000 83 901 0.70 0.45 0.96 0.01 0.02 0.01000 - 0.03162 219 1120 0.87 4.16 5.11 0.07 0.09 0.03162 - 0.10000 88 1208 0.93 4.69 9.81 0.08 0.17 0.10000 - 0.31623 48 1256 0.97 8.71 18.51 0.15 0.32 0.31623 - 1.00000 18 1274 0.99 10.33 28.85 0.18 0.50 1.00000 - 3.16228 17 1291 1.00 24.87 53.72 0.43 0.94 3.16228 - 10.00000 1 1292 1.00 3.63 57.35 0.06 1.00 legend: bucket: time it took to run the test in seconds count: # of tests in the bucket ccount: cumulative # of tests %ccount: cumulative % of tests in this plus the preceding buckets time: total time of all tests in this bucket ctime: cumulative time in this plus the preceding buckets %time: time in this bucket as a %age of total execution time %ctime: cumulative time of tests So, we see, for example, that 87% of our tests complete in under 32 milliseconds each (five seconds total), and 18 tests (~1%) account for half of the total test run time, with another 18 accounting for another sixth. It seems like a reasonable definition of a 'slow' test would therefore be one that takes more than somewhere between 32-320ms, depending on how aggressive we wanted to be. I'd probably fall closer to the 32ms end.
marking as blocking bug 76022 for the 'slow' decorator part of this.
Note that this includes all of the tests run by test-webkitpy by default, so it includes the tests in run_webkit_tests_integrationtest, but does not include scm_unittest or the integration_test_* routines in webkitpy.layout_tests.*
Can we have a list of the slow tests?
Created attachment 126395 [details] test times for webkitpy sorted by execution time as of r107291
(In reply to comment #3) > Can we have a list of the slow tests? Surely. attached.
I think most tests are fast enough now. Closing. Maybe at some point I'll get around to adding the @slow decorator once my other test-webkitpy changes have landed.