Move Safari performance scripts into the WebKit public repository Safari should move all (or at least most) of its performance testing scripts into the public repository. At least enough to support the PLT build slaves and allow external developers to run external PLT suites (such as the SVG PLT suites as part of bug 10265).
jorlow: can a page cycler be easily hooked up to DRT? [5:48pm] dglazkov: jorlow: it's not easy, because we can't keep snapshots of sites in public repo. [5:48pm] • dglazkov is not a lawyer. [5:49pm] jorlow: dglazkov: um..why do we need to put it in a public repo? [5:49pm] dglazkov: jorlow: well.. that's a good question! [5:49pm] jorlow: chromium doesn't but runs page cycler bots publicly [5:49pm] jorlow: the most important thing is knowing that a regression happened with perf stuff [5:50pm] jorlow: it does suck that we can't release that (and apple can't release theirs, etc) but....something is better than nothing [5:50pm] dglazkov: jorlow: let's file a bug and start talking about it there -- maybe we'll figure something out [5:50pm] smfr: can't we write a script that bowdlerizes the test pages? [5:50pm] smfr: and replace all the images with dummy images or something? [5:51pm] jorlow: smfr: what test pages? [5:51pm] jorlow: the layout tests? [5:51pm] smfr: pages used by a page cycler [5:51pm] dglazkov: smfr: hmm, that may work. We just need to make sure we still have meaningful tests [5:51pm] jorlow: well, then you're doing network requests [5:51pm] smfr: e.g. the infamous PLT tests [5:51pm] dglazkov: what are the PLT tests, btw? [5:52pm] smfr: page load tests [5:52pm] smfr: a bunch of pages scraped back in 2000 and cleaned up [5:53pm] zecke: dglazkov: did you guys look into Mozilla's talos? at least they use GFDL licensed content... (even if it is not really representative for the web) [5:53pm] dglazkov: smfr: well, duh I mean, are they public? [5:53pm] smfr: dglazkov: no [5:53pm] jorlow: exactly [5:53pm] jorlow: zecke: I haven't heard of that [5:53pm] jorlow: does sound interesting tho [5:53pm] dglazkov: smfr: I quite like the bowdlerizer idea [5:54pm] zecke: jorlow: IIRC: https://wiki.mozilla.org/StandaloneTalos [5:54pm] jorlow: dglazkov: ^^^ [5:54pm] jorlow: seems like a good start [5:55pm] jorlow: er....dglazkov, smfr [5:55pm] zecke: and back when working for Nokia, I did this: http://trac.webkit.org/wiki/QtWebKitPerformanceUtilities [5:56pm] zecke: this way the scaper is public, the content is not.
run-page-load-test is already in teh public repository. But it's not as useful w/o Apple's "autovicki" harness which knows how to restart your machine in a new user, clear the caches, and turn off the network before running the tests. None of this is useful at all w/o data. run-page-load-test uses the SVG PLT (which is public) in svn.webkit.org for now, but we need some HTML PLTs too.
Thanks for the info, Eric! It made me realize I forgot to mention that we should be running this stuff on the bots. Note that in the IRC log I posted there were some suggestions for how we could do HTML PLTestss.
http://trac.webkit.org/browser/trunk/WebCore/benchmarks/parser/html-parser.html might be a good place to start since it's a fully public benchmark. It tests a relatively narrow slice of the code though.
<rdar://problem/9509295>
I intend to work on this bug this quarter.
Sadly more issues are popping up as I integrate things :(
Created attachment 123686 [details] Adds a Chromium Mac Perf bot
+darin, +fishd just to let them know we now have an ability to add more perf tests & perf bots.
Comment on attachment 123686 [details] Adds a Chromium Mac Perf bot View in context: https://bugs.webkit.org/attachment.cgi?id=123686&action=review > Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:654 > + command = ["curl", "--request", "POST", "--data", "@perf-test-results.json", > + ] You can drop the trailing "," if you want to put the ] on the same line and pass PEP8.
Comment on attachment 123686 [details] Adds a Chromium Mac Perf bot View in context: https://bugs.webkit.org/attachment.cgi?id=123686&action=review >> Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:654 >> + ] > > You can drop the trailing "," if you want to put the ] on the same line and pass PEP8. Hmm, I wonder if curl is installed on all the bots. It looks like DownloadBuiltProduct uses curl, but maybe that's not used by all the bots.
Comment on attachment 123686 [details] Adds a Chromium Mac Perf bot View in context: https://bugs.webkit.org/attachment.cgi?id=123686&action=review >>> Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:654 >>> + ] >> >> You can drop the trailing "," if you want to put the ] on the same line and pass PEP8. > > Hmm, I wonder if curl is installed on all the bots. It looks like DownloadBuiltProduct uses curl, but maybe that's not used by all the bots. Oops, this command is never used :( It's a left over before I added --test-results-server to run-perf-tests. Will remove this command before landing it.
Committed r105848: <http://trac.webkit.org/changeset/105848>