| Summary: | WebKit needs public performance tests running on public bots | ||||||
|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Eric Seidel (no email) <eric> | ||||
| Component: | Tools / Tests | Assignee: | Ryosuke Niwa <rniwa> | ||||
| Status: | RESOLVED FIXED | ||||||
| Severity: | Normal | CC: | abarth, ap, aroben, darin, dbates, dglazkov, dominicc, fishd, jorlow, rniwa, simon.fraser, tony, zecke | ||||
| Priority: | P3 | Keywords: | InRadar, ToolsHitList | ||||
| Version: | 420+ | ||||||
| Hardware: | Mac | ||||||
| OS: | OS X 10.4 | ||||||
| Bug Depends on: | 76132, 76312, 76504, 76594, 76680, 76764, 76802 | ||||||
| Bug Blocks: | |||||||
| Attachments: |
|
||||||
|
Description
Eric Seidel (no email)
2006-08-05 01:41:20 PDT
jorlow: can a page cycler be easily hooked up to DRT? [5:48pm] dglazkov: jorlow: it's not easy, because we can't keep snapshots of sites in public repo. [5:48pm] • dglazkov is not a lawyer. [5:49pm] jorlow: dglazkov: um..why do we need to put it in a public repo? [5:49pm] dglazkov: jorlow: well.. that's a good question! [5:49pm] jorlow: chromium doesn't but runs page cycler bots publicly [5:49pm] jorlow: the most important thing is knowing that a regression happened with perf stuff [5:50pm] jorlow: it does suck that we can't release that (and apple can't release theirs, etc) but....something is better than nothing [5:50pm] dglazkov: jorlow: let's file a bug and start talking about it there -- maybe we'll figure something out [5:50pm] smfr: can't we write a script that bowdlerizes the test pages? [5:50pm] smfr: and replace all the images with dummy images or something? [5:51pm] jorlow: smfr: what test pages? [5:51pm] jorlow: the layout tests? [5:51pm] smfr: pages used by a page cycler [5:51pm] dglazkov: smfr: hmm, that may work. We just need to make sure we still have meaningful tests [5:51pm] jorlow: well, then you're doing network requests [5:51pm] smfr: e.g. the infamous PLT tests [5:51pm] dglazkov: what are the PLT tests, btw? [5:52pm] smfr: page load tests [5:52pm] smfr: a bunch of pages scraped back in 2000 and cleaned up [5:53pm] zecke: dglazkov: did you guys look into Mozilla's talos? at least they use GFDL licensed content... (even if it is not really representative for the web) [5:53pm] dglazkov: smfr: well, duh I mean, are they public? [5:53pm] smfr: dglazkov: no [5:53pm] jorlow: exactly [5:53pm] jorlow: zecke: I haven't heard of that [5:53pm] jorlow: does sound interesting tho [5:53pm] dglazkov: smfr: I quite like the bowdlerizer idea [5:54pm] zecke: jorlow: IIRC: https://wiki.mozilla.org/StandaloneTalos [5:54pm] jorlow: dglazkov: ^^^ [5:54pm] jorlow: seems like a good start [5:55pm] jorlow: er....dglazkov, smfr [5:55pm] zecke: and back when working for Nokia, I did this: http://trac.webkit.org/wiki/QtWebKitPerformanceUtilities [5:56pm] zecke: this way the scaper is public, the content is not. run-page-load-test is already in teh public repository. But it's not as useful w/o Apple's "autovicki" harness which knows how to restart your machine in a new user, clear the caches, and turn off the network before running the tests. None of this is useful at all w/o data. run-page-load-test uses the SVG PLT (which is public) in svn.webkit.org for now, but we need some HTML PLTs too. Thanks for the info, Eric! It made me realize I forgot to mention that we should be running this stuff on the bots. Note that in the IRC log I posted there were some suggestions for how we could do HTML PLTestss. http://trac.webkit.org/browser/trunk/WebCore/benchmarks/parser/html-parser.html might be a good place to start since it's a fully public benchmark. It tests a relatively narrow slice of the code though. I intend to work on this bug this quarter. Sadly more issues are popping up as I integrate things :( Created attachment 123686 [details]
Adds a Chromium Mac Perf bot
+darin, +fishd just to let them know we now have an ability to add more perf tests & perf bots. Comment on attachment 123686 [details] Adds a Chromium Mac Perf bot View in context: https://bugs.webkit.org/attachment.cgi?id=123686&action=review > Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:654 > + command = ["curl", "--request", "POST", "--data", "@perf-test-results.json", > + ] You can drop the trailing "," if you want to put the ] on the same line and pass PEP8. Comment on attachment 123686 [details] Adds a Chromium Mac Perf bot View in context: https://bugs.webkit.org/attachment.cgi?id=123686&action=review >> Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:654 >> + ] > > You can drop the trailing "," if you want to put the ] on the same line and pass PEP8. Hmm, I wonder if curl is installed on all the bots. It looks like DownloadBuiltProduct uses curl, but maybe that's not used by all the bots. Comment on attachment 123686 [details] Adds a Chromium Mac Perf bot View in context: https://bugs.webkit.org/attachment.cgi?id=123686&action=review >>> Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:654 >>> + ] >> >> You can drop the trailing "," if you want to put the ] on the same line and pass PEP8. > > Hmm, I wonder if curl is installed on all the bots. It looks like DownloadBuiltProduct uses curl, but maybe that's not used by all the bots. Oops, this command is never used :( It's a left over before I added --test-results-server to run-perf-tests. Will remove this command before landing it. Committed r105848: <http://trac.webkit.org/changeset/105848> |