10266 2006-08-05 01:41:20 -0700 WebKit needs public performance tests running on public bots 2012-01-24 22:08:47 -0800 1 1 1 Unclassified WebKit Tools / Tests 420+ Mac OS X 10.4 RESOLVED FIXED InRadar, ToolsHitList P3 Normal --- 76132 76312 76504 76594 76680 76764 76802 1 eric rniwa abarth ap aroben darin dbates dglazkov dominicc fishd jorlow rniwa simon.fraser tony zecke oldest_to_newest 51841 0 eric 2006-08-05 01:41:20 -0700 Move Safari performance scripts into the WebKit public repository Safari should move all (or at least most) of its performance testing scripts into the public repository. At least enough to support the PLT build slaves and allow external developers to run external PLT suites (such as the SVG PLT suites as part of bug 10265). 241904 1 jorlow 2010-06-23 09:59:34 -0700 jorlow: can a page cycler be easily hooked up to DRT? [5:48pm] dglazkov: jorlow: it's not easy, because we can't keep snapshots of sites in public repo. [5:48pm] • dglazkov is not a lawyer. [5:49pm] jorlow: dglazkov: um..why do we need to put it in a public repo? [5:49pm] dglazkov: jorlow: well.. that's a good question! [5:49pm] jorlow: chromium doesn't but runs page cycler bots publicly [5:49pm] jorlow: the most important thing is knowing that a regression happened with perf stuff [5:50pm] jorlow: it does suck that we can't release that (and apple can't release theirs, etc) but....something is better than nothing [5:50pm] dglazkov: jorlow: let's file a bug and start talking about it there -- maybe we'll figure something out [5:50pm] smfr: can't we write a script that bowdlerizes the test pages? [5:50pm] smfr: and replace all the images with dummy images or something? [5:51pm] jorlow: smfr: what test pages? [5:51pm] jorlow: the layout tests? [5:51pm] smfr: pages used by a page cycler [5:51pm] dglazkov: smfr: hmm, that may work. We just need to make sure we still have meaningful tests [5:51pm] jorlow: well, then you're doing network requests [5:51pm] smfr: e.g. the infamous PLT tests [5:51pm] dglazkov: what are the PLT tests, btw? [5:52pm] smfr: page load tests [5:52pm] smfr: a bunch of pages scraped back in 2000 and cleaned up [5:53pm] zecke: dglazkov: did you guys look into Mozilla's talos? at least they use GFDL licensed content... (even if it is not really representative for the web) [5:53pm] dglazkov: smfr: well, duh I mean, are they public? [5:53pm] smfr: dglazkov: no [5:53pm] jorlow: exactly [5:53pm] jorlow: zecke: I haven't heard of that [5:53pm] jorlow: does sound interesting tho [5:53pm] dglazkov: smfr: I quite like the bowdlerizer idea [5:54pm] zecke: jorlow: IIRC: https://wiki.mozilla.org/StandaloneTalos [5:54pm] jorlow: dglazkov: ^^^ [5:54pm] jorlow: seems like a good start [5:55pm] jorlow: er....dglazkov, smfr [5:55pm] zecke: and back when working for Nokia, I did this: http://trac.webkit.org/wiki/QtWebKitPerformanceUtilities [5:56pm] zecke: this way the scaper is public, the content is not. 241973 2 eric 2010-06-23 11:31:12 -0700 run-page-load-test is already in teh public repository. But it's not as useful w/o Apple's "autovicki" harness which knows how to restart your machine in a new user, clear the caches, and turn off the network before running the tests. None of this is useful at all w/o data. run-page-load-test uses the SVG PLT (which is public) in svn.webkit.org for now, but we need some HTML PLTs too. 241987 3 jorlow 2010-06-23 11:41:47 -0700 Thanks for the info, Eric! It made me realize I forgot to mention that we should be running this stuff on the bots. Note that in the IRC log I posted there were some suggestions for how we could do HTML PLTestss. 241991 4 abarth 2010-06-23 11:43:53 -0700 http://trac.webkit.org/browser/trunk/WebCore/benchmarks/parser/html-parser.html might be a good place to start since it's a fully public benchmark. It tests a relatively narrow slice of the code though. 410642 5 aroben 2011-05-26 11:43:47 -0700 <rdar://problem/9509295> 530792 6 rniwa 2012-01-05 17:02:25 -0800 I intend to work on this bug this quarter. 539992 7 rniwa 2012-01-20 18:32:24 -0800 Sadly more issues are popping up as I integrate things :( 541060 8 123686 rniwa 2012-01-23 18:21:21 -0800 Created attachment 123686 Adds a Chromium Mac Perf bot 541062 9 rniwa 2012-01-23 18:22:29 -0800 +darin, +fishd just to let them know we now have an ability to add more perf tests & perf bots. 541781 10 123686 abarth 2012-01-24 15:41:10 -0800 Comment on attachment 123686 Adds a Chromium Mac Perf bot View in context: https://bugs.webkit.org/attachment.cgi?id=123686&action=review > Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:654 > + command = ["curl", "--request", "POST", "--data", "@perf-test-results.json", > + ] You can drop the trailing "," if you want to put the ] on the same line and pass PEP8. 541789 11 123686 tony 2012-01-24 15:46:55 -0800 Comment on attachment 123686 Adds a Chromium Mac Perf bot View in context: https://bugs.webkit.org/attachment.cgi?id=123686&action=review >> Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:654 >> + ] > > You can drop the trailing "," if you want to put the ] on the same line and pass PEP8. Hmm, I wonder if curl is installed on all the bots. It looks like DownloadBuiltProduct uses curl, but maybe that's not used by all the bots. 541801 12 123686 rniwa 2012-01-24 16:03:53 -0800 Comment on attachment 123686 Adds a Chromium Mac Perf bot View in context: https://bugs.webkit.org/attachment.cgi?id=123686&action=review >>> Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:654 >>> + ] >> >> You can drop the trailing "," if you want to put the ] on the same line and pass PEP8. > > Hmm, I wonder if curl is installed on all the bots. It looks like DownloadBuiltProduct uses curl, but maybe that's not used by all the bots. Oops, this command is never used :( It's a left over before I added --test-results-server to run-perf-tests. Will remove this command before landing it. 542002 13 rniwa 2012-01-24 22:08:47 -0800 Committed r105848: <http://trac.webkit.org/changeset/105848> 123686 2012-01-23 18:21:21 -0800 2012-01-24 16:03:53 -0800 Adds a Chromium Mac Perf bot bug-10266-20120123182120.patch text/plain 5408 rniwa SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDEwNTY3NCkKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDIxIEBACisyMDEyLTAxLTIzICBSeW9zdWtlIE5pd2EgIDxybml3YUB3ZWJraXQub3JnPgor CisgICAgICAgIFdlYktpdCBuZWVkcyBwdWJsaWMgcGVyZm9ybWFuY2UgdGVzdHMgcnVubmluZyBv biBwdWJsaWMgYm90cworICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5j Z2k/aWQ9MTAyNjYKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAg ICAgICBDb21wbGV0ZXMgdGhlIGluaXRpYWwgaW1wbGVtZW50YXRpb24gb2YgV2ViS2l0IFBlcmYg Ym90cy4gQWRkIGEgYnVpbGQgc3RlcCBmb3IgcnVuLXBlcmYtdGVzdAorICAgICAgICBhbmQgbWFr ZSBnb29nbGUtbWFjLTQgcnVuIGl0LCBhbmQgdXBsb2FkIHRoZSByZXN1bHRzIHRvIHdlYmtpdC1w ZXJmLndlYmtpdC5vcmcuCisKKyAgICAgICAgKiBCdWlsZFNsYXZlU3VwcG9ydC9idWlsZC53ZWJr aXQub3JnLWNvbmZpZy9jb25maWcuanNvbjoKKyAgICAgICAgKiBCdWlsZFNsYXZlU3VwcG9ydC9i dWlsZC53ZWJraXQub3JnLWNvbmZpZy9tYXN0ZXIuY2ZnOgorICAgICAgICAoUnVuQW5kVXBsb2Fk UGVyZlRlc3RzKToKKyAgICAgICAgKFJ1bkFuZFVwbG9hZFBlcmZUZXN0cy5zdGFydCk6CisgICAg ICAgIChVcGxvYWRQZXJmVGVzdFJlc3VsdHMpOgorICAgICAgICAoQnVpbGRBbmRQZXJmVGVzdEZh Y3RvcnkpOgorICAgICAgICAoQnVpbGRBbmRQZXJmVGVzdEZhY3RvcnkuX19pbml0X18pOgorCiAy MDEyLTAxLTIzICBEaXJrIFByYW5rZSAgPGRwcmFua2VAY2hyb21pdW0ub3JnPgogCiAgICAgICAg IG5yd3Q6IG1ha2UgLS1jaHJvbWl1bSB3b3JrIGxpa2UgLS1xdApJbmRleDogVG9vbHMvQnVpbGRT bGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvY29uZmlnLmpzb24KPT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PQotLS0gVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvY29u ZmlnLmpzb24JKHJldmlzaW9uIDEwNTY1OCkKKysrIFRvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2J1 aWxkLndlYmtpdC5vcmctY29uZmlnL2NvbmZpZy5qc29uCSh3b3JraW5nIGNvcHkpCkBAIC0yMjgs NyArMjI4LDcgQEAKICAgICAgICAgICAgICAgICAgICAgewogICAgICAgICAgICAgICAgICAgICAg ICJuYW1lIjogIkNocm9taXVtIE1hYyBSZWxlYXNlIChUZXN0cykiLCAidHlwZSI6ICJOZXdCdWls ZEFuZFRlc3QiLCAiYnVpbGRkaXIiOiAiY2hyb21pdW0tbWFjLXJlbGVhc2UtdGVzdHMiLAogICAg ICAgICAgICAgICAgICAgICAgICJwbGF0Zm9ybSI6ICJjaHJvbWl1bS1tYWMiLCAiY29uZmlndXJh dGlvbiI6ICJyZWxlYXNlIiwgImFyY2hpdGVjdHVyZXMiOiBbImkzODYiXSwKLSAgICAgICAgICAg ICAgICAgICAgICAic2xhdmVuYW1lcyI6IFsiZ29vZ2xlLW1hYy0yIiwgImdvb2dsZS1tYWMtNCJd CisgICAgICAgICAgICAgICAgICAgICAgInNsYXZlbmFtZXMiOiBbImdvb2dsZS1tYWMtMiJdCiAg ICAgICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAgICAgIHsKICAgICAgICAgICAg ICAgICAgICAgICAibmFtZSI6ICJDaHJvbWl1bSBMaW51eCBSZWxlYXNlIiwgInR5cGUiOiAiQnVp bGQiLCAiYnVpbGRkaXIiOiAiY2hyb21pdW0tbGludXgtcmVsZWFzZSIsCkBAIC0yNTEsNiArMjUx LDExIEBACiAgICAgICAgICAgICAgICAgICAgICAgInNsYXZlbmFtZXMiOiBbImdvb2dsZS1saW51 eC1hbmRyb2lkIl0KICAgICAgICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICAgICAg eworICAgICAgICAgICAgICAgICAgICAgICJuYW1lIjogIkNocm9taXVtIE1hYyBSZWxlYXNlIChQ ZXJmKSIsICJ0eXBlIjogIkJ1aWxkQW5kUGVyZlRlc3QiLCAiYnVpbGRkaXIiOiAiY2hyb21pdW0t bWFjLXBlcmYtdGVzdHMiLAorICAgICAgICAgICAgICAgICAgICAgICJwbGF0Zm9ybSI6ICJjaHJv bWl1bS1tYWMiLCAiY29uZmlndXJhdGlvbiI6ICJyZWxlYXNlIiwgImFyY2hpdGVjdHVyZXMiOiBb ImkzODYiXSwKKyAgICAgICAgICAgICAgICAgICAgICAic2xhdmVuYW1lcyI6IFsiZ29vZ2xlLW1h Yy00Il0KKyAgICAgICAgICAgICAgICAgICAgfSwKKyAgICAgICAgICAgICAgICAgICAgewogICAg ICAgICAgICAgICAgICAgICAgICJuYW1lIjogIldpbkNhaXJvIFJlbGVhc2UiLCAidHlwZSI6ICJC dWlsZEFuZFRlc3QiLCAiYnVpbGRkaXIiOiAid2luLWNhaXJvLXJlbGVhc2UiLAogICAgICAgICAg ICAgICAgICAgICAgICJwbGF0Zm9ybSI6ICJ3aW5jYWlybyIsICJjb25maWd1cmF0aW9uIjogInJl bGVhc2UiLCAiYXJjaGl0ZWN0dXJlcyI6IFsiaTM4NiJdLAogICAgICAgICAgICAgICAgICAgICAg ICJzbGF2ZW5hbWVzIjogWyJ3aW5jYWlyby0xIl0KSW5kZXg6IFRvb2xzL0J1aWxkU2xhdmVTdXBw b3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL21hc3Rlci5jZmcKPT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9v bHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvbWFzdGVyLmNmZwko cmV2aXNpb24gMTA1NjU4KQorKysgVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0 Lm9yZy1jb25maWcvbWFzdGVyLmNmZwkod29ya2luZyBjb3B5KQpAQCAtNjE0LDYgKzYxNCwyNCBA QCBjbGFzcyBSdW5DaHJvbWl1bVdlYktpdFVuaXRUZXN0cyhzaGVsbC5UCiAgICAgICAgICAgICAg ICBXaXRoUHJvcGVydGllcygiLS0lKGNvbmZpZ3VyYXRpb24pcyIpXQogCiAKK2NsYXNzIFJ1bkFu ZFVwbG9hZFBlcmZUZXN0cyhzaGVsbC5UZXN0KToKKyAgIG5hbWUgPSAicGVyZi10ZXN0IgorICAg ZGVzY3JpcHRpb24gPSBbInBlcmYtdGVzdHMgcnVubmluZyJdCisgICBkZXNjcmlwdGlvbkRvbmUg PSBbInBlcmYtdGVzdHMiXQorICAgY29tbWFuZCA9IFsicHl0aG9uIiwgIi4vVG9vbHMvU2NyaXB0 cy9ydW4tcGVyZi10ZXN0cyIsCisgICAgICAgICAgICAgICItLW91dHB1dC1qc29uLXBhdGgiLCAi cGVyZi10ZXN0LXJlc3VsdHMuanNvbiIsCisgICAgICAgICAgICAgICItLXNvdXJjZS1qc29uLXBh dGgiLCAiLi4vLi4vcGVyZi10ZXN0LWNvbmZpZy5qc29uIiwKKyAgICAgICAgICAgICAgIi0tdGVz dC1yZXN1bHRzLXNlcnZlciIsICJ3ZWJraXQtcGVyZi5hcHBzcG90LmNvbSIsCisgICAgICAgICAg ICAgICItLWJ1aWxkZXItbmFtZSIsIFdpdGhQcm9wZXJ0aWVzKCIlKGJ1aWxkZXJuYW1lKXMiKSwK KyAgICAgICAgICAgICAgIi0tYnVpbGQtbnVtYmVyIiwgV2l0aFByb3BlcnRpZXMoIiUoYnVpbGRu dW1iZXIpcyIpLAorICAgICAgICAgICAgICAiLS1wbGF0Zm9ybSIsIFdpdGhQcm9wZXJ0aWVzKCIl KGZ1bGxQbGF0Zm9ybSlzIiksCisgICAgICAgICAgICAgIFdpdGhQcm9wZXJ0aWVzKCItLSUoY29u ZmlndXJhdGlvbilzIildCisKKyAgIGRlZiBzdGFydChzZWxmKToKKyAgICAgICBzZWxmLnNldENv bW1hbmQoc2VsZi5jb21tYW5kKQorICAgICAgIHJldHVybiBzaGVsbC5UZXN0LnN0YXJ0KHNlbGYp CisKKwogY2xhc3MgQXJjaGl2ZVRlc3RSZXN1bHRzKHNoZWxsLlNoZWxsQ29tbWFuZCk6CiAgICAg Y29tbWFuZCA9IFsicHl0aG9uIiwgIi4vVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvdGVzdC1yZXN1 bHQtYXJjaGl2ZSIsCiAgICAgICAgICAgICAgICBXaXRoUHJvcGVydGllcygiLS1wbGF0Zm9ybT0l KHBsYXRmb3JtKXMiKSwgV2l0aFByb3BlcnRpZXMoIi0tJShjb25maWd1cmF0aW9uKXMiKSwgImFy Y2hpdmUiXQpAQCAtNjMxLDYgKzY0OSwxNSBAQCBjbGFzcyBVcGxvYWRUZXN0UmVzdWx0cyh0cmFu c2Zlci5GaWxlVXBsCiAgICAgICAgIHRyYW5zZmVyLkZpbGVVcGxvYWQuX19pbml0X18oc2VsZiwg c2VsZi5zbGF2ZXNyYywgc2VsZi5tYXN0ZXJkZXN0LCBtb2RlPTA2NDQpCiAKIAorY2xhc3MgVXBs b2FkUGVyZlRlc3RSZXN1bHRzKHNoZWxsLlNoZWxsQ29tbWFuZCk6CisgICAgY29tbWFuZCA9IFsi Y3VybCIsICItLXJlcXVlc3QiLCAiUE9TVCIsICItLWRhdGEiLCAiQHBlcmYtdGVzdC1yZXN1bHRz Lmpzb24iLAorICAgICAgICBdCisgICAgbmFtZSA9ICJ1cGxvYWQtcGVyZi10ZXN0LXJlc3VsdHMi CisgICAgZGVzY3JpcHRpb24gPSBbInVwbG9hZGluZyBwZXJmIHRlc3QgcmVzdWx0cyJdCisgICAg ZGVzY3JpcHRpb25Eb25lID0gWyJ1cGxvYWRlZCBwZXJmIHRlc3QgcmVzdWx0cyJdCisgICAgaGFs Zk9uRmFpbHVyZSA9IFRydWUKKworCiBjbGFzcyBFeHRyYWN0VGVzdFJlc3VsdHMobWFzdGVyLk1h c3RlclNoZWxsQ29tbWFuZCk6CiAgICAgemlwRmlsZSA9IFdpdGhQcm9wZXJ0aWVzKCJwdWJsaWNf aHRtbC9yZXN1bHRzLyUoYnVpbGRlcm5hbWUpcy9yJShnb3RfcmV2aXNpb24pcyAoJShidWlsZG51 bWJlcilzKS56aXAiKQogICAgIHJlc3VsdERpcmVjdG9yeSA9IFdpdGhQcm9wZXJ0aWVzKCJwdWJs aWNfaHRtbC9yZXN1bHRzLyUoYnVpbGRlcm5hbWUpcy9yJShnb3RfcmV2aXNpb24pcyAoJShidWls ZG51bWJlcilzKSIpCkBAIC03NDksNiArNzc2LDE0IEBAIGNsYXNzIE5ld0J1aWxkQW5kVGVzdEZh Y3RvcnkoQnVpbGRBbmRUZXMKIGNsYXNzIFRlc3RXZWJLaXQyRmFjdG9yeShUZXN0RmFjdG9yeSk6 CiAgICAgVGVzdENsYXNzID0gUnVuV2ViS2l0MlRlc3RzCiAKK2NsYXNzIEJ1aWxkQW5kUGVyZlRl c3RGYWN0b3J5KEZhY3RvcnkpOgorICAgIGRlZiBfX2luaXRfXyhzZWxmLCBwbGF0Zm9ybSwgY29u ZmlndXJhdGlvbiwgYXJjaGl0ZWN0dXJlcywgKiprd2FyZ3MpOgorICAgICAgICBGYWN0b3J5Ll9f aW5pdF9fKHNlbGYsIHBsYXRmb3JtLCBjb25maWd1cmF0aW9uLCBhcmNoaXRlY3R1cmVzLCBGYWxz ZSwgKiprd2FyZ3MpCisgICAgICAgIGlmIHBsYXRmb3JtLnN0YXJ0c3dpdGgoImNocm9taXVtIik6 CisgICAgICAgICAgICBzZWxmLmFkZFN0ZXAoQ2xlYW51cENocm9taXVtQ3Jhc2hMb2dzKQorICAg ICAgICBzZWxmLmFkZFN0ZXAoQ29tcGlsZVdlYktpdCkKKyAgICAgICAgc2VsZi5hZGRTdGVwKFJ1 bkFuZFVwbG9hZFBlcmZUZXN0cykKKwogY2xhc3MgUGxhdGZvcm1TcGVjaWZpY1NjaGVkdWxlcihB bnlCcmFuY2hTY2hlZHVsZXIpOgogICAgIGRlZiBfX2luaXRfXyhzZWxmLCBwbGF0Zm9ybSwgYnJh bmNoLCAqKmt3YXJncyk6CiAgICAgICAgIHNlbGYucGxhdGZvcm0gPSBwbGF0Zm9ybQo=