160057 2016-07-21 15:51:16 -0700 Benchmark test content-animation timeouts always 2016-08-03 11:07:23 -0700 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED P2 Normal --- 156827 1 clopez clopez bugs-noreply cdumez cgarcia commit-queue glenn lforschler rniwa sam simon.fraser slewis oldest_to_newest 1213366 0 clopez 2016-07-21 15:51:16 -0700 The benchmark test content-animation added in https://trac.webkit.org/changeset/199805 timeouts. The GTK+ performance bots is running all the benchmark tests plans, and this is one is constantly timing out. Check: https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Perf%29 It uses the "--allplans" command line switch that runs all *.plan one after other. I tried running it on the GTK+ platform: $ Tools/Scripts/run-benchmark --platform gtk --browser minibrowser --plan content-animation and it timeouts I tried running it also on Mac platform: $ Tools/Scripts/run-benchmark --platform osx --browser safari --plan content-animation and it also timeouts. ... looking at the changelog of r199805 it says: """ When run as a benchmark, test completion involves a couple of bounces between the test and MobileSafari. Test completion is initiated by a setTimeout() in the test, at which point location.hash is set to "#done". MobileSafari detects that, and asynchronously dumps performance data. Once this is complete, MobileSafari changes the URL has to "#submit", which triggers the XHR to the test relay that indicates the test is complete. """ So, if I understand things correctly, this tests depends on some black-magic that only happens on MobileSafari to work? This is very unfortunate. I think we should either: Option 1) Fix this test so it can run with any browser. Option 2) Add a way to skip test plans by platform/browser or make some tests to only run on some very specific platform/browser. Opinions? 1213375 1 simon.fraser 2016-07-21 16:01:54 -0700 I thought I skipped them but can't find that commit now. 1213377 2 simon.fraser 2016-07-21 16:04:13 -0700 Ah, I did in https://trac.webkit.org/changeset/199906 1213382 3 clopez 2016-07-21 16:10:55 -0700 (In reply to comment #2) > Ah, I did in https://trac.webkit.org/changeset/199906 That is for performance tests (that are run with the Script Tools/Scripts/run-perf-tests Here I talk about the benchmark test that is ran with the script Tools/Scripts/run-benchmark The GTK perf test bot runs both kind of performance tests. It has 2 steps. First runs perf-tests and after that it runs benchmark-tests 1217000 4 285248 clopez 2016-08-03 10:55:29 -0700 Created attachment 285248 Patch 1217014 5 285248 clopez 2016-08-03 11:07:14 -0700 Comment on attachment 285248 Patch Clearing flags on attachment: 285248 Committed r204087: <http://trac.webkit.org/changeset/204087> 1217015 6 clopez 2016-08-03 11:07:23 -0700 All reviewed patches have been landed. Closing bug. 285248 2016-08-03 10:55:29 -0700 2016-08-03 11:07:14 -0700 Patch bug-160057-20160803195405.patch text/plain 2589 clopez U3VidmVyc2lvbiBSZXZpc2lvbjogMjA0MDg1CmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggZjI1Y2QxOTkyODBjMGFhZGY4NWMzMTk2ZTg3OTQyNmEz ODgwM2ZiZC4uNmFkMWEwZjliOTY4NGY4OWE1MTAyMTBiZTI4MDgwYzI4ZjE2MGVmZiAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDE0 IEBACisyMDE2LTA4LTAzICBDYXJsb3MgQWxiZXJ0byBMb3BleiBQZXJleiAgPGNsb3BlekBpZ2Fs aWEuY29tPgorCisgICAgICAgIEJlbmNobWFyayB0ZXN0IGNvbnRlbnQtYW5pbWF0aW9uIHRpbWVv dXRzIGFsd2F5cworICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/ aWQ9MTYwMDU3CisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAg ICAgKiBTY3JpcHRzL3dlYmtpdHB5L2JlbmNobWFya19ydW5uZXIvZGF0YS9wbGFucy9Ta2lwcGVk OiBBZGRlZC4KKyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2JlbmNobWFya19ydW5uZXIvcnVu X2JlbmNobWFyay5weToKKyAgICAgICAgKHN0YXJ0KToKKwogMjAxNi0wOC0wMiAgTmFuIFdhbmcg IDxuX3dhbmdAYXBwbGUuY29tPgogCiAgICAgICAgIEFYOiBTaW11bGF0ZWQgdG91Y2ggZXZlbnRz IGFyZSBub3Qgd29ya2luZyBvbiBpT1MKZGlmZiAtLWdpdCBhL1Rvb2xzL1NjcmlwdHMvd2Via2l0 cHkvYmVuY2htYXJrX3J1bm5lci9kYXRhL3BsYW5zL1NraXBwZWQgYi9Ub29scy9TY3JpcHRzL3dl YmtpdHB5L2JlbmNobWFya19ydW5uZXIvZGF0YS9wbGFucy9Ta2lwcGVkCm5ldyBmaWxlIG1vZGUg MTAwNjQ0CmluZGV4IDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAuLjcz ZjY0M2Y2NTJiMmVhMjVkYmRhNGE1OWNjOTdmNTA5ZjE4ZTRjN2MKLS0tIC9kZXYvbnVsbAorKysg Yi9Ub29scy9TY3JpcHRzL3dlYmtpdHB5L2JlbmNobWFya19ydW5uZXIvZGF0YS9wbGFucy9Ta2lw cGVkCkBAIC0wLDAgKzEsMiBAQAorIyBUaGlzIGZpbGUgY29udGFpbnMgdGhlIHBsYW5zIHRoYXQg c2hvdWxkIG5vdCBiZSBydW4gYXV0b21hdGljYWxseSB3aXRoIHRoZSAtLWFsbHBsYW5zIGNvbW1h bmQgbGluZSBzd2l0Y2guCitjb250ZW50LWFuaW1hdGlvbgpkaWZmIC0tZ2l0IGEvVG9vbHMvU2Ny aXB0cy93ZWJraXRweS9iZW5jaG1hcmtfcnVubmVyL3J1bl9iZW5jaG1hcmsucHkgYi9Ub29scy9T Y3JpcHRzL3dlYmtpdHB5L2JlbmNobWFya19ydW5uZXIvcnVuX2JlbmNobWFyay5weQppbmRleCA5 YzVlMDYyZGFhOTQ0OGZjZTVlZjNlODliZmM3YzNiMjhjYTAzNDlmLi44ODIyMzkyYzQ5N2Q1ODdm YTBmMjQ2NWM2Y2YwM2Y5OWYwNWFlOTFkIDEwMDc1NQotLS0gYS9Ub29scy9TY3JpcHRzL3dlYmtp dHB5L2JlbmNobWFya19ydW5uZXIvcnVuX2JlbmNobWFyay5weQorKysgYi9Ub29scy9TY3JpcHRz L3dlYmtpdHB5L2JlbmNobWFya19ydW5uZXIvcnVuX2JlbmNobWFyay5weQpAQCAtNDgsMTEgKzQ4 LDE4IEBAIGRlZiBzdGFydChhcmdzKToKICAgICAgICAgcmV0dXJuCiAgICAgaWYgYXJncy5hbGxw bGFuczoKICAgICAgICAgZmFpbGVkID0gW10KKyAgICAgICAgc2tpcHBlZCA9IFtdCiAgICAgICAg IHBsYW5kaXIgPSBvcy5wYXRoLmpvaW4ob3MucGF0aC5kaXJuYW1lKF9fZmlsZV9fKSwgJ2RhdGEv cGxhbnMnKQogICAgICAgICBwbGFubGlzdCA9IFtvcy5wYXRoLnNwbGl0ZXh0KGYpWzBdIGZvciBm IGluIG9zLmxpc3RkaXIocGxhbmRpcikgaWYgZi5lbmRzd2l0aCgnLnBsYW4nKV0KKyAgICAgICAg c2tpcHBlZGZpbGUgPSBvcy5wYXRoLmpvaW4ocGxhbmRpciwgJ1NraXBwZWQnKQogICAgICAgICBp ZiBub3QgcGxhbmxpc3Q6CiAgICAgICAgICAgICByYWlzZSBFeGNlcHRpb24oJ0NhbnQgZmluZCBh bnkgLnBsYW4gZmlsZSBpbiBkaXJlY3RvcnkgJXMnICUgcGxhbmRpcikKKyAgICAgICAgaWYgb3Mu cGF0aC5pc2ZpbGUoc2tpcHBlZGZpbGUpOgorICAgICAgICAgICAgc2tpcHBlZCA9IFtsaW5lLnN0 cmlwKCkgZm9yIGxpbmUgaW4gb3Blbihza2lwcGVkZmlsZSkgaWYgbm90IGxpbmUuc3RhcnRzd2l0 aCgnIycpIGFuZCBsZW4obGluZSkgPiAxXQogICAgICAgICBmb3IgcGxhbiBpbiBzb3J0ZWQocGxh bmxpc3QpOgorICAgICAgICAgICAgaWYgcGxhbiBpbiBza2lwcGVkOgorICAgICAgICAgICAgICAg IF9sb2cuaW5mbygnU2tpcHBpbmcgYmVuY2htYXJrIHBsYW46ICVzIGJlY2F1c2UgaXMgbGlzdGVk IG9uIHRoZSBTa2lwcGVkIGZpbGUnICUgcGxhbikKKyAgICAgICAgICAgICAgICBjb250aW51ZQog ICAgICAgICAgICAgX2xvZy5pbmZvKCdTdGFydGluZyBiZW5jaG1hcmsgcGxhbjogJXMnICUgcGxh bikKICAgICAgICAgICAgIHRyeToKICAgICAgICAgICAgICAgICBydW5uZXIgPSBCZW5jaG1hcmtS dW5uZXIocGxhbiwgYXJncy5sb2NhbENvcHksIGFyZ3MuY291bnRPdmVycmlkZSwgYXJncy5idWls ZERpciwgYXJncy5vdXRwdXQsIGFyZ3MucGxhdGZvcm0sIGFyZ3MuYnJvd3NlciwgYXJncy5zY2Fs ZV91bml0LCBhcmdzLmRldmljZV9pZCkK