186635 2018-06-14 16:07:27 -0700 [GTK][buildbot] Raise timeouts for the step benchmark-test on the GTK perf bot 2018-06-15 00:46:34 -0700 1 1 1 Unclassified WebKit WebKitGTK WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED P2 Normal --- 1 clopez clopez bugs-noreply calvaris clopez ews-watchlist gustavo lforschler mcatanzaro psaavedra ysuzuki oldest_to_newest 1433159 0 clopez 2018-06-14 16:07:27 -0700 The step benchmark-test on the GTK perf bot runs all the benchmark plans one by one (one after the other). Each one of this benchmarks plans have different timeouts. The problem is that buildbot itself will kill the whole step when no output is produced after a 1200 seconds timeout, but some of the plans for run-benchmark have higher timeouts than 1200... that means that when things go wrong (or the machine/built-product is slower than expected) the buildbot timeout will happen before the plan one. And when that happens, that means that the whole buildbot step will be stopped instead of having simply one test of the test set failing because its timeout has been reached. This has been an already an issue on the past, and we already fixed this by growing the buildbot timeout for this step on the past But it seems now some plans (like motionmark) have grown the plan timeout further, so we should adjust. 1433160 1 clopez 2018-06-14 16:10:36 -0700 Example of the issue: * Build: https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Perf%29/builds/3150 * Stdio log: https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Perf%29/builds/3150/steps/benchmark-test/logs/stdio 2018-06-14 00:22:07,450 - INFO - Copied the benchmark into: /tmp/tmpxeBC7R/motionmark patching file resources/runner/motionmark.js 2018-06-14 00:22:07,455 - INFO - Start the iteration 1 of 1 for current benchmark 2018-06-14 00:22:07,456 - INFO - Launching an http server 2018-06-14 00:22:07,460 - INFO - Start to fetching the port number of the http server 2018-06-14 00:22:07,460 - INFO - Server port is not found this time, retry after 0.500000 seconds 2018-06-14 00:22:07,964 - INFO - HTTP Server is serving at port: 36079 2018-06-14 00:22:07,981 - INFO - Executing: /home/slave/webkitgtk/gtk-linux-64-release-perf-tests/build/Tools/Scripts/run-minibrowser --gtk --geometry=1024x768 http://127.0.0.1:36079/motionmark/index.html command timed out: 1200 seconds without output running ['python', './Tools/Scripts/run-benchmark', '--allplans', '--browser', 'minibrowser-gtk'], attempting to kill process killed by signal 9 program finished with exit code -1 elapsedTime=5473.596568 Buildbot aborts at 1200 seconds without output.. but the motionmark timeout was 1800 according to https://svn.webkit.org/repository/webkit/trunk/Tools/Scripts/webkitpy/benchmark_runner/data/plans/motionmark.plan so the buildbot one happened before the plan one.. we need to avoid this 1433181 2 342779 clopez 2018-06-14 17:04:54 -0700 Created attachment 342779 Patch 1433182 3 ews-watchlist 2018-06-14 17:07:05 -0700 Attachment 342779 did not pass style-queue: ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/factories.py:203: [BuildAndPerfTestFactory.__init__] Undefined variable 'RunBenchmarkTests' [pylint/E0602] [5] ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/factories.py:213: [DownloadAndPerfTestFactory.__init__] Undefined variable 'RunBenchmarkTests' [pylint/E0602] [5] Total errors found: 2 in 2 files If any of these errors are false positives, please file a bug against check-webkit-style. 1433213 4 342779 mcatanzaro 2018-06-14 19:04:24 -0700 Comment on attachment 342779 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=342779&action=review > Tools/BuildSlaveSupport/build.webkit.org-config/steps.py:-831 > - # Buildbot default timeout without output for a step is 1200. > - # The current maximum timeout for a benchmark plan is also 1200. > - # So raise the buildbot timeout to avoid aborting this whole step when a test timeouts. > - timeout = 1500 Weird that this was added if it has been ignored this whole time... maybe it regressed at some point in buildbot...? 1433215 5 clopez 2018-06-14 19:09:04 -0700 (In reply to Michael Catanzaro from comment #4) > Comment on attachment 342779 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=342779&action=review > > > Tools/BuildSlaveSupport/build.webkit.org-config/steps.py:-831 > > - # Buildbot default timeout without output for a step is 1200. > > - # The current maximum timeout for a benchmark plan is also 1200. > > - # So raise the buildbot timeout to avoid aborting this whole step when a test timeouts. > > - timeout = 1500 > > Weird that this was added if it has been ignored this whole time... maybe it > regressed at some point in buildbot...? I think it never worked. I guess I just assumed it would, but it didn't ... Testing buildbot master changes is kind of difficult because its not easy to access to a test environment. Now its a bit easier thanks to the script run-buildbot-test.py that was added on r225336 ... I used that now to verify this will work. But that script was not available back then. 1433216 6 342779 clopez 2018-06-14 19:11:41 -0700 Comment on attachment 342779 Patch Clearing flags on attachment: 342779 Committed r232866: <https://trac.webkit.org/changeset/232866> 1433217 7 clopez 2018-06-14 19:11:45 -0700 All reviewed patches have been landed. Closing bug. 342779 2018-06-14 17:04:54 -0700 2018-06-14 19:11:41 -0700 Patch bug-186635-20180615020453.patch text/plain 3803 clopez U3VidmVyc2lvbiBSZXZpc2lvbjogMjMyODU5CmRpZmYgLS1naXQgYS9Ub29scy9DaGFuZ2VMb2cg Yi9Ub29scy9DaGFuZ2VMb2cKaW5kZXggMmU3NjI1NjI1ZjJjZWM5MTE3YWQ4ODUyNDQxMjYzZDIz MGRlODQ2Yy4uYTE5OGNmNThkMWJhOTcxMTNhMDg3MWEwMWYxNjYyZDBiMjVhN2MzMyAxMDA2NDQK LS0tIGEvVG9vbHMvQ2hhbmdlTG9nCisrKyBiL1Rvb2xzL0NoYW5nZUxvZwpAQCAtMSwzICsxLDMx IEBACisyMDE4LTA2LTE0ICBDYXJsb3MgQWxiZXJ0byBMb3BleiBQZXJleiAgPGNsb3BlekBpZ2Fs aWEuY29tPgorCisgICAgICAgIFtHVEtdW2J1aWxkYm90XSBSYWlzZSB0aW1lb3V0cyBmb3IgdGhl IHN0ZXAgYmVuY2htYXJrLXRlc3Qgb24gdGhlIEdUSyBwZXJmIGJvdAorICAgICAgICBodHRwczov L2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MTg2NjM1CisKKyAgICAgICAgUmV2aWV3 ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgVGhlIGN1cnJlbnQgbWF4aW11bSB0aW1l b3V0IGZvciBhIHJ1bi1iZW5jaG1hcmsgdGVzdCBpcyAxODAwIChNb3Rpb25tYXJrIHBsYW4pLAor ICAgICAgICBidXQgdGhlIGJ1aWxkYm90IGRlZmF1bHQgdGltZW91dCBmb3IgdGhlIHdob2xlIHN0 ZXAgd2l0aG91dCB0ZXh0IG91dHB1dCBpcyAxMjAwLgorICAgICAgICBUaGF0IG1lYW5zIHRoYXQg YSB0aW1lb3V0IG9uIGEgc2luZ2xlIHRlc3QgY2FuIGNhdXNlIGEgd2hvbGUgc3RlcCBhYm9ydCBi ZWNhdXNlCisgICAgICAgIHRoZSBidWlsZGJvdCBvbmUgd2lsbCB0cmlnZ2VyIGZpcnN0LgorCisg ICAgICAgIFNldCB0aGUgYmVuY2htYXJrLXRlc3Qgc3RlcCB0aW1lb3V0IHRvIDIwMDAgdG8gYXZv aWQgdGhpcy4KKworICAgICAgICBXaGVuIHRoaXMgYmVuY2htYXJrLXRlc3Qgd2FzIGFkZGVkIGlu IHIxOTc1MDUgaXQgd2FzIGFscmVhZHkgYWRkZWQgd2l0aCBhIGhpZ2hlcgorICAgICAgICB0aW1l b3V0IHRoYW4gZGVmYXVsdCAoMTUwMCkgZHVlIHRvIHRoZSBhYm92ZSBwcm9ibGVtLCBidXQgaXQg c2VlbXMgdGhpcyBoYXMgYmVlbgorICAgICAgICBpZ25vcmVkIHNvIGZhciBiZWNhdXNlIGJ1aWxk Ym90IHdvbid0IGFjY2VwdCB0aGUgdGltZW91dCB0byBiZSBzZXQgb24gdGhlIGNsYXNzCisgICAg ICAgIGRlZmluaXRpb24gKG5vdCBldmVuIG9uIHRoZSBjbGFzcyBjb25zdHJ1Y3RvcikuCisgICAg ICAgIFRoZSBuZXcgdGltZW91dCBoYXMgdG8gYmUgZGVmaW5lZCBpbiB0aGUgYWRkU3RlcCBjYWxs IGl0c2VsZi4KKyAgICAgICAgSSBoYXZlIGRvdWJsZS1jaGVja2VkIHRoYXQgYnkgdGVzdGluZyBs b2NhbGx5IHdpdGggdGhlIHNjcmlwdCBydW4tYnVpbGRib3QtdGVzdC5weQorICAgICAgICBBbmQg dGhpcyB1cHN0cmVhbSBpc3N1ZSBjb25maXJtcyB0aGF0IGh0dHBzOi8vZ2l0aHViLmNvbS9idWls ZGJvdC9idWlsZGJvdC9pc3N1ZXMvMzg4NworCisgICAgICAgICogQnVpbGRTbGF2ZVN1cHBvcnQv YnVpbGQud2Via2l0Lm9yZy1jb25maWcvZmFjdG9yaWVzLnB5OgorICAgICAgICAoQnVpbGRBbmRQ ZXJmVGVzdEZhY3RvcnkuX19pbml0X18pOgorICAgICAgICAoRG93bmxvYWRBbmRQZXJmVGVzdEZh Y3RvcnkuX19pbml0X18pOgorICAgICAgICAqIEJ1aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtp dC5vcmctY29uZmlnL3N0ZXBzLnB5OgorICAgICAgICAoUnVuQmVuY2htYXJrVGVzdHMpOgorCiAy MDE4LTA2LTE0ICBNYXR0IExld2lzICA8amxld2lzM0BhcHBsZS5jb20+CiAKICAgICAgICAgVW5y ZXZpZXdlZCwgcm9sbGluZyBvdXQgcjIzMjgyMy4KZGlmZiAtLWdpdCBhL1Rvb2xzL0J1aWxkU2xh dmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL2ZhY3Rvcmllcy5weSBiL1Rvb2xzL0J1 aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL2ZhY3Rvcmllcy5weQppbmRl eCBmOGI3MzNlMzE2NjcyZjk4YzIxM2E2ZDg2NGJjYjY3ZTU0ZjBjNWE5Li5mOTk2MjViYzdhMWNl NjczZTA1Mjc3MzdmYzk5NDU1ZTBlZTUyMzE1IDEwMDY0NAotLS0gYS9Ub29scy9CdWlsZFNsYXZl U3VwcG9ydC9idWlsZC53ZWJraXQub3JnLWNvbmZpZy9mYWN0b3JpZXMucHkKKysrIGIvVG9vbHMv QnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvZmFjdG9yaWVzLnB5CkBA IC0yMDAsNyArMjAwLDcgQEAgY2xhc3MgQnVpbGRBbmRQZXJmVGVzdEZhY3RvcnkoRmFjdG9yeSk6 CiAgICAgICAgIHNlbGYuYWRkU3RlcChDb21waWxlV2ViS2l0KCkpCiAgICAgICAgIHNlbGYuYWRk U3RlcChSdW5BbmRVcGxvYWRQZXJmVGVzdHMoKSkKICAgICAgICAgaWYgcGxhdGZvcm0gPT0gImd0 ayI6Ci0gICAgICAgICAgICBzZWxmLmFkZFN0ZXAoUnVuQmVuY2htYXJrVGVzdHMoKSkKKyAgICAg ICAgICAgIHNlbGYuYWRkU3RlcChSdW5CZW5jaG1hcmtUZXN0cyh0aW1lb3V0PTIwMDApKQogCiAK IGNsYXNzIERvd25sb2FkQW5kUGVyZlRlc3RGYWN0b3J5KEZhY3RvcnkpOgpAQCAtMjEwLDQgKzIx MCw0IEBAIGNsYXNzIERvd25sb2FkQW5kUGVyZlRlc3RGYWN0b3J5KEZhY3RvcnkpOgogICAgICAg ICBzZWxmLmFkZFN0ZXAoRXh0cmFjdEJ1aWx0UHJvZHVjdCgpKQogICAgICAgICBzZWxmLmFkZFN0 ZXAoUnVuQW5kVXBsb2FkUGVyZlRlc3RzKCkpCiAgICAgICAgIGlmIHBsYXRmb3JtID09ICJndGsi OgotICAgICAgICAgICAgc2VsZi5hZGRTdGVwKFJ1bkJlbmNobWFya1Rlc3RzKCkpCisgICAgICAg ICAgICBzZWxmLmFkZFN0ZXAoUnVuQmVuY2htYXJrVGVzdHModGltZW91dD0yMDAwKSkKZGlmZiAt LWdpdCBhL1Rvb2xzL0J1aWxkU2xhdmVTdXBwb3J0L2J1aWxkLndlYmtpdC5vcmctY29uZmlnL3N0 ZXBzLnB5IGIvVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcv c3RlcHMucHkKaW5kZXggMTlkNWRmNzFiNjVmYjQyMWY1NzJlYTc2OThjY2Y4YTc4MzM1MzcwMC4u Y2FkOWI5YmI1MTkxYTJhM2YyZDczYjM5MDM3ZmJiNjY2YTg4MWM3NiAxMDA2NDQKLS0tIGEvVG9v bHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvc3RlcHMucHkKKysr IGIvVG9vbHMvQnVpbGRTbGF2ZVN1cHBvcnQvYnVpbGQud2Via2l0Lm9yZy1jb25maWcvc3RlcHMu cHkKQEAgLTgyNSwxMCArODI1LDYgQEAgY2xhc3MgUnVuQmVuY2htYXJrVGVzdHMoc2hlbGwuVGVz dCk6CiAgICAgbmFtZSA9ICJiZW5jaG1hcmstdGVzdCIKICAgICBkZXNjcmlwdGlvbiA9IFsiYmVu Y2htYXJrIHRlc3RzIHJ1bm5pbmciXQogICAgIGRlc2NyaXB0aW9uRG9uZSA9IFsiYmVuY2htYXJr IHRlc3RzIl0KLSAgICAjIEJ1aWxkYm90IGRlZmF1bHQgdGltZW91dCB3aXRob3V0IG91dHB1dCBm b3IgYSBzdGVwIGlzIDEyMDAuCi0gICAgIyBUaGUgY3VycmVudCBtYXhpbXVtIHRpbWVvdXQgZm9y IGEgYmVuY2htYXJrIHBsYW4gaXMgYWxzbyAxMjAwLgotICAgICMgU28gcmFpc2UgdGhlIGJ1aWxk Ym90IHRpbWVvdXQgdG8gYXZvaWQgYWJvcnRpbmcgdGhpcyB3aG9sZSBzdGVwIHdoZW4gYSB0ZXN0 IHRpbWVvdXRzLgotICAgIHRpbWVvdXQgPSAxNTAwCiAgICAgY29tbWFuZCA9IFsicHl0aG9uIiwg Ii4vVG9vbHMvU2NyaXB0cy9icm93c2VycGVyZmRhc2gtYmVuY2htYXJrIiwgIi0tYWxscGxhbnMi LAogICAgICAgICAgICAgICAgIi0tY29uZmlnLWZpbGUiLCAiLi4vLi4vYnJvd3NlcnBlcmZkYXNo LWJlbmNobWFyay1jb25maWcudHh0IiwKICAgICAgICAgICAgICAgICItLWJyb3dzZXItdmVyc2lv biIsIFdpdGhQcm9wZXJ0aWVzKCJyJShnb3RfcmV2aXNpb24pcyIpXQo=