Bug 154595 - [GTK] Run the run-benchmark script on the performance bot.
Summary: [GTK] Run the run-benchmark script on the performance bot.
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Carlos Alberto Lopez Perez
URL:
Keywords:
Depends on: 153993
Blocks:
  Show dependency treegraph
 
Reported: 2016-02-23 10:57 PST by Carlos Alberto Lopez Perez
Modified: 2017-07-31 14:46 PDT (History)
9 users (show)

See Also:


Attachments
Patch (12.27 KB, patch)
2016-03-01 08:35 PST, Carlos Alberto Lopez Perez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Carlos Alberto Lopez Perez 2016-02-23 10:57:47 PST
After bug 153993 is possible to use the run-benchmark script to execute the benchmarks with the WebKitGTK+ MiniBrowser

Now we would to implement a new step for the performance bots to run this tests automatically (running all plans available).

The idea is to only enable this for the GTK+ Performance Test bot for the moment. Other ports can follow if they wish.

Previous context: https://lists.webkit.org/pipermail/webkit-dev/2016-February/027948.html
Comment 1 Carlos Alberto Lopez Perez 2016-03-01 08:35:17 PST
Created attachment 272564 [details]
Patch
Comment 2 WebKit Commit Bot 2016-03-01 08:37:07 PST
Attachment 272564 [details] did not pass style-queue:


ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:534:  [RunBenchmarkTest.assertResults] Undefined variable 'RunBenchmarkTests'  [pylint/E0602] [5]
Total errors found: 1 in 6 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 3 Carlos Alberto Lopez Perez 2016-03-01 08:39:39 PST
(In reply to comment #2)
> Attachment 272564 [details] did not pass style-queue:
> 
> 
> ERROR:
> Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:534: 
> [RunBenchmarkTest.assertResults] Undefined variable 'RunBenchmarkTests' 
> [pylint/E0602] [5]
> Total errors found: 1 in 6 files
> 
> 
> If any of these errors are false positives, please file a bug against
> check-webkit-style.

This is a false positive.

The unit tests are executed as follows:

( cd Tools/BuildSlaveSupport/build.webkit.org-config/; ./mastercfg_unittest.py )
Comment 4 Carlos Garcia Campos 2016-03-03 08:10:36 PST
Comment on attachment 272564 [details]
Patch

Looks good, let's try it out! Please, watch the bots after landing this.
Comment 5 Csaba Osztrogonác 2016-03-03 08:20:17 PST
(In reply to comment #4)
> Comment on attachment 272564 [details]
> Patch
> 
> Looks good, let's try it out! Please, watch the bots after landing this.

You will need a buildmaster restart to see its effect.
Comment 6 Carlos Alberto Lopez Perez 2016-03-03 08:40:15 PST
Comment on attachment 272564 [details]
Patch

Clearing flags on attachment: 272564

Committed r197505: <http://trac.webkit.org/changeset/197505>
Comment 7 Carlos Alberto Lopez Perez 2016-03-03 08:40:22 PST
All reviewed patches have been landed.  Closing bug.
Comment 8 Carlos Alberto Lopez Perez 2016-03-03 10:26:46 PST
(In reply to comment #5)
> (In reply to comment #4)
> > Comment on attachment 272564 [details]
> > Patch
> > 
> > Looks good, let's try it out! Please, watch the bots after landing this.
> 
> You will need a buildmaster restart to see its effect.

yes, it seems.

I have sent a mail to lucas asking for a restart.
Comment 9 Carlos Alberto Lopez Perez 2016-03-03 10:45:27 PST
Lucas just restarted the master moments ago :)

I can see now the new step on the GTK+ perf bot. Let's see how it goes
Comment 10 Carlos Alberto Lopez Perez 2016-03-03 15:57:11 PST
Committed r197525: <http://trac.webkit.org/changeset/197525>
Comment 11 Carlos Alberto Lopez Perez 2016-03-05 11:44:36 PST
Committed r197615: <http://trac.webkit.org/changeset/197615>
Comment 12 Csaba Osztrogonác 2017-07-31 14:46:33 PDT
Comment on attachment 272564 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=272564&action=review

> Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:712
> +class RunBenchmarkTests(shell.Test):
> +    name = "benchmark-test"
> +    description = ["benchmark tests running"]
> +    descriptionDone = ["benchmark tests"]
> +    # Buildbot default timeout without output for a step is 1200.
> +    # The current maximum timeout for a benchmark plan is also 1200.
> +    # So raise the buildbot timeout to avoid aborting this whole step when a test timeouts.
> +    timeout = 1500
> +    command = ["python", "./Tools/Scripts/run-benchmark", "--allplans"]

Setting timeout here doesn't have any effect, see the buildbot logs:
https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Perf%29/builds/831/steps/benchmark-test/logs/stdio

The timeout is still the default 1200 seconds.

You need a fix like this: https://trac.webkit.org/changeset/179043/webkit