Bug 154595 – [GTK] Run the run-benchmark script on the performance bot.

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Carlos Alberto Lopez Perez]

After bug 153993 is possible to use the run-benchmark script to execute the benchmarks with the WebKitGTK+ MiniBrowser

Now we would to implement a new step for the performance bots to run this tests automatically (running all plans available).

The idea is to only enable this for the GTK+ Performance Test bot for the moment. Other ports can follow if they wish.

Previous context: https://lists.webkit.org/pipermail/webkit-dev/2016-February/027948.html

[Carlos Alberto Lopez Perez]

Created attachment 272564 [details]
Patch

[WebKit Commit Bot]

Attachment 272564 [details] did not pass style-queue:


ERROR: Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:534:  [RunBenchmarkTest.assertResults] Undefined variable 'RunBenchmarkTests'  [pylint/E0602] [5]
Total errors found: 1 in 6 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[Carlos Alberto Lopez Perez]

(In reply to comment #2)
> Attachment 272564 [details] did not pass style-queue:
> 
> 
> ERROR:
> Tools/BuildSlaveSupport/build.webkit.org-config/mastercfg_unittest.py:534: 
> [RunBenchmarkTest.assertResults] Undefined variable 'RunBenchmarkTests' 
> [pylint/E0602] [5]
> Total errors found: 1 in 6 files
> 
> 
> If any of these errors are false positives, please file a bug against
> check-webkit-style.

This is a false positive.

The unit tests are executed as follows:

( cd Tools/BuildSlaveSupport/build.webkit.org-config/; ./mastercfg_unittest.py )

[Carlos Garcia Campos]

Comment on attachment 272564 [details]
Patch

Looks good, let's try it out! Please, watch the bots after landing this.

[Csaba Osztrogonác]

(In reply to comment #4)
> Comment on attachment 272564 [details]
> Patch
> 
> Looks good, let's try it out! Please, watch the bots after landing this.

You will need a buildmaster restart to see its effect.

[Carlos Alberto Lopez Perez]

Comment on attachment 272564 [details]
Patch

Clearing flags on attachment: 272564

Committed r197505: <http://trac.webkit.org/changeset/197505>

[Carlos Alberto Lopez Perez]

All reviewed patches have been landed.  Closing bug.

[Carlos Alberto Lopez Perez]

(In reply to comment #5)
> (In reply to comment #4)
> > Comment on attachment 272564 [details]
> > Patch
> > 
> > Looks good, let's try it out! Please, watch the bots after landing this.
> 
> You will need a buildmaster restart to see its effect.

yes, it seems.

I have sent a mail to lucas asking for a restart.

[Carlos Alberto Lopez Perez]

Lucas just restarted the master moments ago :)

I can see now the new step on the GTK+ perf bot. Let's see how it goes

[Carlos Alberto Lopez Perez]

Committed r197525: <http://trac.webkit.org/changeset/197525>

[Carlos Alberto Lopez Perez]

Committed r197615: <http://trac.webkit.org/changeset/197615>

[Csaba Osztrogonác]

Comment on attachment 272564 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=272564&action=review

> Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg:712
> +class RunBenchmarkTests(shell.Test):
> +    name = "benchmark-test"
> +    description = ["benchmark tests running"]
> +    descriptionDone = ["benchmark tests"]
> +    # Buildbot default timeout without output for a step is 1200.
> +    # The current maximum timeout for a benchmark plan is also 1200.
> +    # So raise the buildbot timeout to avoid aborting this whole step when a test timeouts.
> +    timeout = 1500
> +    command = ["python", "./Tools/Scripts/run-benchmark", "--allplans"]

Setting timeout here doesn't have any effect, see the buildbot logs:
https://build.webkit.org/builders/GTK%20Linux%2064-bit%20Release%20%28Perf%29/builds/831/steps/benchmark-test/logs/stdio

The timeout is still the default 1200 seconds.

You need a fix like this: https://trac.webkit.org/changeset/179043/webkit